[PATCH v4 0/3] raw/ntb: add AMD NTB support
Raghavendra Ningoji
raghavendra.ningoji at amd.com
Mon Oct 26 09:55:12 CET 2026
This series adds support for the Non-Transparent Bridge (NTB) endpoints
integrated in AMD EPYC Embedded "Turin", "Genoa" and "Siena" processors
to the raw/ntb driver.
The common NTB code validates the vendor ops once at probe time, and the
vendor abstraction is built around a small set of mandatory ops instead
of optional hooks with built-in fallbacks:
- Patch 1 is a preparation commit that enforces the vendor ops once, at
probe time, and removes the per-call NULL checks that are scattered
throughout the common code.
- Patch 2 generalizes the framework for multiple vendors. The link
bring-up/tear-down is now expressed through mandatory ops
(dev_handshake, peer_mw_validate, read_peer_config, link_cleanup); the
former built-in scratchpad logic moved into the Intel driver. The
mw_set_trans op is gone (it is now a vendor-internal concern of the
handshake), and there is a single common interrupt handler with just
one optional op (handle_hw_event) for vendor-specific hardware events.
- Patch 3 adds the AMD hardware driver, its documentation, release note
and MAINTAINERS entry.
A few AMD-specific hardware details worth noting for reviewers:
- Primary/secondary topology. One endpoint enumerates as the primary
(device ID 0x14c0) and the other as the secondary (0x14c3), rather
than the Intel back-to-back model.
- Packed scratchpad handshake. The hardware exposes a single shared
16-register scratchpad bank split into two disjoint 8-register sets
(one per side), so the driver uses a packed handshake layout that
fits in 8 registers, plugged in through the dev_handshake,
peer_mw_validate and read_peer_config ops.
- Outbound translation window. AMD NTB uses an outbound translation
window: a write to a BARxx memory-window offset is forwarded to
(xlat_base | offset) in the peer's memory rather than
(xlat_base + offset). The translation base must therefore be aligned
to a power of two >= the window length so that no offset bit collides
with a set bit in the base. This is reported to applications through
the rte_pmd_ntb_get_mem_align() API and honoured by the ntb example
when reserving the memory window memzone.
- Secondary-side link status. On the secondary side the device's own
PCIe link status does not reflect the true inter-host link, so the
speed and width are read from the upstream switch port via sysfs.
Tested on a two-node setup (primary + secondary EPYC Embedded hosts)
with the dpdk-ntb example: the link comes up at the expected speed and
width on both sides, and file transfers are byte-correct end to end for
both small and large files.
Changes since v3:
- Simplified the probe-time vendor ops check (patch 1) to a plain
per-member macro instead of an offsetof() table.
- Removed the mw_size_align field from struct ntb_dev_info entirely
(patch 2): applications must now use rte_pmd_ntb_get_mem_align(). This
is documented in the API changes section of the release notes.
- Removed the unused pmd_private field from struct ntb_hw and the whole
struct amd_ntb_hw (patch 3): self_mmio is just ntb->hw_addr, peer_mmio
becomes AMD_PEER_MMIO(ntb), the scratchpad offsets become an
AMD_SPAD_BASE(ntb, peer) macro, and int_mask/peer_status/ctl_status
were unused.
- Sorted AMD alphabetically before Intel in meson.build, the ntb_hw
header include, the PCI id mapping table and usertools/dpdk-devbind.py.
- Moved the primary/secondary topology definitions (NTB_TOPO_PRI/SEC)
from patch 2 to patch 3 where they are first used.
- Updated the Limitation section of doc/guides/rawdevs/ntb.rst to list
the supported AMD processors.
Changes since v2:
- The series is freshly rebased on main; the release note and
dpdk-devbind.py now apply cleanly and CI should pass.
- New preparation patch (patch 1): the vendor ops are validated once at
probe time and all the runtime ntb_ops NULL checks in the common code
are removed.
- Reworked the vendor abstraction (patch 2): dev_handshake,
peer_mw_validate, read_peer_config and link_cleanup are now mandatory
ops provided by both the Intel and AMD drivers. The built-in
scratchpad handshake/validate/cleanup/peer-config logic that used to
live in the common code moved into the Intel driver.
- Dropped the mw_set_trans op: it is now a vendor-internal detail of the
handshake.
- Dropped the vendor interrupt_handler op: there is now a single common
interrupt handler, with handle_hw_event as the only optional op, used
by AMD for its SMU/power-management events.
Changes since v1:
- Replaced the mw_addr_align field in struct ntb_dev_info with a
mem_align op and the rte_pmd_ntb_get_mem_align() API, so the
base-address alignment computation lives in the driver and the
application just queries it (Bruce Richardson's review).
- Squashed the documentation, release note and MAINTAINERS update into
the AMD driver patch, and documented the alignment API in the guide.
Raghavendra Ningoji (3):
raw/ntb: validate vendor ops at probe
raw/ntb: generalize framework for multiple vendors
raw/ntb: add AMD NTB support
MAINTAINERS | 2 +
doc/guides/rawdevs/ntb.rst | 31 +-
doc/guides/rel_notes/release_26_11.rst | 13 +
drivers/raw/ntb/meson.build | 7 +-
drivers/raw/ntb/ntb.c | 310 ++++--------
drivers/raw/ntb/ntb.h | 24 +-
drivers/raw/ntb/ntb_hw_amd.c | 658 +++++++++++++++++++++++++
drivers/raw/ntb/ntb_hw_amd.h | 111 +++++
drivers/raw/ntb/ntb_hw_intel.c | 165 ++++++-
drivers/raw/ntb/rte_pmd_ntb.h | 28 +-
examples/ntb/ntb_fwd.c | 6 +-
usertools/dpdk-devbind.py | 1 +
12 files changed, 1124 insertions(+), 232 deletions(-)
create mode 100644 drivers/raw/ntb/ntb_hw_amd.c
create mode 100644 drivers/raw/ntb/ntb_hw_amd.h
--
2.34.1
More information about the dev
mailing list