[PATCH] net/stmmac: add PMD for Synopsys DWC EQoS (GMAC4)
Kiran Patchala
kiran.patchala at oss.qualcomm.com
Sat Nov 14 15:55:39 CET 2026
From: Kiran Patchala <kiran.patchala at oss.qualcomm.com>
Add a new poll-mode driver for the Synopsys DesignWare Cores Enhanced
Quality-of-Service (DWC EQoS / GMAC4, "stmmac") Ethernet MAC present.
The device is enumerated by the platform bus and bound to vfio-platform.
The kernel recovery driver handles clock and PHY reset before the PMD
takes over.
Driver features:
- One TX queue and one RX queue on DMA channel 0; single-segment mbufs
only; no TSO, PTP, or secondary process support.
- Non-coherent DMA: DC CVAC/CIVAC cache maintenance on arm64 for
descriptor and payload lines; no-op stubs on other architectures.
- Ethernet ops: configure, start, stop, close, link_update,
promiscuous enable/disable, mac_addr_set, stats_get, stats_reset,
dev_infos_get, rx/tx_queue_setup, rxq/txq_info_get.
Limitations:
- Single DMA channel (queue 0 only); no multi-queue support.
- Single-segment mbufs; scatter-gather, TSO and jumbo frames are not
supported.
- Tested on arm64 only.
Testing:
dpdk-testpmd: txonly / rxonly / io forward at 1 Gb/s link
Co-developed-by: Mohan Degalwade <mohan.degalwade at oss.qualcomm.com>
Signed-off-by: Mohan Degalwade <mohan.degalwade at oss.qualcomm.com>
Signed-off-by: Kiran Patchala <kiran.patchala at oss.qualcomm.com>
---
doc/guides/nics/index.rst | 1 +
doc/guides/nics/stmmac.rst | 304 +++++++++
drivers/net/meson.build | 1 +
drivers/net/stmmac/meson.build | 15 +
drivers/net/stmmac/stmmac.h | 95 +++
drivers/net/stmmac/stmmac_ethdev.c | 1007 ++++++++++++++++++++++++++++
drivers/net/stmmac/stmmac_logs.h | 25 +
drivers/net/stmmac/stmmac_regs.h | 302 +++++++++
drivers/net/stmmac/stmmac_rxtx.c | 416 ++++++++++++
9 files changed, 2166 insertions(+)
create mode 100644 doc/guides/nics/stmmac.rst
create mode 100644 drivers/net/stmmac/meson.build
create mode 100644 drivers/net/stmmac/stmmac.h
create mode 100644 drivers/net/stmmac/stmmac_ethdev.c
create mode 100644 drivers/net/stmmac/stmmac_logs.h
create mode 100644 drivers/net/stmmac/stmmac_regs.h
create mode 100644 drivers/net/stmmac/stmmac_rxtx.c
diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
index e20be478f8..b5ca9bddb5 100644
--- a/doc/guides/nics/index.rst
+++ b/doc/guides/nics/index.rst
@@ -68,6 +68,7 @@ Network Interface Controller Drivers
rnp
sfc_efx
softnic
+ stmmac
sxe2
tap
thunderx
diff --git a/doc/guides/nics/stmmac.rst b/doc/guides/nics/stmmac.rst
new file mode 100644
index 0000000000..3cb9cc5af0
--- /dev/null
+++ b/doc/guides/nics/stmmac.rst
@@ -0,0 +1,304 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright 2026 Qualcomm Technologies, Inc.
+
+Stmmac Poll Mode Driver
+=======================
+
+The stmmac PMD (**librte_net_stmmac**) provides poll mode driver
+support for the Synopsys DesignWare Cores Enhanced Quality-of-Service
+(DWC EQoS / GMAC4) Ethernet MAC present on Qualcomm SoCs such as the
+IQ9075 (QCS9075) found on the Shikra EVK platform.
+
+More information on the Qualcomm Shikra platform can be found at
+`Qualcomm Developer Network <https://www.qualcomm.com/developer>`_.
+
+Synopsys DWC EQoS (GMAC4, "stmmac")
+-------------------------------------
+
+This section provides an overview of the Synopsys DWC EQoS MAC
+architecture and how it is integrated into DPDK.
+
+Contents summary
+
+- DWC EQoS overview
+- Driver architecture overview
+- Device configuration and devargs
+
+.. _stmmac_overview:
+
+DWC EQoS Overview
+~~~~~~~~~~~~~~~~~~
+
+The Synopsys DesignWare Cores Enhanced Quality-of-Service (DWC EQoS)
+is a fully integrated 10/100/1000 Mb/s Ethernet MAC core. The "stmmac"
+name follows the Linux kernel driver family
+(``drivers/net/ethernet/stmicro/stmmac``).
+
+The MAC core revision used on the Qualcomm Shikra EVK is GMAC4 v5.30a.
+
+DWC EQoS includes:
+
+- MAC core (GMAC4): frame transmission, reception, MDIO management,
+ address filtering, and hardware checksum offload
+- MTL (MAC Transaction Layer): configurable TX/RX FIFOs with
+ store-and-forward mode; FIFO sizes are read from HW_FEATURE1 at probe
+- DMA engine: AXI system bus with fixed burst, mixed burst, AAL and
+ EAME (>4 GiB IOVA support); per-channel burst (PBL 32, PBLx8, OSP)
+- RGMII IO macro: Qualcomm-specific SerDes block for RGMII-ID timing
+
+The Network and packet I/O component:
+
+- One DMA channel (channel 0) with one TX queue and one RX queue.
+ The PMD operates DMA channel 0 exclusively.
+- The DMA is not cache-coherent on the Shikra platform. The PMD
+ performs explicit DC CVAC/CIVAC cache maintenance on arm64 for both
+ descriptor lines and packet payload. No-op stubs are provided for
+ other architectures.
+
+Stmmac DPDK - Poll Mode Driver Overview
+-----------------------------------------
+
+This section provides an overview of the stmmac PMD for DPDK.
+
+The PMD is probed by the DPDK platform bus and uses ``vfio-platform``
+for MMIO access and DMA mapping. The kernel recovery driver must be
+loaded first to perform clock enable and PHY reset; the PMD then takes
+over and must not be unloaded while the PMD is running.
+
+.. code-block:: console
+
+ +-------------------+
+ | DPDK Application |
+ +--------+----------+
+ |
+ +--------+----------+
+ | stmmac PMD |
+ | (librte_net_stmmac)|
+ +----+----------+---+
+ | |
+ +---------+--+ +---+---------+
+ | TX Burst | | RX Burst |
+ | (DMA ch 0) | | (DMA ch 0) |
+ +---------+--+ +---+---------+
+ | |
+ ===================|==========|================ HARDWARE
+ +----+----------+---+
+ | DWC EQoS (GMAC4) |
+ | MAC / MTL / DMA |
+ +--------+----------+
+ |
+ +--------+----------+
+ | RGMII IO Macro |
+ +--------+----------+
+ |
+ +--------+----------+
+ | DP83867 PHY |
+ +-------------------+
+
+Platform Bus Driver
+~~~~~~~~~~~~~~~~~~~
+
+The stmmac PMD registers as an ``rte_platform_driver``. The platform bus
+scans the kernel sysfs tree for a device whose driver alias matches
+``vfio-platform``. Two MMIO resources are consumed:
+
+- ``resource[0]``: MAC/MTL/DMA register block (``stmmaceth`` in DT)
+- ``resource[1]``: RGMII IO macro register block (``rgmii`` in DT)
+
+The kernel ``vfio-platform`` driver must be bound to the stmmac device
+before launching a DPDK application.
+
+Stmmac NIC Driver (PMD)
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The stmmac PMD implements the standard DPDK ethdev API for the DWC EQoS
+MAC. It directly programs the MAC, MTL and DMA register blocks over
+the MMIO mapping provided by vfio-platform.
+
+- On probe, the PMD reads devargs, maps MMIO resources, sets RGMII IO
+ macro timing, programs DP83867 RGMII internal delays via MMD indirect
+ access, and registers the ethdev.
+- On ``dev_start``, the PMD enables the RGMII functional clock,
+ performs a DMA soft reset, programs AXI sysbus and per-channel burst
+ settings, sets up TX and RX descriptor rings, initialises the MAC
+ and MTL, and starts DMA.
+- Link state is polled from the DP83867 PHY using IEEE 802.3 clause-22
+ MDIO. BMSR is double-read to clear the latched link bit; live speed
+ and duplex come from PHYSTS.
+
+Features
+^^^^^^^^
+
+Features of the stmmac PMD are:
+
+- Single TX queue and single RX queue on DMA channel 0
+- IPv4, TCP and UDP hardware checksum offload (TX and RX)
+- Promiscuous mode
+- MAC address set
+- Link state update via DP83867 PHY MDIO (clause-22)
+- Per-queue packet and byte counters (stats_get / stats_reset)
+- Configurable ring sizes (64..1024 descriptors, multiples of 4)
+- Non-coherent DMA support (arm64 DC CVAC/CIVAC cache maintenance)
+- 36-bit DMA addressing (EAME enabled, mbufs above 4 GiB supported)
+
+Allowing & Blocking
+-------------------
+
+To block a stmmac device from being used by DPDK, the following EAL
+argument can be used:
+
+.. code-block:: console
+
+ <dpdk app> <EAL args> -b "platform:<device_name>" -- ...
+ e.g. -b "platform:a000000.ethernet"
+
+Supported Platforms
+-------------------
+
+- Qualcomm IQ9075 (QCS9075) — Shikra EVK (arm64)
+
+Prerequisites
+-------------
+
+- Follow the DPDK :doc:`/linux_gsg/index` to set up the basic DPDK
+ environment.
+- An arm64 cross-compilation toolchain is required to build for the
+ Shikra EVK target from an x86 host.
+- The kernel ``vfio-platform`` and ``vfio_iommu_type1`` modules must be
+ loaded on the target before launching a DPDK application.
+- The Qualcomm stmmac recovery kernel driver must be loaded first to
+ enable clocks and reset the PHY. The PMD does not perform these steps.
+
+.. note::
+
+ The stmmac PMD is supported on Linux only. It requires the platform
+ bus (``rte_bus_platform``) and ``vfio-platform`` kernel support.
+
+Configuration
+-------------
+
+Devargs
+~~~~~~~
+
+The stmmac PMD accepts the following device arguments:
+
+.. list-table:: stmmac devargs
+ :header-rows: 1
+ :widths: 20 10 70
+
+ * - Parameter
+ - Default
+ - Description
+ * - ``has_emac_ge_3``
+ - ``1``
+ - DMA/MTL channel stride override. Set to ``1`` for EMAC >= 3.0
+ platforms (Shikra IQ9075), which use a 0x1000 channel stride
+ (base 0x8100 for DMA, 0x8000 for MTL). Set to ``0`` for
+ generic dwmac4 layout (base 0x1100, stride 0x80).
+ * - ``mdio_addr``
+ - ``7``
+ - PHY MDIO address. The DP83867 on the Shikra EVK is at address 7.
+ Override if the PHY is at a different address.
+
+Example devarg usage:
+
+.. code-block:: console
+
+ --dev "platform:a000000.ethernet,has_emac_ge_3=1,mdio_addr=7"
+
+Driver Compilation and Testing
+-------------------------------
+
+Refer to the document :doc:`build_and_test` for details.
+
+#. Cross-compile for arm64 (from an x86 host):
+
+ .. code-block:: console
+
+ meson setup build-arm64 \
+ --cross-file config/arm/arm64_armv8_linux_gcc \
+ -Dplatform=generic
+ ninja -C build-arm64
+
+#. Bind the device to ``vfio-platform`` on the target:
+
+ .. code-block:: console
+
+ # Load required kernel modules
+ modprobe vfio
+ modprobe vfio-platform
+ modprobe vfio_iommu_type1
+
+ # Identify the device name from sysfs
+ ls /sys/bus/platform/devices/ | grep ethernet
+ # e.g. a000000.ethernet
+
+ # Bind to vfio-platform
+ echo vfio-platform > /sys/bus/platform/devices/a000000.ethernet/driver_override
+ echo a000000.ethernet > /sys/bus/platform/drivers/vfio-platform/bind
+
+ # Verify the bind succeeded
+ ls /sys/bus/platform/devices/a000000.ethernet/driver
+ # Should show: .../vfio-platform
+
+#. Running testpmd on the Shikra EVK:
+
+ .. code-block:: console
+
+ ./<build_dir>/app/dpdk-testpmd \
+ -l 0-3 -n 1 --vdev="" \
+ -- -i --portmask=0x1 --nb-cores=1 --no-flush-rx
+ .....
+ EAL: Registered [platform] bus.
+ EAL: Detected 8 lcore(s)
+ .....
+ Configuring Port 0 (socket 0)
+ Port 0: 02:02:02:02:02:02
+ .....
+ Checking link statuses...
+ Port 0 Link Up - speed 1000 Mbps - full-duplex
+ Done
+ testpmd>
+
+ * Use devarg ``has_emac_ge_3=0`` for non-Shikra platforms with the
+ generic dwmac4 DMA channel layout.
+ * Use devarg ``mdio_addr=<n>`` if the PHY MDIO address differs from
+ the Shikra default of 7.
+
+Limitations
+-----------
+
+Single Queue
+~~~~~~~~~~~~
+
+The stmmac PMD supports one TX queue and one RX queue only (DMA
+channel 0). Multi-queue configurations are not supported in this
+release.
+
+Single-segment Mbufs
+~~~~~~~~~~~~~~~~~~~~~
+
+Only single-segment mbufs are supported. Scatter-gather, TSO, jumbo
+frames, and multi-segment packet transmission are not supported. The
+application must ensure that ``nb_segs == 1`` for every transmitted
+mbuf.
+
+No Interrupt Support
+~~~~~~~~~~~~~~~~~~~~~
+
+The PMD is polling-only. No interrupt-driven TX completion or RX
+notification is supported. The platform bus does not expose an interrupt
+to the PMD.
+
+No Secondary Process Support
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Multi-process operation where I/O is performed by a secondary DPDK
+process is not supported.
+
+Platform Requirement
+~~~~~~~~~~~~~~~~~~~~~
+
+The stmmac PMD is supported on Linux on arm64 only. It has been tested
+on the Qualcomm Shikra EVK (IQ9075 / QCS9075 SoC). Operation on other
+platforms is not guaranteed.
diff --git a/drivers/net/meson.build b/drivers/net/meson.build
index 4e8ccb945f..4d8549c4f9 100644
--- a/drivers/net/meson.build
+++ b/drivers/net/meson.build
@@ -58,6 +58,7 @@ drivers = [
'rnp',
'sfc',
'softnic',
+ 'stmmac',
'sxe2',
'tap',
'thunderx',
diff --git a/drivers/net/stmmac/meson.build b/drivers/net/stmmac/meson.build
new file mode 100644
index 0000000000..fb3ed0e162
--- /dev/null
+++ b/drivers/net/stmmac/meson.build
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2026 Qualcomm Technologies, Inc.
+
+if not is_linux
+ build = false
+ reason = 'only supported on linux'
+ subdir_done()
+endif
+
+deps += ['bus_platform']
+
+sources = files(
+ 'stmmac_ethdev.c',
+ 'stmmac_rxtx.c',
+)
diff --git a/drivers/net/stmmac/stmmac.h b/drivers/net/stmmac/stmmac.h
new file mode 100644
index 0000000000..ccfc9309c3
--- /dev/null
+++ b/drivers/net/stmmac/stmmac.h
@@ -0,0 +1,95 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2026 Qualcomm Technologies, Inc.
+ */
+
+#ifndef _STMMAC_H_
+#define _STMMAC_H_
+
+#include <stdint.h>
+
+#include <rte_ether.h>
+#include <rte_ethdev.h>
+#include <rte_mempool.h>
+#include <rte_mbuf.h>
+
+#include "stmmac_regs.h"
+
+#define STMMAC_MAX_Q 1
+
+#define STMMAC_DEFAULT_RING_SIZE 256
+#define STMMAC_MAX_RING_SIZE 1024 /* Linux DMA_MAX_RX/TX_SIZE */
+
+/*
+ * Descriptor overlay: raw 32-bit words, interpreted through the bit macros
+ * in stmmac_regs.h, as the kernel driver does for the GMAC4 normal descriptor.
+ */
+struct stmmac_desc {
+ uint32_t des[4];
+};
+
+struct stmmac_tx_queue {
+ struct stmmac_desc *tx_ring;
+ uint64_t tx_ring_iova;
+ struct rte_mbuf **sw_ring;
+ uint16_t nb_desc;
+ uint16_t tx_tail; /* next slot to fill */
+ uint16_t tx_free; /* next slot to reclaim */
+ uint16_t queue_id;
+ uint16_t port_id;
+ void *chan_base; /* mac_base + DMA chan(0) offset */
+ const struct rte_memzone *mz;
+ uint64_t tx_packets;
+ uint64_t tx_bytes;
+ uint64_t tx_errors; /* TDES3 ES seen on reclaim */
+ uint64_t tx_ring_full; /* burst stopped: no free slot */
+};
+
+struct stmmac_rx_queue {
+ struct stmmac_desc *rx_ring;
+ uint64_t rx_ring_iova;
+ struct rte_mbuf **sw_ring;
+ uint16_t nb_desc;
+ uint16_t rx_tail; /* next slot to check */
+ uint16_t rx_refill; /* next slot to refill */
+ uint16_t queue_id;
+ uint16_t port_id;
+ struct rte_mempool *mb_pool;
+ void *chan_base; /* mac_base + DMA chan(0) offset */
+ const struct rte_memzone *mz;
+ uint64_t rx_packets;
+ uint64_t rx_bytes;
+ uint64_t rx_errors;
+ uint64_t rx_nombuf;
+};
+
+struct stmmac_priv {
+ void *mac_base; /* resource[0]: MAC/MTL/DMA block */
+ void *rgmii_base; /* resource[1]: RGMII IO macro (NULL if absent) */
+ struct stmmac_chan_addrs chan_addrs;
+
+ struct stmmac_tx_queue *txq[STMMAC_MAX_Q];
+ struct stmmac_rx_queue *rxq[STMMAC_MAX_Q];
+
+ struct rte_ether_addr mac_addr;
+
+ uint16_t mdio_addr; /* PHY MDIO address, from devarg */
+ int has_emac_ge_3; /* DMA/MTL channel stride override */
+
+ int link_up;
+ uint32_t link_speed;
+
+ struct rte_eth_stats stats;
+};
+
+/* stmmac_rxtx.c */
+uint16_t stmmac_rx_pkt_burst_nc(void *rxq, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts);
+uint16_t stmmac_tx_pkt_burst_nc(void *txq, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+
+uint16_t stmmac_mdio_read(void *mac_base, uint16_t mdio_addr,
+ uint16_t phyreg);
+void stmmac_mdio_write(void *mac_base, uint16_t mdio_addr,
+ uint16_t phyreg, uint16_t phydata);
+
+#endif /* _STMMAC_H_ */
diff --git a/drivers/net/stmmac/stmmac_ethdev.c b/drivers/net/stmmac/stmmac_ethdev.c
new file mode 100644
index 0000000000..ecbca764a5
--- /dev/null
+++ b/drivers/net/stmmac/stmmac_ethdev.c
@@ -0,0 +1,1007 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2026 Qualcomm Technologies, Inc.
+ *
+ * DPDK poll-mode driver for the Synopsys DWC EQoS (GMAC4, "stmmac") MAC.
+ *
+ * Scope:
+ * - DMA channel 0 with one TX and one RX queue.
+ * - Polling only: the platform bus has no interrupt support, so TX and RX
+ * completion and link state are polled.
+ * - Single-segment mbufs only; no TSO, PTP, or secondary process support.
+ * - Link state from the DP83867 PHY (BMSR and PHYSTS), using generic
+ * IEEE 802.3 clause-22 MDIO.
+ * - The RGMII IO macro and the DP83867 RGMII delays are configured here.
+ * The kernel recovery driver handles the clocks and the PHY reset.
+ */
+
+#include <stdlib.h>
+#include <string.h>
+
+#include <bus_platform_driver.h>
+#include <ethdev_driver.h>
+#include <rte_common.h>
+#include <rte_cycles.h>
+#include <rte_ethdev.h>
+#include <rte_io.h>
+#include <rte_kvargs.h>
+#include <rte_malloc.h>
+
+#include "stmmac.h"
+#include "stmmac_regs.h"
+#include "stmmac_logs.h"
+
+RTE_LOG_REGISTER_DEFAULT(stmmac_logtype_pmd, NOTICE);
+
+#define STMMAC_DRIVER_NAME net_stmmac
+
+#define STMMAC_DEVARG_HAS_EMAC_GE_3 "has_emac_ge_3"
+#define STMMAC_DEVARG_MDIO_ADDR "mdio_addr"
+
+/* Shikra EVK PHY (DP83867) MDIO address */
+#define STMMAC_DEFAULT_MDIO_ADDR 7
+
+#define STMMAC_DMA_RESET_POLL_US 1000
+#define STMMAC_DMA_RESET_POLLS 1000 /* 1 s in total, as dwmac4_dma_reset() */
+
+#define STMMAC_DMA_PBL 32
+#define STMMAC_DMA_PBL_SHIFT 16
+#define STMMAC_DMA_PBL_MASK (0x3fu << STMMAC_DMA_PBL_SHIFT)
+
+/* MTL RX/TX queue FIFO size fields (see stmmac_mtl_init) */
+#define STMMAC_ACTIVE_QUEUES 1
+
+/* Qualcomm RGMII IO macro and SDCC registers (offsets within the rgmii resource) */
+#define RGMII_IO_MACRO_CONFIG 0x00
+#define RGMII_IO_MACRO_CONFIG2 0x1c
+#define SDCC_HC_REG_DLL_CONFIG 0x04
+#define SDCC_USR_CTL 0x18
+
+#define RGMII_CONFIG_FUNC_CLK_EN RTE_BIT32(30)
+#define RGMII_CFG_DDR_MODE RTE_BIT32(0)
+#define RGMII_CFG_PROG_SWAP RTE_BIT32(1)
+#define RGMII_CFG_LOOPBACK_EN RTE_BIT32(2)
+#define RGMII_CFG_BYPASS_TX_ID_EN RTE_BIT32(3)
+#define RGMII_CFG_POS_NEG_DATA_SEL RTE_BIT32(23)
+#define RGMII_CFG2_TX_TO_RX_LOOPBACK RTE_BIT32(13)
+#define RGMII_CFG2_RSVD_CONFIG15 (0x7fffu << 17)
+#define RGMII_CFG2_RX_PROG_SWAP RTE_BIT32(7)
+#define RGMII_CFG2_DATA_DIV_CLK_SEL RTE_BIT32(6)
+#define RGMII_CFG2_TX_PHASE_SHIFT_EN RTE_BIT32(5)
+#define SDCC_DLL_CONFIG_PDN RTE_BIT32(29)
+#define SDCC_USR_CTL_DDR_BYPASS RTE_BIT32(30)
+
+/* DP83867 RGMII internal delays, set through the MMD indirect registers */
+#define MII_PHYSID1 0x02
+#define MII_PHYSID2 0x03
+#define MII_REGCR 0x0d
+#define MII_ADDAR 0x0e
+#define MII_REGCR_ADDR 0x001f
+#define MII_REGCR_DATA 0x401f
+#define DP83867_RGMIICTL 0x0032
+#define DP83867_RGMIIDCTL 0x0086
+#define DP83867_RGMII_TX_CLK_DELAY_EN RTE_BIT32(1)
+#define DP83867_RGMII_RX_CLK_DELAY_EN RTE_BIT32(0)
+#define DP83867_RGMIIDCTL_2_00_NS 0x7
+#define DP83867_PHYCTRL 0x10
+#define DP83867_PHY_ID 0x2000a231
+
+static const char * const stmmac_valid_args[] = {
+ STMMAC_DEVARG_HAS_EMAC_GE_3,
+ STMMAC_DEVARG_MDIO_ADDR,
+ NULL,
+};
+
+static uint64_t dev_rx_offloads_sup = RTE_ETH_RX_OFFLOAD_IPV4_CKSUM;
+static uint64_t dev_tx_offloads_sup = RTE_ETH_TX_OFFLOAD_IPV4_CKSUM |
+ RTE_ETH_TX_OFFLOAD_TCP_CKSUM |
+ RTE_ETH_TX_OFFLOAD_UDP_CKSUM;
+
+static int
+stmmac_kvarg_handler(const char *key, const char *value, void *opaque)
+{
+ unsigned long v;
+ char *end;
+
+ RTE_SET_USED(key);
+
+ v = strtoul(value, &end, 0);
+ if (*end != '\0')
+ return -EINVAL;
+
+ *(uint32_t *)opaque = (uint32_t)v;
+ return 0;
+}
+
+static void
+stmmac_parse_devargs(struct rte_platform_device *pdev, struct stmmac_priv *priv)
+{
+ struct rte_kvargs *kvlist;
+ uint32_t v;
+
+ priv->has_emac_ge_3 = 1;
+ priv->mdio_addr = STMMAC_DEFAULT_MDIO_ADDR;
+
+ if (pdev->device.devargs == NULL ||
+ pdev->device.devargs->args == NULL)
+ return;
+
+ kvlist = rte_kvargs_parse(pdev->device.devargs->args,
+ stmmac_valid_args);
+ if (kvlist == NULL)
+ return;
+
+ if (rte_kvargs_count(kvlist, STMMAC_DEVARG_HAS_EMAC_GE_3) > 0) {
+ v = 0;
+ if (rte_kvargs_process(kvlist, STMMAC_DEVARG_HAS_EMAC_GE_3,
+ stmmac_kvarg_handler, &v) == 0)
+ priv->has_emac_ge_3 = (int)v;
+ }
+
+ if (rte_kvargs_count(kvlist, STMMAC_DEVARG_MDIO_ADDR) > 0) {
+ v = 0;
+ if (rte_kvargs_process(kvlist, STMMAC_DEVARG_MDIO_ADDR,
+ stmmac_kvarg_handler, &v) == 0)
+ priv->mdio_addr = (uint16_t)v;
+ }
+
+ rte_kvargs_free(kvlist);
+}
+
+/*
+ * Keep the RGMII IO macro clock enabled. Without FUNC_CLK_EN the DMA reset
+ * can time out because the PHY clock is missing.
+ */
+static void
+stmmac_rgmii_func_clk_en(struct stmmac_priv *priv)
+{
+ void *reg;
+
+ if (priv->rgmii_base == NULL) {
+ STMMAC_PMD_WARN("RGMII region not mapped, FUNC_CLK_EN not set");
+ return;
+ }
+ reg = (uint8_t *)priv->rgmii_base + RGMII_IO_MACRO_CONFIG;
+ rte_write32(rte_read32(reg) | RGMII_CONFIG_FUNC_CLK_EN, reg);
+}
+
+static void
+stmmac_rgmii_rmw(struct stmmac_priv *priv, uint32_t off, uint32_t clr, uint32_t set)
+{
+ void *reg = (uint8_t *)priv->rgmii_base + off;
+
+ rte_write32((rte_read32(reg) & ~clr) | set, reg);
+}
+
+/*
+ * RGMII IO macro setup for rgmii-id at 1 Gb/s, as the kernel does at link-up.
+ * The PHY adds the RGMII delays, so the MAC-side DLL is powered down and the
+ * DDR path is bypassed. TX-to-RX loopback is disabled so RX data comes from
+ * the PHY.
+ */
+static void
+stmmac_rgmii_id_1g_init(struct stmmac_priv *priv)
+{
+ if (priv->rgmii_base == NULL)
+ return;
+
+ stmmac_rgmii_rmw(priv, SDCC_HC_REG_DLL_CONFIG, 0, SDCC_DLL_CONFIG_PDN);
+ stmmac_rgmii_rmw(priv, SDCC_USR_CTL, 0, SDCC_USR_CTL_DDR_BYPASS);
+
+ stmmac_rgmii_rmw(priv, RGMII_IO_MACRO_CONFIG2, RGMII_CFG2_TX_TO_RX_LOOPBACK, 0);
+ stmmac_rgmii_rmw(priv, RGMII_IO_MACRO_CONFIG,
+ RGMII_CFG_POS_NEG_DATA_SEL | RGMII_CFG_PROG_SWAP |
+ RGMII_CFG_LOOPBACK_EN,
+ RGMII_CFG_DDR_MODE | RGMII_CFG_BYPASS_TX_ID_EN);
+ stmmac_rgmii_rmw(priv, RGMII_IO_MACRO_CONFIG2,
+ RGMII_CFG2_DATA_DIV_CLK_SEL | RGMII_CFG2_TX_PHASE_SHIFT_EN |
+ RGMII_CFG2_RSVD_CONFIG15,
+ RGMII_CFG2_RX_PROG_SWAP);
+}
+
+static uint16_t
+stmmac_phy_mmd_read(struct stmmac_priv *priv, uint16_t reg)
+{
+ stmmac_mdio_write(priv->mac_base, priv->mdio_addr, MII_REGCR, MII_REGCR_ADDR);
+ stmmac_mdio_write(priv->mac_base, priv->mdio_addr, MII_ADDAR, reg);
+ stmmac_mdio_write(priv->mac_base, priv->mdio_addr, MII_REGCR, MII_REGCR_DATA);
+ return stmmac_mdio_read(priv->mac_base, priv->mdio_addr, MII_ADDAR);
+}
+
+static void
+stmmac_phy_mmd_write(struct stmmac_priv *priv, uint16_t reg, uint16_t val)
+{
+ stmmac_mdio_write(priv->mac_base, priv->mdio_addr, MII_REGCR, MII_REGCR_ADDR);
+ stmmac_mdio_write(priv->mac_base, priv->mdio_addr, MII_ADDAR, reg);
+ stmmac_mdio_write(priv->mac_base, priv->mdio_addr, MII_REGCR, MII_REGCR_DATA);
+ stmmac_mdio_write(priv->mac_base, priv->mdio_addr, MII_ADDAR, val);
+}
+
+/*
+ * Set the DP83867 RGMII internal delays to 2.00 ns on both TX and RX. The
+ * kernel does the same for phy-mode "rgmii-id". The MAC does not add these
+ * delays, so without them RX frames are sampled off the data eye.
+ */
+static void
+stmmac_phy_rgmii_id_init(struct stmmac_priv *priv)
+{
+ uint16_t id1, id2, ctl;
+
+ id1 = stmmac_mdio_read(priv->mac_base, priv->mdio_addr, MII_PHYSID1);
+ id2 = stmmac_mdio_read(priv->mac_base, priv->mdio_addr, MII_PHYSID2);
+ if (id1 == 0xffff || id2 == 0xffff) {
+ STMMAC_PMD_WARN("PHY not responding on MDIO, RGMII delays not set");
+ return;
+ }
+ if (((uint32_t)id1 << 16 | id2) != DP83867_PHY_ID)
+ STMMAC_PMD_WARN("unexpected PHY ID 0x%04x%04x", id1, id2);
+
+ ctl = stmmac_phy_mmd_read(priv, DP83867_RGMIICTL);
+ stmmac_phy_mmd_write(priv, DP83867_RGMIICTL,
+ ctl | DP83867_RGMII_TX_CLK_DELAY_EN |
+ DP83867_RGMII_RX_CLK_DELAY_EN);
+ stmmac_phy_mmd_write(priv, DP83867_RGMIIDCTL,
+ DP83867_RGMIIDCTL_2_00_NS |
+ (DP83867_RGMIIDCTL_2_00_NS << 4));
+}
+
+/*
+ * DMA soft reset, as dwmac4_dma_reset(): poll the SWR bit for up to 1 s.
+ */
+static int
+stmmac_dma_reset(void *mac_base)
+{
+ void *reg = (uint8_t *)mac_base + DMA_BUS_MODE;
+ unsigned int i;
+
+ rte_write32(rte_read32(reg) | DMA_BUS_MODE_SFT_RESET, reg);
+
+ for (i = 0; i < STMMAC_DMA_RESET_POLLS; i++) {
+ if (!(rte_read32(reg) & DMA_BUS_MODE_SFT_RESET))
+ return 0;
+ rte_delay_us(STMMAC_DMA_RESET_POLL_US);
+ }
+
+ STMMAC_PMD_ERR("DMA soft reset timed out");
+ return -ETIMEDOUT;
+}
+
+/*
+ * DMA system and AXI bus configuration. Fixed burst, mixed burst and AAL are
+ * enabled, with BLEN16 and the maximum outstanding requests. EAME is set
+ * because the rings and mbufs are above 4 GiB.
+ */
+static void
+stmmac_dma_sysbus_init(void *mac_base)
+{
+ void *sysbus_reg = (uint8_t *)mac_base + DMA_SYS_BUS_MODE;
+ void *axi_reg = (uint8_t *)mac_base + DMA_AXI_BUS_MODE;
+ uint32_t value;
+
+ value = rte_read32(sysbus_reg);
+ value |= DMA_SYS_BUS_FB | DMA_SYS_BUS_MB | DMA_SYS_BUS_AAL;
+ value |= DMA_SYS_BUS_EAME;
+ rte_write32(value, sysbus_reg);
+
+ value = rte_read32(axi_reg);
+ value &= ~(DMA_AXI_WR_OSR_LMT | DMA_AXI_RD_OSR_LMT);
+ value |= (DMA_AXI_OSR_MAX << DMA_AXI_WR_OSR_LMT_SHIFT) & DMA_AXI_WR_OSR_LMT;
+ value |= (DMA_AXI_OSR_MAX << DMA_AXI_RD_OSR_LMT_SHIFT) & DMA_AXI_RD_OSR_LMT;
+ value |= DMA_AXI_BLEN16;
+ rte_write32(value, axi_reg);
+}
+
+/*
+ * Per-channel DMA burst settings: PBL 32, PBLx8 on, and OSP on TX. The DMA is
+ * too slow to keep up with the wire if these stay at the reset values.
+ */
+static void
+stmmac_dma_chan_cfg(void *chan_base)
+{
+ void *ch_ctrl = (uint8_t *)chan_base + DMA_CHAN_OFF_CONTROL;
+ void *tx_ctrl = (uint8_t *)chan_base + DMA_CHAN_OFF_TX_CONTROL;
+ void *rx_ctrl = (uint8_t *)chan_base + DMA_CHAN_OFF_RX_CONTROL;
+ uint32_t v;
+
+ v = rte_read32(ch_ctrl);
+ rte_write32(v | DMA_CONTROL_PBLX8, ch_ctrl);
+
+ v = rte_read32(tx_ctrl);
+ v &= ~STMMAC_DMA_PBL_MASK;
+ v |= ((uint32_t)STMMAC_DMA_PBL << STMMAC_DMA_PBL_SHIFT) | DMA_CHAN_TX_CONTROL_OSP;
+ rte_write32(v, tx_ctrl);
+
+ v = rte_read32(rx_ctrl);
+ v &= ~STMMAC_DMA_PBL_MASK;
+ v |= (uint32_t)STMMAC_DMA_PBL << STMMAC_DMA_PBL_SHIFT;
+ rte_write32(v, rx_ctrl);
+}
+
+static int
+stmmac_tx_ring_setup(struct stmmac_tx_queue *txq)
+{
+ uint8_t *chan = txq->chan_base;
+ unsigned int i;
+
+ for (i = 0; i < txq->nb_desc; i++)
+ memset(&txq->tx_ring[i], 0, sizeof(struct stmmac_desc));
+ /* the non-coherent DMA must see the zeroed ring */
+ stmmac_dcache_clean(txq->tx_ring, (size_t)txq->nb_desc * sizeof(struct stmmac_desc));
+
+ rte_write32((uint32_t)(txq->tx_ring_iova >> 32), chan + DMA_CHAN_OFF_TX_BASE_ADDR_HI);
+ rte_write32((uint32_t)txq->tx_ring_iova, chan + DMA_CHAN_OFF_TX_BASE_ADDR);
+ /* the ring length register holds nb_desc - 1 */
+ rte_write32(txq->nb_desc - 1, chan + DMA_CHAN_OFF_TX_RING_LEN);
+ rte_write32((uint32_t)txq->tx_ring_iova, chan + DMA_CHAN_OFF_TX_END_ADDR);
+
+ return 0;
+}
+
+static int
+stmmac_rx_ring_setup(struct stmmac_rx_queue *rxq)
+{
+ uint8_t *chan = rxq->chan_base;
+ unsigned int i;
+ uint64_t tail;
+ uint32_t bufsz, v;
+
+ for (i = 0; i < rxq->nb_desc; i++) {
+ struct rte_mbuf *mb = rte_pktmbuf_alloc(rxq->mb_pool);
+ uint64_t iova;
+
+ if (mb == NULL) {
+ STMMAC_PMD_ERR("failed to allocate RX mbuf %u", i);
+ return -ENOMEM;
+ }
+
+ iova = rte_pktmbuf_iova(mb);
+ rxq->sw_ring[i] = mb;
+ /* drop any dirty lines over the buffer before the DMA writes it */
+ stmmac_dcache_flush(rte_pktmbuf_mtod(mb, void *),
+ rte_pktmbuf_data_room_size(rxq->mb_pool) -
+ RTE_PKTMBUF_HEADROOM);
+ rxq->rx_ring[i].des[0] = (uint32_t)iova;
+ rxq->rx_ring[i].des[1] = (uint32_t)(iova >> 32);
+ rxq->rx_ring[i].des[2] = 0;
+ rxq->rx_ring[i].des[3] = RDES3_OWN | RDES3_BUFFER1_VALID_ADDR |
+ RDES3_INT_ON_COMPLETION_EN;
+ }
+ /* the non-coherent DMA must see every OWN=1 descriptor before RXTAIL is set */
+ stmmac_dcache_clean(rxq->rx_ring, (size_t)rxq->nb_desc * sizeof(struct stmmac_desc));
+
+ /* RX buffer size, which must be non-zero and a multiple of 4 */
+ bufsz = (rte_pktmbuf_data_room_size(rxq->mb_pool) - RTE_PKTMBUF_HEADROOM) & ~3u;
+ v = rte_read32(chan + DMA_CHAN_OFF_RX_CONTROL);
+ v = (v & ~DMA_CONTROL_RBSZ_MASK) | ((bufsz << DMA_CONTROL_RBSZ_SHIFT) &
+ DMA_CONTROL_RBSZ_MASK);
+ rte_write32(v, chan + DMA_CHAN_OFF_RX_CONTROL);
+
+ rte_write32((uint32_t)(rxq->rx_ring_iova >> 32), chan + DMA_CHAN_OFF_RX_BASE_ADDR_HI);
+ rte_write32((uint32_t)rxq->rx_ring_iova, chan + DMA_CHAN_OFF_RX_BASE_ADDR);
+ /* the ring length register holds nb_desc - 1 */
+ rte_write32(rxq->nb_desc - 1, chan + DMA_CHAN_OFF_RX_RING_LEN);
+ tail = rxq->rx_ring_iova + (uint64_t)rxq->nb_desc * sizeof(struct stmmac_desc);
+ rte_write32((uint32_t)tail, chan + DMA_CHAN_OFF_RX_END_ADDR);
+
+ return 0;
+}
+
+/*
+ * MTL operating mode for the single active queue. Store-and-forward is
+ * enabled, and the queue FIFO sizes (RQS/TQS) are programmed from
+ * HW_FEATURE1. The reset value of 256 bytes is too small for store-and-forward.
+ * Queue 0 owns the whole FIFO and maps to DMA channel 0. Flow control is not
+ * enabled.
+ */
+static void
+stmmac_mtl_init(void *mac_base, const struct stmmac_chan_addrs *addrs)
+{
+ void *tx_op = (uint8_t *)mac_base + MTL_CHAN_TX_OP_MODE(addrs, 0);
+ void *rx_op = (uint8_t *)mac_base + MTL_CHAN_RX_OP_MODE(addrs, 0);
+ void *rxq_map = (uint8_t *)mac_base + MTL_RXQ_DMA_MAP0;
+ uint32_t hwf1 = rte_read32((uint8_t *)mac_base + GMAC_HW_FEATURE1);
+ uint32_t rx_fifo = 128u << (hwf1 & GMAC_HWF1_RXFIFOSIZE_MASK);
+ uint32_t tx_fifo = 128u << ((hwf1 >> GMAC_HWF1_TXFIFOSIZE_SHIFT) &
+ GMAC_HWF1_TXFIFOSIZE_MASK);
+ uint32_t rqs = rx_fifo / STMMAC_ACTIVE_QUEUES >= 256 ?
+ rx_fifo / STMMAC_ACTIVE_QUEUES / 256 - 1 : 0;
+ uint32_t tqs = tx_fifo / STMMAC_ACTIVE_QUEUES >= 256 ?
+ tx_fifo / STMMAC_ACTIVE_QUEUES / 256 - 1 : 0;
+ uint32_t value;
+
+ if (rqs > (MTL_OP_MODE_RQS_MASK >> MTL_OP_MODE_RQS_SHIFT))
+ rqs = MTL_OP_MODE_RQS_MASK >> MTL_OP_MODE_RQS_SHIFT;
+ if (tqs > (MTL_OP_MODE_TQS_MASK >> MTL_OP_MODE_TQS_SHIFT))
+ tqs = MTL_OP_MODE_TQS_MASK >> MTL_OP_MODE_TQS_SHIFT;
+
+ value = rte_read32(tx_op);
+ value &= ~MTL_OP_MODE_TQS_MASK;
+ value |= MTL_OP_MODE_TSF | MTL_OP_MODE_TXQEN | (tqs << MTL_OP_MODE_TQS_SHIFT);
+ rte_write32(value, tx_op);
+
+ value = rte_read32(rx_op);
+ value &= ~MTL_OP_MODE_RQS_MASK;
+ value |= MTL_OP_MODE_RSF | (rqs << MTL_OP_MODE_RQS_SHIFT);
+ rte_write32(value, rx_op);
+
+ /* RX queue 0 maps to DMA channel 0 */
+ value = rte_read32(rxq_map);
+ value &= ~0xfu;
+ rte_write32(value, rxq_map);
+}
+
+static void
+stmmac_mac_init(void *mac_base, struct rte_ether_addr *mac_addr)
+{
+ void *config_reg = (uint8_t *)mac_base + GMAC_CONFIG;
+ uint32_t value;
+ uint32_t addr_lo, addr_hi;
+
+ addr_lo = mac_addr->addr_bytes[0] | (mac_addr->addr_bytes[1] << 8) |
+ (mac_addr->addr_bytes[2] << 16) | (mac_addr->addr_bytes[3] << 24);
+ addr_hi = mac_addr->addr_bytes[4] | (mac_addr->addr_bytes[5] << 8);
+
+ rte_write32(addr_lo, (uint8_t *)mac_base + GMAC_ADDR_LOW(0));
+ rte_write32(addr_hi, (uint8_t *)mac_base + GMAC_ADDR_HIGH(0));
+
+ value = rte_read32(config_reg);
+ value |= GMAC_CONFIG_IPC | GMAC_CONFIG_ACS;
+ value |= GMAC_CONFIG_TE | GMAC_CONFIG_RE;
+ rte_write32(value, config_reg);
+}
+
+static void
+stmmac_dma_start(struct stmmac_priv *priv)
+{
+ uint8_t *chan = (uint8_t *)priv->mac_base + stmmac_dma_chan_base(&priv->chan_addrs, 0);
+ void *tx_ctrl = chan + DMA_CHAN_OFF_TX_CONTROL;
+ void *rx_ctrl = chan + DMA_CHAN_OFF_RX_CONTROL;
+ uint32_t value;
+
+ value = rte_read32(tx_ctrl);
+ value |= DMA_CONTROL_ST;
+ rte_write32(value, tx_ctrl);
+
+ value = rte_read32(rx_ctrl);
+ value &= ~DMA_CONTROL_RPF;
+ value |= DMA_CONTROL_SR;
+ rte_write32(value, rx_ctrl);
+}
+
+static void
+stmmac_dma_stop(struct stmmac_priv *priv)
+{
+ uint8_t *chan = (uint8_t *)priv->mac_base + stmmac_dma_chan_base(&priv->chan_addrs, 0);
+ void *tx_ctrl = chan + DMA_CHAN_OFF_TX_CONTROL;
+ void *rx_ctrl = chan + DMA_CHAN_OFF_RX_CONTROL;
+ uint32_t value;
+
+ value = rte_read32(tx_ctrl);
+ value &= ~DMA_CONTROL_ST;
+ rte_write32(value, tx_ctrl);
+
+ value = rte_read32(rx_ctrl);
+ value |= DMA_CONTROL_RPF;
+ value &= ~DMA_CONTROL_SR;
+ rte_write32(value, rx_ctrl);
+}
+
+static int
+stmmac_eth_configure(struct rte_eth_dev *dev)
+{
+ RTE_SET_USED(dev);
+ return 0;
+}
+
+static int
+stmmac_eth_start(struct rte_eth_dev *dev)
+{
+ struct stmmac_priv *priv = dev->data->dev_private;
+ int ret;
+
+ if (priv->rxq[0] == NULL || priv->txq[0] == NULL) {
+ STMMAC_PMD_ERR("queues not set up");
+ return -EINVAL;
+ }
+
+ /* the RGMII clock must run before the DMA reset */
+ stmmac_rgmii_func_clk_en(priv);
+ stmmac_rgmii_id_1g_init(priv);
+ stmmac_phy_rgmii_id_init(priv);
+
+ ret = stmmac_dma_reset(priv->mac_base);
+ if (ret != 0)
+ return ret;
+
+ stmmac_dma_sysbus_init(priv->mac_base);
+ stmmac_dma_chan_cfg((uint8_t *)priv->mac_base +
+ stmmac_dma_chan_base(&priv->chan_addrs, 0));
+
+ ret = stmmac_tx_ring_setup(priv->txq[0]);
+ if (ret != 0)
+ return ret;
+
+ ret = stmmac_rx_ring_setup(priv->rxq[0]);
+ if (ret != 0)
+ return ret;
+
+ stmmac_mac_init(priv->mac_base, &priv->mac_addr);
+
+ /* enable RX queue 0 in generic (non-DCB) mode */
+ {
+ void *rxq_ctrl = (uint8_t *)priv->mac_base + GMAC_RXQ_CTRL0;
+ uint32_t v = rte_read32(rxq_ctrl);
+
+ rte_write32((v & ~0x3u) | 0x2u, rxq_ctrl);
+ }
+
+ stmmac_mtl_init(priv->mac_base, &priv->chan_addrs);
+ stmmac_dma_start(priv);
+
+ dev->rx_pkt_burst = stmmac_rx_pkt_burst_nc;
+ dev->tx_pkt_burst = stmmac_tx_pkt_burst_nc;
+
+ dev->data->dev_started = 1;
+ dev->data->rx_queue_state[0] = RTE_ETH_QUEUE_STATE_STARTED;
+ dev->data->tx_queue_state[0] = RTE_ETH_QUEUE_STATE_STARTED;
+
+ return 0;
+}
+
+static int
+stmmac_eth_stop(struct rte_eth_dev *dev)
+{
+ struct stmmac_priv *priv = dev->data->dev_private;
+
+ dev->data->dev_started = 0;
+ stmmac_dma_stop(priv);
+ dev->data->rx_queue_state[0] = RTE_ETH_QUEUE_STATE_STOPPED;
+ dev->data->tx_queue_state[0] = RTE_ETH_QUEUE_STATE_STOPPED;
+ return 0;
+}
+
+static void
+stmmac_queue_release(struct stmmac_priv *priv)
+{
+ if (priv->txq[0] != NULL) {
+ rte_free(priv->txq[0]->sw_ring);
+ rte_free(priv->txq[0]);
+ priv->txq[0] = NULL;
+ }
+ if (priv->rxq[0] != NULL) {
+ unsigned int i;
+
+ for (i = 0; i < priv->rxq[0]->nb_desc; i++)
+ rte_pktmbuf_free(priv->rxq[0]->sw_ring[i]);
+ rte_free(priv->rxq[0]->sw_ring);
+ rte_free(priv->rxq[0]);
+ priv->rxq[0] = NULL;
+ }
+}
+
+static int
+stmmac_eth_close(struct rte_eth_dev *dev)
+{
+ struct stmmac_priv *priv = dev->data->dev_private;
+
+ stmmac_queue_release(priv);
+
+ return 0;
+}
+
+/*
+ * Link state. The DP83867 BMSR link bit is latched low, so BMSR is read twice
+ * to clear the latch. The live link state and the speed and duplex come from
+ * PHYSTS, which is not latched. BMSR is used only if PHYSTS does not respond.
+ */
+static int
+stmmac_eth_link_update(struct rte_eth_dev *dev, int wait_to_complete)
+{
+ struct stmmac_priv *priv = dev->data->dev_private;
+ struct rte_eth_link link;
+ uint16_t bmsr, physts;
+
+ RTE_SET_USED(wait_to_complete);
+
+ memset(&link, 0, sizeof(link));
+
+ /* the first read clears the latched link bit */
+ (void)stmmac_mdio_read(priv->mac_base, priv->mdio_addr, STMMAC_MII_BMSR);
+ bmsr = stmmac_mdio_read(priv->mac_base, priv->mdio_addr, STMMAC_MII_BMSR);
+ physts = stmmac_mdio_read(priv->mac_base, priv->mdio_addr, DP83867_PHYSTS);
+
+ if (physts != 0xffff)
+ priv->link_up = !!(physts & DP83867_PHYSTS_LINK_BIT);
+ else
+ priv->link_up = !!(bmsr & STMMAC_MII_BMSR_LSTATUS);
+ link.link_status = priv->link_up;
+
+ if (physts != 0xffff && priv->link_up) {
+ uint16_t spd = physts & DP83867_PHYSTS_SPEED_MASK;
+
+ if (spd == DP83867_PHYSTS_SPEED_1G)
+ link.link_speed = RTE_ETH_SPEED_NUM_1G;
+ else if (spd == DP83867_PHYSTS_SPEED_100M)
+ link.link_speed = RTE_ETH_SPEED_NUM_100M;
+ else
+ link.link_speed = RTE_ETH_SPEED_NUM_10M;
+
+ link.link_duplex = (physts & DP83867_PHYSTS_DUPLEX_BIT) ?
+ RTE_ETH_LINK_FULL_DUPLEX : RTE_ETH_LINK_HALF_DUPLEX;
+ } else {
+ link.link_speed = RTE_ETH_SPEED_NUM_1G;
+ link.link_duplex = RTE_ETH_LINK_FULL_DUPLEX;
+ }
+
+ link.link_autoneg = RTE_ETH_LINK_AUTONEG;
+
+ return rte_eth_linkstatus_set(dev, &link);
+}
+
+static int
+stmmac_promiscuous_enable(struct rte_eth_dev *dev)
+{
+ struct stmmac_priv *priv = dev->data->dev_private;
+ void *filter_reg = (uint8_t *)priv->mac_base + GMAC_PACKET_FILTER;
+
+ rte_write32(rte_read32(filter_reg) | GMAC_PACKET_FILTER_PR, filter_reg);
+
+ return 0;
+}
+
+static int
+stmmac_promiscuous_disable(struct rte_eth_dev *dev)
+{
+ struct stmmac_priv *priv = dev->data->dev_private;
+ void *filter_reg = (uint8_t *)priv->mac_base + GMAC_PACKET_FILTER;
+
+ rte_write32(rte_read32(filter_reg) & ~GMAC_PACKET_FILTER_PR, filter_reg);
+
+ return 0;
+}
+
+static int
+stmmac_mac_addr_set(struct rte_eth_dev *dev, struct rte_ether_addr *addr)
+{
+ struct stmmac_priv *priv = dev->data->dev_private;
+
+ rte_ether_addr_copy(addr, &priv->mac_addr);
+
+ rte_write32(addr->addr_bytes[0] | (addr->addr_bytes[1] << 8) |
+ (addr->addr_bytes[2] << 16) | (addr->addr_bytes[3] << 24),
+ (uint8_t *)priv->mac_base + GMAC_ADDR_LOW(0));
+ rte_write32(addr->addr_bytes[4] | (addr->addr_bytes[5] << 8),
+ (uint8_t *)priv->mac_base + GMAC_ADDR_HIGH(0));
+
+ return 0;
+}
+
+static int
+stmmac_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats,
+ struct eth_queue_stats *qstats)
+{
+ struct stmmac_priv *priv = dev->data->dev_private;
+
+ RTE_SET_USED(qstats);
+ memcpy(stats, &priv->stats, sizeof(*stats));
+
+ /* the queue counters are updated by the burst functions */
+ if (priv->rxq[0] != NULL) {
+ stats->ipackets = priv->rxq[0]->rx_packets;
+ stats->ibytes = priv->rxq[0]->rx_bytes;
+ stats->ierrors = priv->rxq[0]->rx_errors;
+ stats->rx_nombuf = priv->rxq[0]->rx_nombuf;
+ }
+ if (priv->txq[0] != NULL) {
+ stats->opackets = priv->txq[0]->tx_packets;
+ stats->obytes = priv->txq[0]->tx_bytes;
+ stats->oerrors = priv->txq[0]->tx_errors;
+ }
+
+ return 0;
+}
+
+static int
+stmmac_stats_reset(struct rte_eth_dev *dev)
+{
+ struct stmmac_priv *priv = dev->data->dev_private;
+
+ memset(&priv->stats, 0, sizeof(priv->stats));
+ if (priv->rxq[0] != NULL) {
+ priv->rxq[0]->rx_packets = 0;
+ priv->rxq[0]->rx_bytes = 0;
+ priv->rxq[0]->rx_errors = 0;
+ priv->rxq[0]->rx_nombuf = 0;
+ }
+ if (priv->txq[0] != NULL) {
+ priv->txq[0]->tx_packets = 0;
+ priv->txq[0]->tx_bytes = 0;
+ priv->txq[0]->tx_errors = 0;
+ priv->txq[0]->tx_ring_full = 0;
+ }
+
+ return 0;
+}
+
+static int
+stmmac_dev_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
+{
+ RTE_SET_USED(dev);
+
+ dev_info->max_rx_queues = STMMAC_MAX_Q;
+ dev_info->max_tx_queues = STMMAC_MAX_Q;
+ dev_info->min_rx_bufsize = RTE_ETHER_MIN_LEN;
+ dev_info->max_rx_pktlen = RTE_ETHER_MAX_LEN;
+ dev_info->rx_offload_capa = dev_rx_offloads_sup;
+ dev_info->tx_offload_capa = dev_tx_offloads_sup;
+
+ /*
+ * Ring sizes follow the Linux limits (64..1024). Sizes must be a
+ * multiple of 4, one descriptor cache line. One segment per packet.
+ */
+ dev_info->rx_desc_lim = (struct rte_eth_desc_lim) {
+ .nb_max = STMMAC_MAX_RING_SIZE,
+ .nb_min = 64,
+ .nb_align = 4,
+ .nb_seg_max = 1,
+ .nb_mtu_seg_max = 1,
+ };
+ dev_info->tx_desc_lim = dev_info->rx_desc_lim;
+ dev_info->default_rxportconf.ring_size = STMMAC_MAX_RING_SIZE;
+ dev_info->default_txportconf.ring_size = STMMAC_MAX_RING_SIZE;
+
+ return 0;
+}
+
+static int
+stmmac_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id,
+ uint16_t nb_rx_desc, unsigned int socket_id,
+ const struct rte_eth_rxconf *rx_conf,
+ struct rte_mempool *mb_pool)
+{
+ struct stmmac_priv *priv = dev->data->dev_private;
+ struct stmmac_rx_queue *rxq;
+ const struct rte_memzone *mz;
+ size_t ring_size;
+
+ RTE_SET_USED(rx_conf);
+
+ if (rx_queue_id >= STMMAC_MAX_Q) {
+ STMMAC_PMD_ERR("invalid rx queue id %u, max %u",
+ rx_queue_id, STMMAC_MAX_Q);
+ return -EINVAL;
+ }
+
+ if (nb_rx_desc == 0)
+ nb_rx_desc = STMMAC_DEFAULT_RING_SIZE;
+
+ rxq = rte_zmalloc_socket("stmmac_rxq", sizeof(*rxq),
+ RTE_CACHE_LINE_SIZE, socket_id);
+ if (rxq == NULL)
+ return -ENOMEM;
+
+ ring_size = nb_rx_desc * sizeof(struct stmmac_desc);
+ mz = rte_eth_dma_zone_reserve(dev, "rx_ring", rx_queue_id, ring_size,
+ RTE_CACHE_LINE_SIZE, socket_id);
+ if (mz == NULL) {
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ rxq->sw_ring = rte_zmalloc_socket("stmmac_rx_sw_ring",
+ nb_rx_desc * sizeof(struct rte_mbuf *),
+ RTE_CACHE_LINE_SIZE, socket_id);
+ if (rxq->sw_ring == NULL) {
+ rte_free(rxq);
+ return -ENOMEM;
+ }
+
+ rxq->rx_ring = mz->addr;
+ rxq->rx_ring_iova = mz->iova;
+ rxq->nb_desc = nb_rx_desc;
+ rxq->rx_tail = 0;
+ rxq->rx_refill = 0;
+ rxq->queue_id = rx_queue_id;
+ rxq->port_id = dev->data->port_id;
+ rxq->mb_pool = mb_pool;
+ rxq->chan_base = (uint8_t *)priv->mac_base +
+ stmmac_dma_chan_base(&priv->chan_addrs, 0);
+ rxq->mz = mz;
+
+ priv->rxq[rx_queue_id] = rxq;
+ dev->data->rx_queues[rx_queue_id] = rxq;
+ return 0;
+}
+
+static int
+stmmac_tx_queue_setup(struct rte_eth_dev *dev, uint16_t tx_queue_id,
+ uint16_t nb_tx_desc, unsigned int socket_id,
+ const struct rte_eth_txconf *tx_conf)
+{
+ struct stmmac_priv *priv = dev->data->dev_private;
+ struct stmmac_tx_queue *txq;
+ const struct rte_memzone *mz;
+ size_t ring_size;
+
+ RTE_SET_USED(tx_conf);
+
+ if (tx_queue_id >= STMMAC_MAX_Q) {
+ STMMAC_PMD_ERR("invalid tx queue id %u, max %u",
+ tx_queue_id, STMMAC_MAX_Q);
+ return -EINVAL;
+ }
+
+ if (nb_tx_desc == 0)
+ nb_tx_desc = STMMAC_DEFAULT_RING_SIZE;
+ /* the TX burst writes whole descriptor cache lines: the ring must be a multiple of 4 */
+ if (nb_tx_desc % STMMAC_DESCS_PER_LINE) {
+ STMMAC_PMD_ERR("nb_tx_desc %u must be a multiple of %u",
+ nb_tx_desc, STMMAC_DESCS_PER_LINE);
+ return -EINVAL;
+ }
+
+ txq = rte_zmalloc_socket("stmmac_txq", sizeof(*txq),
+ RTE_CACHE_LINE_SIZE, socket_id);
+ if (txq == NULL)
+ return -ENOMEM;
+
+ ring_size = nb_tx_desc * sizeof(struct stmmac_desc);
+ mz = rte_eth_dma_zone_reserve(dev, "tx_ring", tx_queue_id, ring_size,
+ RTE_CACHE_LINE_SIZE, socket_id);
+ if (mz == NULL) {
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ txq->sw_ring = rte_zmalloc_socket("stmmac_tx_sw_ring",
+ nb_tx_desc * sizeof(struct rte_mbuf *),
+ RTE_CACHE_LINE_SIZE, socket_id);
+ if (txq->sw_ring == NULL) {
+ rte_free(txq);
+ return -ENOMEM;
+ }
+
+ txq->tx_ring = mz->addr;
+ txq->tx_ring_iova = mz->iova;
+ txq->nb_desc = nb_tx_desc;
+ txq->tx_tail = 0;
+ txq->tx_free = 0;
+ txq->queue_id = tx_queue_id;
+ txq->port_id = dev->data->port_id;
+ txq->chan_base = (uint8_t *)priv->mac_base +
+ stmmac_dma_chan_base(&priv->chan_addrs, 0);
+ txq->mz = mz;
+
+ priv->txq[tx_queue_id] = txq;
+ dev->data->tx_queues[tx_queue_id] = txq;
+ return 0;
+}
+
+static void
+stmmac_rxq_info_get(struct rte_eth_dev *dev, uint16_t qid,
+ struct rte_eth_rxq_info *qinfo)
+{
+ struct stmmac_rx_queue *rxq = dev->data->rx_queues[qid];
+
+ qinfo->mp = rxq->mb_pool;
+ qinfo->nb_desc = rxq->nb_desc;
+}
+
+static void
+stmmac_txq_info_get(struct rte_eth_dev *dev, uint16_t qid,
+ struct rte_eth_txq_info *qinfo)
+{
+ struct stmmac_tx_queue *txq = dev->data->tx_queues[qid];
+
+ qinfo->nb_desc = txq->nb_desc;
+}
+
+static const struct eth_dev_ops stmmac_eth_ops = {
+ .dev_configure = stmmac_eth_configure,
+ .dev_start = stmmac_eth_start,
+ .dev_stop = stmmac_eth_stop,
+ .dev_close = stmmac_eth_close,
+ .link_update = stmmac_eth_link_update,
+ .promiscuous_enable = stmmac_promiscuous_enable,
+ .promiscuous_disable = stmmac_promiscuous_disable,
+ .mac_addr_set = stmmac_mac_addr_set,
+ .stats_get = stmmac_stats_get,
+ .stats_reset = stmmac_stats_reset,
+ .dev_infos_get = stmmac_dev_infos_get,
+ .rx_queue_setup = stmmac_rx_queue_setup,
+ .tx_queue_setup = stmmac_tx_queue_setup,
+ .rxq_info_get = stmmac_rxq_info_get,
+ .txq_info_get = stmmac_txq_info_get,
+};
+
+static int
+stmmac_platform_probe(struct rte_platform_device *pdev)
+{
+ struct rte_eth_dev *dev;
+ struct stmmac_priv *priv;
+ unsigned int i;
+
+ if (pdev->num_resource < 1 || pdev->resource[0].mem.addr == NULL) {
+ STMMAC_PMD_ERR("probe failed: no MMIO resource[0] (MAC/DMA/MTL block)");
+ return -EINVAL;
+ }
+
+ dev = rte_eth_dev_allocate(pdev->device.name);
+ if (dev == NULL) {
+ STMMAC_PMD_ERR("probe failed: rte_eth_dev_allocate() returned NULL for %s",
+ pdev->device.name);
+ return -ENOMEM;
+ }
+ dev->device = &pdev->device;
+
+ priv = rte_zmalloc_socket(NULL, sizeof(*priv), RTE_CACHE_LINE_SIZE,
+ dev->data->numa_node);
+ if (priv == NULL) {
+ STMMAC_PMD_ERR("probe failed: unable to allocate stmmac_priv for %s",
+ pdev->device.name);
+ rte_eth_dev_release_port(dev);
+ return -ENOMEM;
+ }
+ dev->data->dev_private = priv;
+
+ priv->mac_base = pdev->resource[0].mem.addr;
+ /* DT reg-names = "stmmaceth", "rgmii": resource[1] is the RGMII IO macro */
+ if (pdev->num_resource > 1)
+ priv->rgmii_base = pdev->resource[1].mem.addr;
+
+ stmmac_parse_devargs(pdev, priv);
+
+ if (priv->has_emac_ge_3) {
+ struct stmmac_chan_addrs ge3 = STMMAC_CHAN_ADDRS_EMAC_GE3_INIT;
+
+ priv->chan_addrs = ge3;
+ } else {
+ struct stmmac_chan_addrs generic = STMMAC_CHAN_ADDRS_GENERIC_INIT;
+
+ priv->chan_addrs = generic;
+ }
+
+ dev->data->mac_addrs = rte_zmalloc(NULL, RTE_ETHER_ADDR_LEN, 0);
+ if (dev->data->mac_addrs == NULL) {
+ STMMAC_PMD_ERR("probe failed: unable to allocate mac_addrs for %s",
+ pdev->device.name);
+ rte_free(priv);
+ rte_eth_dev_release_port(dev);
+ return -ENOMEM;
+ }
+ for (i = 0; i < RTE_ETHER_ADDR_LEN; i++)
+ dev->data->mac_addrs[0].addr_bytes[i] = 0x02;
+ rte_ether_addr_copy(&dev->data->mac_addrs[0], &priv->mac_addr);
+
+ dev->dev_ops = &stmmac_eth_ops;
+
+ rte_eth_dev_probing_finish(dev);
+
+ return 0;
+}
+
+static int
+stmmac_platform_remove(struct rte_platform_device *pdev)
+{
+ struct rte_eth_dev *dev;
+
+ dev = rte_eth_dev_allocated(pdev->device.name);
+ if (dev == NULL)
+ return 0;
+
+ stmmac_eth_close(dev);
+ rte_free(dev->data->dev_private);
+ rte_eth_dev_release_port(dev);
+
+ return 0;
+}
+
+static struct rte_platform_driver stmmac_platform_drv = {
+ .probe = stmmac_platform_probe,
+ .remove = stmmac_platform_remove,
+};
+
+RTE_PMD_REGISTER_PLATFORM(STMMAC_DRIVER_NAME, stmmac_platform_drv);
+RTE_PMD_REGISTER_ALIAS(STMMAC_DRIVER_NAME, vfio-platform);
+RTE_PMD_REGISTER_PARAM_STRING(STMMAC_DRIVER_NAME,
+ STMMAC_DEVARG_HAS_EMAC_GE_3 "=<0|1> "
+ STMMAC_DEVARG_MDIO_ADDR "=<uint16>");
diff --git a/drivers/net/stmmac/stmmac_logs.h b/drivers/net/stmmac/stmmac_logs.h
new file mode 100644
index 0000000000..37db8e3ea9
--- /dev/null
+++ b/drivers/net/stmmac/stmmac_logs.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2026 Qualcomm Technologies, Inc.
+ */
+
+#ifndef _STMMAC_LOGS_H_
+#define _STMMAC_LOGS_H_
+
+#include <rte_log.h>
+
+extern int stmmac_logtype_pmd;
+#define RTE_LOGTYPE_STMMAC_NET stmmac_logtype_pmd
+
+#define STMMAC_PMD_LOG(level, ...) \
+ RTE_LOG_LINE_PREFIX(level, STMMAC_NET, "%s()", __func__, __VA_ARGS__)
+
+#define STMMAC_PMD_DEBUG(fmt, ...) \
+ STMMAC_PMD_LOG(DEBUG, fmt, ## __VA_ARGS__)
+#define STMMAC_PMD_ERR(fmt, ...) \
+ STMMAC_PMD_LOG(ERR, fmt, ## __VA_ARGS__)
+#define STMMAC_PMD_INFO(fmt, ...) \
+ STMMAC_PMD_LOG(INFO, fmt, ## __VA_ARGS__)
+#define STMMAC_PMD_WARN(fmt, ...) \
+ STMMAC_PMD_LOG(WARNING, fmt, ## __VA_ARGS__)
+
+#endif /* _STMMAC_LOGS_H_ */
diff --git a/drivers/net/stmmac/stmmac_regs.h b/drivers/net/stmmac/stmmac_regs.h
new file mode 100644
index 0000000000..2fac1d9f21
--- /dev/null
+++ b/drivers/net/stmmac/stmmac_regs.h
@@ -0,0 +1,302 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2026 Qualcomm Technologies, Inc.
+ */
+
+#ifndef _STMMAC_REGS_H_
+#define _STMMAC_REGS_H_
+
+#include <stdint.h>
+#include <rte_bitops.h>
+#include <rte_common.h>
+#include <rte_io.h>
+
+/*
+ * Register and bitfield definitions for the Synopsys DWC EQoS GMAC4 core
+ * ("stmmac" / "dwmac4" in the Linux driver family). Offsets and bit positions
+ * follow drivers/net/ethernet/stmicro/stmmac/{dwmac4.h, dwmac4_dma.h,
+ * dwmac4_descs.h, descs.h, stmmac_mdio.c, dwmac4_core.c}.
+ */
+
+/*
+ * Per-channel DMA/MTL register stride. The generic dwmac4 layout is used by
+ * default; platforms with has_emac_ge_3 use a 0x1000 stride.
+ */
+struct stmmac_chan_addrs {
+ uint32_t dma_chan_base;
+ uint32_t dma_chan_stride;
+ uint32_t mtl_chan_base;
+ uint32_t mtl_chan_stride;
+};
+
+/* ---------------------------------------------------------------- */
+/* MAC registers */
+/* ---------------------------------------------------------------- */
+
+#define GMAC_CONFIG 0x00000000
+#define GMAC_PACKET_FILTER 0x00000008
+#define GMAC_HW_FEATURE1 0x00000120
+#define GMAC_MDIO_ADDR 0x00000200
+#define GMAC_MDIO_DATA 0x00000204
+#define GMAC_RXQ_CTRL0 0x000000a0
+#define GMAC_ADDR_HIGH(reg) (0x300 + (reg) * 8)
+#define GMAC_ADDR_LOW(reg) (0x304 + (reg) * 8)
+
+/* GMAC_CONFIG bits */
+#define GMAC_CONFIG_IPC RTE_BIT32(27)
+#define GMAC_CONFIG_ACS RTE_BIT32(20)
+#define GMAC_CONFIG_TE RTE_BIT32(1)
+#define GMAC_CONFIG_RE RTE_BIT32(0)
+
+/* GMAC_PACKET_FILTER bits */
+#define GMAC_PACKET_FILTER_PR RTE_BIT32(0)
+
+/* GMAC_HW_FEATURE1: RX/TX FIFO size encodings */
+#define GMAC_HWF1_RXFIFOSIZE_MASK 0x1fu /* [4:0] */
+#define GMAC_HWF1_TXFIFOSIZE_SHIFT 6 /* [10:6] */
+#define GMAC_HWF1_TXFIFOSIZE_MASK 0x1fu
+
+/* ---------------------------------------------------------------- */
+/* MTL registers */
+/* ---------------------------------------------------------------- */
+
+#define MTL_CHAN_BASE_ADDR 0x00000d00
+#define MTL_CHAN_BASE_OFFSET 0x40
+#define MTL_RXQ_DMA_MAP0 0x00000c30
+
+static inline uint32_t
+stmmac_mtl_chan_base(const struct stmmac_chan_addrs *addrs, uint32_t x)
+{
+ if (addrs != NULL)
+ return addrs->mtl_chan_base + x * addrs->mtl_chan_stride;
+ return MTL_CHAN_BASE_ADDR + x * MTL_CHAN_BASE_OFFSET;
+}
+
+#define MTL_CHAN_TX_OP_MODE(addrs, x) \
+ (stmmac_mtl_chan_base((addrs), (x)))
+#define MTL_CHAN_RX_OP_MODE(addrs, x) \
+ (stmmac_mtl_chan_base((addrs), (x)) + 0x30)
+
+#define MTL_OP_MODE_TSF RTE_BIT32(1)
+#define MTL_OP_MODE_TXQEN RTE_BIT32(3)
+#define MTL_OP_MODE_TXQEN_MASK (0x3u << 2)
+#define MTL_OP_MODE_RSF RTE_BIT32(5)
+#define MTL_OP_MODE_TQS_MASK (0x1ffu << 16)
+#define MTL_OP_MODE_TQS_SHIFT 16
+#define MTL_OP_MODE_RQS_MASK (0x3ffu << 20)
+#define MTL_OP_MODE_RQS_SHIFT 20
+
+/* ---------------------------------------------------------------- */
+/* DMA registers */
+/* ---------------------------------------------------------------- */
+
+#define DMA_BUS_MODE 0x00001000
+#define DMA_SYS_BUS_MODE 0x00001004
+#define DMA_AXI_BUS_MODE 0x00001028
+
+#define DMA_BUS_MODE_SFT_RESET RTE_BIT32(0)
+
+/* DMA_SYS_BUS_MODE bits */
+#define DMA_SYS_BUS_MB RTE_BIT32(14)
+#define DMA_SYS_BUS_AAL RTE_BIT32(12)
+#define DMA_SYS_BUS_EAME RTE_BIT32(11)
+#define DMA_SYS_BUS_FB RTE_BIT32(0)
+
+/* DMA_AXI_BUS_MODE bits */
+#define DMA_AXI_WR_OSR_LMT (0xfu << 24)
+#define DMA_AXI_WR_OSR_LMT_SHIFT 24
+#define DMA_AXI_RD_OSR_LMT (0xfu << 16)
+#define DMA_AXI_RD_OSR_LMT_SHIFT 16
+#define DMA_AXI_BLEN16 RTE_BIT32(3)
+#define DMA_AXI_OSR_MAX 0xf
+
+/* Per-channel DMA registers */
+#define DMA_CHAN_BASE_ADDR 0x00001100
+#define DMA_CHAN_BASE_OFFSET 0x80
+
+static inline uint32_t
+stmmac_dma_chan_base(const struct stmmac_chan_addrs *addrs, uint32_t x)
+{
+ if (addrs != NULL)
+ return addrs->dma_chan_base + x * addrs->dma_chan_stride;
+ return DMA_CHAN_BASE_ADDR + x * DMA_CHAN_BASE_OFFSET;
+}
+
+/*
+ * Offsets relative to a resolved per-channel base (chan_base already holds
+ * mac_base + stmmac_dma_chan_base(addrs, 0)).
+ */
+#define DMA_CHAN_OFF_CONTROL 0x00
+#define DMA_CHAN_OFF_TX_CONTROL 0x04
+#define DMA_CHAN_OFF_RX_CONTROL 0x08
+#define DMA_CHAN_OFF_TX_BASE_ADDR_HI 0x10
+#define DMA_CHAN_OFF_TX_BASE_ADDR 0x14
+#define DMA_CHAN_OFF_RX_BASE_ADDR_HI 0x18
+#define DMA_CHAN_OFF_RX_BASE_ADDR 0x1c
+#define DMA_CHAN_OFF_RX_END_ADDR_HI 0x24
+#define DMA_CHAN_OFF_TX_END_ADDR 0x20
+#define DMA_CHAN_OFF_RX_END_ADDR 0x28
+#define DMA_CHAN_OFF_TX_RING_LEN 0x2c
+#define DMA_CHAN_OFF_RX_RING_LEN 0x30
+#define DMA_CHAN_OFF_CURTXD 0x44
+#define DMA_CHAN_OFF_STATUS 0x60
+
+#define DMA_CHAN_TX_CONTROL_OSP RTE_BIT32(4)
+
+/* DMA_CHAN_CONTROL / TX / RX control bits */
+#define DMA_CONTROL_PBLX8 RTE_BIT32(16)
+#define DMA_CONTROL_ST RTE_BIT32(0)
+#define DMA_CONTROL_RPF RTE_BIT32(31)
+#define DMA_CONTROL_SR RTE_BIT32(0)
+#define DMA_CONTROL_PBL_SHIFT 16
+#define DMA_CONTROL_PBL_MASK (0x3fu << DMA_CONTROL_PBL_SHIFT)
+#define DMA_CONTROL_RBSZ_SHIFT 1
+#define DMA_CONTROL_RBSZ_MASK (0x3fffu << DMA_CONTROL_RBSZ_SHIFT)
+
+/* DMA_CHAN_STATUS bits */
+#define DMA_CHAN_STATUS_TI RTE_BIT32(0)
+#define DMA_CHAN_STATUS_RI RTE_BIT32(6)
+#define DMA_CHAN_STATUS_RBU RTE_BIT32(7)
+
+/*
+ * The Shikra EMAC is not DMA-coherent: CPU writes to descriptors must be
+ * cleaned to the point of coherency before the DMA reads them, and lines must
+ * be invalidated before the CPU reads DMA write-back.
+ */
+#define STMMAC_CACHE_LINE 64
+/* 16-byte descriptors, four per cache line; the ring base is 64-byte aligned */
+#define STMMAC_DESCS_PER_LINE (STMMAC_CACHE_LINE / 16)
+
+/*
+ * Cache maintenance for a non-coherent DMA (arm64). Clean writes CPU-written
+ * lines back so the DMA reads them; flush also discards CPU copies before the
+ * CPU reads DMA write-back. dsb orders the maintenance before the DMA sees it.
+ */
+static inline void
+stmmac_dcache_clean(const void *addr, size_t len)
+{
+ uintptr_t p = (uintptr_t)addr & ~(uintptr_t)(STMMAC_CACHE_LINE - 1);
+ uintptr_t end = (uintptr_t)addr + len;
+
+ for (; p < end; p += STMMAC_CACHE_LINE)
+ asm volatile("dc cvac, %0" : : "r"(p) : "memory");
+ asm volatile("dsb sy" : : : "memory");
+}
+
+static inline void
+stmmac_dcache_flush(const void *addr, size_t len)
+{
+ uintptr_t p = (uintptr_t)addr & ~(uintptr_t)(STMMAC_CACHE_LINE - 1);
+ uintptr_t end = (uintptr_t)addr + len;
+
+ for (; p < end; p += STMMAC_CACHE_LINE)
+ asm volatile("dc civac, %0" : : "r"(p) : "memory");
+ asm volatile("dsb sy" : : : "memory");
+}
+
+static inline void
+stmmac_dcache_inval_line(const void *addr)
+{
+ asm volatile("dc civac, %0" : : "r"(addr) : "memory");
+ rte_io_rmb();
+}
+
+/* ---------------------------------------------------------------- */
+/* TX descriptor bits */
+/* ---------------------------------------------------------------- */
+
+/* TDES2 (read format) */
+#define TDES2_BUFFER1_SIZE_MASK 0x3fffu
+#define TDES2_INTERRUPT_ON_COMPLETION RTE_BIT32(31)
+
+/* TDES3 (read format) */
+#define TDES3_PACKET_SIZE_MASK 0x7fffu
+#define TDES3_CHECKSUM_INSERTION_MASK (0x3u << 16)
+#define TDES3_CHECKSUM_INSERTION_SHIFT 16
+
+/* Insert IP header and pseudo-header checksums */
+#define TX_CIC_FULL 3
+
+/* TDES3 (write-back and common) */
+#define TDES3_ERROR_SUMMARY RTE_BIT32(15)
+#define TDES3_LAST_DESCRIPTOR RTE_BIT32(28)
+#define TDES3_FIRST_DESCRIPTOR RTE_BIT32(29)
+#define TDES3_CONTEXT_TYPE RTE_BIT32(30)
+#define TDES3_OWN RTE_BIT32(31)
+
+/* ---------------------------------------------------------------- */
+/* RX descriptor bits */
+/* ---------------------------------------------------------------- */
+
+/* RDES1 (write-back format) */
+#define RDES1_IP_HDR_ERROR RTE_BIT32(3)
+#define RDES1_IPV4_HEADER RTE_BIT32(4)
+#define RDES1_IPV6_HEADER RTE_BIT32(5)
+#define RDES1_IP_CSUM_BYPASSED RTE_BIT32(6)
+#define RDES1_IP_CSUM_ERROR RTE_BIT32(7)
+
+/* RDES3 (write-back format) */
+#define RDES3_PACKET_SIZE_MASK 0x7fffu
+#define RDES3_ERROR_SUMMARY RTE_BIT32(15)
+#define RDES3_OVERFLOW_ERROR RTE_BIT32(21)
+#define RDES3_CRC_ERROR RTE_BIT32(24)
+#define RDES3_LAST_DESCRIPTOR RTE_BIT32(28)
+#define RDES3_FIRST_DESCRIPTOR RTE_BIT32(29)
+#define RDES3_CONTEXT_DESCRIPTOR RTE_BIT32(30)
+
+/* RDES3 (read format) */
+#define RDES3_BUFFER1_VALID_ADDR RTE_BIT32(24)
+#define RDES3_INT_ON_COMPLETION_EN RTE_BIT32(30)
+#define RDES3_OWN RTE_BIT32(31)
+
+/* ---------------------------------------------------------------- */
+/* MDIO */
+/* ---------------------------------------------------------------- */
+
+#define MII_BUSY RTE_BIT32(0)
+#define MII_DATA_MASK 0xffffu
+
+#define MII_GMAC4_GOC_SHIFT 2
+#define MII_GMAC4_WRITE (1 << MII_GMAC4_GOC_SHIFT)
+#define MII_GMAC4_READ (3 << MII_GMAC4_GOC_SHIFT)
+
+#define MII_ADDR_SHIFT 21
+#define MII_ADDR_MASK (0x1fu << MII_ADDR_SHIFT)
+#define MII_REG_SHIFT 16
+#define MII_REG_MASK (0x1fu << MII_REG_SHIFT)
+
+/* IEEE 802.3 clause-22 registers */
+#define STMMAC_MII_BMSR 0x01
+#define STMMAC_MII_BMSR_LSTATUS RTE_BIT32(2)
+
+/*
+ * DP83867 PHY registers (clause-22 direct access).
+ * PHYSTS bits: 15:14 speed (11 = 1000, 10 = 100, 01 = 10 Mb/s),
+ * 13 duplex (1 = full), 10 link (not latched).
+ */
+#define DP83867_PHYSTS 0x11
+#define DP83867_PHYSTS_LINK_BIT RTE_BIT32(10)
+#define DP83867_PHYSTS_SPEED_MASK (RTE_BIT32(15) | RTE_BIT32(14))
+#define DP83867_PHYSTS_SPEED_1G RTE_BIT32(15)
+#define DP83867_PHYSTS_SPEED_100M RTE_BIT32(14)
+#define DP83867_PHYSTS_DUPLEX_BIT RTE_BIT32(13)
+
+/* ---------------------------------------------------------------- */
+/* Channel stride presets */
+/* ---------------------------------------------------------------- */
+
+#define STMMAC_CHAN_ADDRS_GENERIC_INIT { \
+ .dma_chan_base = DMA_CHAN_BASE_ADDR, \
+ .dma_chan_stride = DMA_CHAN_BASE_OFFSET, \
+ .mtl_chan_base = MTL_CHAN_BASE_ADDR, \
+ .mtl_chan_stride = MTL_CHAN_BASE_OFFSET, \
+}
+
+/* has_emac_ge_3 layout (dwmac-qcom-ethqos.c emac_v4_0_0_data) */
+#define STMMAC_CHAN_ADDRS_EMAC_GE3_INIT { \
+ .dma_chan_base = 0x8100, \
+ .dma_chan_stride = 0x1000, \
+ .mtl_chan_base = 0x8000, \
+ .mtl_chan_stride = 0x1000, \
+}
+
+#endif /* _STMMAC_REGS_H_ */
diff --git a/drivers/net/stmmac/stmmac_rxtx.c b/drivers/net/stmmac/stmmac_rxtx.c
new file mode 100644
index 0000000000..61f7249759
--- /dev/null
+++ b/drivers/net/stmmac/stmmac_rxtx.c
@@ -0,0 +1,416 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2026 Qualcomm Technologies, Inc.
+ */
+
+#include <rte_branch_prediction.h>
+#include <rte_common.h>
+#include <rte_cycles.h>
+#include <rte_io.h>
+#include <rte_mbuf.h>
+
+#include "stmmac.h"
+#include "stmmac_regs.h"
+#include "stmmac_logs.h"
+
+#define STMMAC_RX_BUNDLE 32
+#define STMMAC_TX_BUNDLE 32
+
+/* MDIO busy-poll bound, matching stmmac_mdio.c (10000 x 1 us) */
+#define STMMAC_MDIO_TIMEOUT_US 10000
+
+/* Index of the TX descriptor the DMA is working on: everything before it is sent. */
+static inline uint16_t
+stmmac_tx_dma_slot(struct stmmac_tx_queue *txq)
+{
+ uint32_t cur = rte_read32((uint8_t *)txq->chan_base + DMA_CHAN_OFF_CURTXD);
+
+ return (uint16_t)(((cur - (uint32_t)txq->tx_ring_iova) /
+ sizeof(struct stmmac_desc)) % txq->nb_desc);
+}
+
+/*
+ * MDIO clause-22 read. Uses the GMAC4 opcode and the address/register field
+ * layout from dwmac4_core.c. The CSR clock divider is left at zero.
+ */
+uint16_t
+stmmac_mdio_read(void *mac_base, uint16_t mdio_addr, uint16_t phyreg)
+{
+ void *addr_reg = (uint8_t *)mac_base + GMAC_MDIO_ADDR;
+ void *data_reg = (uint8_t *)mac_base + GMAC_MDIO_DATA;
+ uint32_t value;
+ unsigned int timeout = STMMAC_MDIO_TIMEOUT_US;
+
+ while ((rte_read32(addr_reg) & MII_BUSY) && timeout > 0) {
+ rte_delay_us(1);
+ timeout--;
+ }
+ if (timeout == 0) {
+ STMMAC_PMD_ERR("MDIO busy timeout before read");
+ return 0xffff;
+ }
+
+ value = MII_GMAC4_READ | MII_BUSY;
+ value |= ((uint32_t)mdio_addr << MII_ADDR_SHIFT) & MII_ADDR_MASK;
+ value |= ((uint32_t)phyreg << MII_REG_SHIFT) & MII_REG_MASK;
+
+ rte_write32(value, addr_reg);
+
+ timeout = STMMAC_MDIO_TIMEOUT_US;
+ while ((rte_read32(addr_reg) & MII_BUSY) && timeout > 0) {
+ rte_delay_us(1);
+ timeout--;
+ }
+ if (timeout == 0) {
+ STMMAC_PMD_ERR("MDIO busy timeout after read");
+ return 0xffff;
+ }
+
+ return (uint16_t)(rte_read32(data_reg) & MII_DATA_MASK);
+}
+
+void
+stmmac_mdio_write(void *mac_base, uint16_t mdio_addr, uint16_t phyreg,
+ uint16_t phydata)
+{
+ void *addr_reg = (uint8_t *)mac_base + GMAC_MDIO_ADDR;
+ void *data_reg = (uint8_t *)mac_base + GMAC_MDIO_DATA;
+ uint32_t value;
+ unsigned int timeout = STMMAC_MDIO_TIMEOUT_US;
+
+ while ((rte_read32(addr_reg) & MII_BUSY) && timeout > 0) {
+ rte_delay_us(1);
+ timeout--;
+ }
+ if (timeout == 0) {
+ STMMAC_PMD_ERR("MDIO busy timeout before write");
+ return;
+ }
+
+ rte_write32(phydata, data_reg);
+
+ value = MII_GMAC4_WRITE | MII_BUSY;
+ value |= ((uint32_t)mdio_addr << MII_ADDR_SHIFT) & MII_ADDR_MASK;
+ value |= ((uint32_t)phyreg << MII_REG_SHIFT) & MII_REG_MASK;
+
+ rte_write32(value, addr_reg);
+
+ timeout = STMMAC_MDIO_TIMEOUT_US;
+ while ((rte_read32(addr_reg) & MII_BUSY) && timeout > 0) {
+ rte_delay_us(1);
+ timeout--;
+ }
+ if (timeout == 0)
+ STMMAC_PMD_ERR("MDIO busy timeout after write");
+}
+
+/*
+ * Refill 'cleaned_cnt' RX descriptors starting at rx_refill. Mbufs are
+ * allocated in bulk, descriptors are written, and each cache line of
+ * descriptors is cleaned once. The DMA doorbell moves to the last descriptor
+ * handed back. Returns the number refilled; a short count means the mbuf pool
+ * ran out, and the remaining slots are retried on the next burst.
+ */
+static uint16_t
+stmmac_refill_rx_ring(struct stmmac_rx_queue *rxq, uint16_t cleaned_cnt)
+{
+ struct rte_mbuf *m[STMMAC_RX_BUNDLE];
+ struct stmmac_desc *grp_start = NULL;
+ uint16_t i = rxq->rx_refill;
+ uint16_t j;
+ uint16_t k = STMMAC_RX_BUNDLE;
+ uint16_t grp_cnt = 0;
+
+ for (j = 0; j < cleaned_cnt; j++) {
+ struct stmmac_desc *desc = &rxq->rx_ring[i];
+ struct rte_mbuf *mb;
+ uint64_t iova;
+
+ /* bulk allocation, sized to the descriptors still to refill */
+ if (k == STMMAC_RX_BUNDLE) {
+ uint16_t m_cnt = RTE_MIN(cleaned_cnt - j, STMMAC_RX_BUNDLE);
+
+ if (rte_pktmbuf_alloc_bulk(rxq->mb_pool, m, m_cnt)) {
+ rxq->rx_nombuf++;
+ break;
+ }
+ k = 0;
+ }
+ mb = m[k++];
+
+ rxq->sw_ring[i] = mb;
+ /* drop stale CPU lines over the buffer the DMA is about to write */
+ stmmac_dcache_flush(rte_pktmbuf_mtod(mb, void *),
+ rte_pktmbuf_data_room_size(rxq->mb_pool) -
+ RTE_PKTMBUF_HEADROOM);
+
+ /* full 36-bit buffer address: the mbufs live above 4 GiB */
+ iova = rte_pktmbuf_iova(mb);
+ desc->des[0] = (uint32_t)iova;
+ desc->des[1] = (uint32_t)(iova >> 32);
+ desc->des[2] = 0;
+ rte_io_wmb();
+ desc->des[3] = RDES3_OWN | RDES3_BUFFER1_VALID_ADDR |
+ RDES3_INT_ON_COMPLETION_EN;
+
+ /* clean the descriptors one cache line at a time */
+ if (grp_cnt == 0)
+ grp_start = desc;
+ grp_cnt++;
+ if (grp_cnt == STMMAC_DESCS_PER_LINE || i == rxq->nb_desc - 1) {
+ stmmac_dcache_clean(grp_start, grp_cnt * sizeof(*desc));
+ grp_cnt = 0;
+ }
+
+ i = (i + 1) % rxq->nb_desc;
+ }
+
+ /* clean the partial group left by the last descriptor */
+ if (grp_cnt)
+ stmmac_dcache_clean(grp_start, grp_cnt * sizeof(struct stmmac_desc));
+
+ rxq->rx_refill = i;
+
+ if (j > 0) {
+ uint16_t last = (i + rxq->nb_desc - 1) % rxq->nb_desc;
+ uint64_t tail_iova = rxq->rx_ring_iova +
+ (uint64_t)last * sizeof(struct stmmac_desc);
+
+ /*
+ * Clear RBU if the DMA suspended on an unavailable descriptor. This
+ * is a write, not a status read: a register read stalls the core.
+ * The DMA only suspends after it has used every slot refilled so
+ * far, so the burst that refills again is where RBU is cleared.
+ */
+ rte_write32(DMA_CHAN_STATUS_RBU,
+ (uint8_t *)rxq->chan_base + DMA_CHAN_OFF_STATUS);
+
+ /*
+ * The doorbell is the last descriptor handed back. Writing the
+ * refill index instead would make RXTAIL equal CURRXD, which the
+ * DMA treats as an empty ring. The 36-bit tail is written HI first.
+ */
+ rte_write32((uint32_t)(tail_iova >> 32),
+ (uint8_t *)rxq->chan_base + DMA_CHAN_OFF_RX_END_ADDR_HI);
+ rte_io_wmb();
+ rte_write32((uint32_t)tail_iova,
+ (uint8_t *)rxq->chan_base + DMA_CHAN_OFF_RX_END_ADDR);
+ }
+
+ return j;
+}
+
+uint16_t
+stmmac_rx_pkt_burst_nc(void *rx_queue, struct rte_mbuf **rx_pkts,
+ uint16_t nb_pkts)
+{
+ struct stmmac_rx_queue *rxq = rx_queue;
+ uint16_t nb_rx = 0;
+ uint16_t start = rxq->rx_tail;
+ uint16_t tail = start;
+ uint16_t pending;
+
+ while (nb_rx < nb_pkts) {
+ struct stmmac_desc *desc = &rxq->rx_ring[tail];
+ struct rte_mbuf *rxm;
+ uint32_t des3;
+ uint32_t pkt_len;
+
+ /* slot is waiting for a refill (mbuf allocation failed) */
+ if (rxq->sw_ring[tail] == NULL)
+ break;
+
+ /*
+ * Shikra is not DMA-coherent: invalidate the descriptor line once
+ * per cache line, at the start of the burst and at each line
+ * boundary. A stale OWN bit on a later slot only defers that frame
+ * to the next burst.
+ */
+ if (tail == start || tail % STMMAC_DESCS_PER_LINE == 0)
+ stmmac_dcache_inval_line(desc);
+ des3 = rte_read32_relaxed(&desc->des[3]);
+
+ if (des3 & RDES3_OWN)
+ break;
+
+ if (des3 & (RDES3_CONTEXT_DESCRIPTOR | RDES3_ERROR_SUMMARY)) {
+ /* not a frame to deliver: free its buffer, refill allocates a new one */
+ if (des3 & RDES3_ERROR_SUMMARY)
+ rxq->rx_errors++;
+ rte_pktmbuf_free(rxq->sw_ring[tail]);
+ rxq->sw_ring[tail] = NULL;
+ tail = (tail + 1) % rxq->nb_desc;
+ continue;
+ }
+
+ rxm = rxq->sw_ring[tail];
+ rxq->sw_ring[tail] = NULL;
+ pkt_len = des3 & RDES3_PACKET_SIZE_MASK;
+
+ /* drop stale CPU lines over the received payload */
+ stmmac_dcache_flush((uint8_t *)rxm->buf_addr + RTE_PKTMBUF_HEADROOM,
+ pkt_len);
+
+ rxm->data_off = RTE_PKTMBUF_HEADROOM;
+ rxm->pkt_len = pkt_len;
+ rxm->data_len = pkt_len;
+ rxm->port = rxq->port_id;
+ rxm->nb_segs = 1;
+ rxm->next = NULL;
+ rxm->ol_flags = 0;
+
+ if (desc->des[1] & (RDES1_IPV4_HEADER | RDES1_IPV6_HEADER)) {
+ if (desc->des[1] & RDES1_IP_CSUM_ERROR)
+ rxm->ol_flags |= RTE_MBUF_F_RX_IP_CKSUM_BAD;
+ else
+ rxm->ol_flags |= RTE_MBUF_F_RX_IP_CKSUM_GOOD;
+ }
+
+ rx_pkts[nb_rx++] = rxm;
+ rxq->rx_packets++;
+ rxq->rx_bytes += pkt_len;
+
+ tail = (tail + 1) % rxq->nb_desc;
+ }
+
+ rxq->rx_tail = tail;
+
+ /* refill what was consumed, plus any slots a failed allocation left pending */
+ pending = (tail + rxq->nb_desc - rxq->rx_refill) % rxq->nb_desc;
+ if (pending)
+ stmmac_refill_rx_ring(rxq, pending);
+
+ return nb_rx;
+}
+
+/*
+ * Reclaim sent TX frames in bulk, up to the descriptor the DMA is working on.
+ * Everything before that slot has been transmitted.
+ */
+static uint16_t
+stmmac_clean_tx_ring(struct stmmac_tx_queue *txq)
+{
+ struct rte_mbuf *m[STMMAC_TX_BUNDLE];
+ uint16_t hwci = stmmac_tx_dma_slot(txq);
+ uint16_t i = txq->tx_free;
+ uint16_t n = 0;
+ uint16_t total = 0;
+
+ while (i != hwci && txq->sw_ring[i] != NULL) {
+ if (i == txq->tx_free || i % STMMAC_DESCS_PER_LINE == 0)
+ stmmac_dcache_inval_line(&txq->tx_ring[i]);
+ if (rte_read32_relaxed(&txq->tx_ring[i].des[3]) & TDES3_ERROR_SUMMARY)
+ txq->tx_errors++;
+
+ m[n++] = txq->sw_ring[i];
+ txq->sw_ring[i] = NULL;
+ if (n == STMMAC_TX_BUNDLE) {
+ rte_pktmbuf_free_bulk(m, n);
+ total += n;
+ n = 0;
+ }
+ i = (i + 1) % txq->nb_desc;
+ }
+
+ if (n) {
+ rte_pktmbuf_free_bulk(m, n);
+ total += n;
+ }
+ txq->tx_free = i;
+ return total;
+}
+
+/* Clean the descriptor lines that hold 'cnt' slots from 'first', once, before the doorbell */
+static void
+stmmac_tx_clean_range(struct stmmac_tx_queue *txq, uint16_t first, uint16_t cnt)
+{
+ uint16_t s = first - first % STMMAC_DESCS_PER_LINE;
+ uint16_t left = cnt + (first - s);
+
+ left = (left + STMMAC_DESCS_PER_LINE - 1) / STMMAC_DESCS_PER_LINE *
+ STMMAC_DESCS_PER_LINE;
+ if (left > txq->nb_desc)
+ left = txq->nb_desc;
+
+ while (left) {
+ uint16_t run = RTE_MIN((uint16_t)(txq->nb_desc - s), left);
+
+ stmmac_dcache_clean(&txq->tx_ring[s], run * sizeof(struct stmmac_desc));
+ s = (s + run) % txq->nb_desc;
+ left -= run;
+ }
+}
+
+uint16_t
+stmmac_tx_pkt_burst_nc(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct stmmac_tx_queue *txq = tx_queue;
+ uint16_t nb_tx = 0;
+ uint16_t first = txq->tx_tail;
+ uint16_t tail = first;
+
+ stmmac_clean_tx_ring(txq);
+
+ /* drop the CPU copy of the first line before filling into it */
+ stmmac_dcache_inval_line(&txq->tx_ring[first - first % STMMAC_DESCS_PER_LINE]);
+
+ while (nb_tx < nb_pkts) {
+ struct stmmac_desc *desc = &txq->tx_ring[tail];
+ struct rte_mbuf *txm = tx_pkts[nb_tx];
+ uint32_t tdes3;
+ uint64_t iova;
+
+ if (unlikely(txm->nb_segs != 1)) {
+ STMMAC_PMD_ERR("multi-segment mbuf not supported");
+ break;
+ }
+
+ /* ring full: this slot still holds a frame the DMA has not finished */
+ if (txq->sw_ring[tail] != NULL) {
+ txq->tx_ring_full++;
+ break;
+ }
+
+ /* entering a new line: drop the CPU copy before writing into it */
+ if (tail != first && tail % STMMAC_DESCS_PER_LINE == 0)
+ stmmac_dcache_inval_line(desc);
+
+ /* the CPU-written payload must reach memory before the DMA reads it */
+ stmmac_dcache_clean(rte_pktmbuf_mtod(txm, void *),
+ rte_pktmbuf_data_len(txm));
+
+ iova = rte_pktmbuf_iova(txm);
+ desc->des[0] = (uint32_t)iova;
+ desc->des[1] = (uint32_t)(iova >> 32);
+ desc->des[2] = rte_pktmbuf_data_len(txm) & TDES2_BUFFER1_SIZE_MASK;
+
+ tdes3 = rte_pktmbuf_pkt_len(txm) & TDES3_PACKET_SIZE_MASK;
+ tdes3 |= TDES3_FIRST_DESCRIPTOR | TDES3_LAST_DESCRIPTOR;
+ if (txm->ol_flags & (RTE_MBUF_F_TX_IP_CKSUM | RTE_MBUF_F_TX_L4_MASK))
+ tdes3 |= (TX_CIC_FULL << TDES3_CHECKSUM_INSERTION_SHIFT);
+
+ txq->sw_ring[tail] = txm;
+
+ rte_io_wmb();
+ desc->des[3] = tdes3 | TDES3_OWN;
+
+ txq->tx_packets++;
+ txq->tx_bytes += rte_pktmbuf_pkt_len(txm);
+
+ tail = (tail + 1) % txq->nb_desc;
+ nb_tx++;
+ }
+
+ if (nb_tx > 0) {
+ uint64_t tail_iova = txq->tx_ring_iova +
+ (uint64_t)tail * sizeof(struct stmmac_desc);
+
+ /* the written descriptor lines are cleaned once, then the doorbell moves */
+ stmmac_tx_clean_range(txq, first, nb_tx);
+ txq->tx_tail = tail;
+ rte_write32((uint32_t)tail_iova,
+ (uint8_t *)txq->chan_base + DMA_CHAN_OFF_TX_END_ADDR);
+ }
+
+ return nb_tx;
+}
--
2.34.1
More information about the dev
mailing list