DPDK 26.03: mlx5 switchdev mode with primary/secondary processes and ethdev hotplug synchronization

[Dylan Schneider]

Hello all,
I am looking for guidance on using the mlx5 PMD in switchdev mode with the DPDK primary/secondary process model.
Environment

  *
DPDK 26.03
  *
mlx5 NIC
  *
NIC configured in switchdev mode
  *
One DPDK primary process and one DPDK secondary process
  *
Both applications use the same PCI allow-list containing the PF and VFs
  *
The PF and VFs remain bound to the mlx5 kernel driver
  *
Both processes use the same DPDK multi-process configuration and shared-memory prefix

The primary successfully probes the PF and available representor ports with the 'representor' devarg. However, the secondary does not successfully probe the PF or representor devices, even though the PF and VFs are included in the allow-list. Consequently, those devices are absent from the secondary’s ethdev list and do not appear when iterating with RTE_FOREACH_DEV after initialization.
The server we are running has other applications that require the NIC to be switchdev mode. The PF is the E-Switch master, while the secondary processes traffic through a VF ethdev. The main question is how the PF, VF, and representor ethdevs are expected to be managed when some devices are visible only to the primary.
Our current understanding is that:

  *
The primary owns the PF and the E-Switch master.
  *
The PF may not be independently attachable or visible to the secondary.
  *
PF-specific operations, such as creating or destroying E-Switch transfer rules, should be performed by the primary.
  *
The secondary may process traffic through a VF ethdev, while control-plane operations involving the PF are handled by the primary.

Could someone confirm whether this is the expected architecture for mlx5 switchdev with DPDK primary/secondary processes?
Primary/secondary hotplug behavior
I reviewed the DPDK 26.03 hotplug implementation and confirmed that primary/secondary synchronization is performed for both hotplug add and remove operations.
For hotplug add, the primary probes the device locally and then sends an attach request to all registered secondary processes. If communication fails or a secondary cannot attach the device, the primary rolls back the local attach and notifies the secondaries of the rollback.
For hotplug remove, the primary first sends a synchronous remove request through the EAL multiprocess channel to all registered secondaries. Only after that coordination does it proceed with the local bus-specific device removal. This generic synchronization is part of the common EAL hotplug path and is not dependent on the bus or PMD.
The mlx5 PMD also performs its own multi-process coordination during device stop and close operations. Therefore, removing an mlx5 device may involve both mlx5-specific notifications and the generic EAL hotplug notification.
This creates a problem in our configuration:

  *
The primary can probe the PF and create representors.
  *
The secondary cannot probe or attach to the PF or representors.
  *
A primary hotplug add still attempts to attach the device in every registered secondary.
  *
A primary hotplug remove still attempts to remove the device from every registered secondary.
  *
The secondary cannot successfully mirror these operations because the device does not exist in its ethdev view.
  *
As a result, the primary hotplug operation can fail or be rolled back because the secondary cannot attach to or remove the device.

This appears to be a mismatch between the generic primary/secondary hotplug model and the mlx5 switchdev model, where the PF and representors may be primary-only resources. Managing ethdevs in the primary application always has implications in secondary, so how is this supposed to be handled in switchdev mode when secondary cannot even probe the PF ports?


Telemetry issue
There is also a related issue involving the DPDK telemetry thread.
The telemetry thread sends an ethdev/info request for port 0. Port 0 (PF E-switch master) has a valid MAC address in our application. However, in the failure path we observed, format_ether_addr() is called with a NULL pointer while formatting the telemetry response. The helper dereferences or formats the address pointer without first checking whether it is NULL, resulting in a segmentation fault.
Would it be appropriate for format_ether_addr() to handle a NULL pointer defensively—for example, by formatting the address as all zeroes—or should the caller guarantee that the pointer is always valid and return an error if it is not?
A possible defensive behavior would be conceptually similar to:

if (addr == NULL) {
    /* Format as 00:00:00:00:00:00,
     * or report an invalid address.
     */
}

I would appreciate feedback on:

  *   Whether a NULL MAC-address pointer is expected to be possible in the ethdev/info telemetry path
  *   Whether a NULL check and all-zero MAC formatting would be acceptable upstream behavior

Regards,
Dylan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mails.dpdk.org/archives/users/attachments/20260814/03249d15/attachment-0001.htm>