[EXT] Re: [PATCH v4-S3 52/65] net/dpaa2: bind per-queue DPIO for rx-queue interrupts
Jun Yang
jun.yang at nxp.com
Thu Nov 19 14:30:21 CET 2026
+dev at dpdk.org
NXP Confidential
From: Jun Yang
Sent: Thursday, November 19, 2026 6:35 PM
To: Vincent JARDIN <vjardin at iliad-free.fr>; Maxime Leroy <maxime at leroys.fr>
Cc: Prashant Gupta <prashant.gupta_3 at nxp.com>; Hemant Agrawal <hemant.agrawal at nxp.com>; Francois Massot-Pellet <francois.massot-pellet at nxp.com>
Subject: RE: [EXT] Re: [PATCH v4-S3 52/65] net/dpaa2: bind per-queue DPIO for rx-queue interrupts
Hi Maxime,
Thanks for your feedback.
The previously posted patch series is currently being reworked. We have received a significant amount of feedback from the community. Some comments point out logical issues within the patches themselves, some identify rework or overlap between patches, and others suggest improvements to the code structure. In addition, we have found several issues through our internal reviews and testing. As a result, we plan to resend the series, with some patches reordered, revised, or superseded by updated versions. I will notify you in advance of re-sending to dpdk.
Now let me reply to the questions you raised:
1. The PMD changes the CPU frequency governor
I will look for a better solution to reduce the core wake-up latency and will drop this patch for now. Some customers have raised concerns about the wake-up time.
1. The Rx burst uses the lcore ID as a Linux CPU number.
This is a valid issue, and I have already worked out a solution locally.
1. The implementation also drains any frames being pulled on the previous core before continuing processing on the new one.
This has been worked out and will share you patches in advance of upstreaming.
1. One DPIO per Rx queue is expensive
I understand your concern; we are simply implementing the semantics defined by the DPDK API. Since this is a per-RXQ interrupt, we do not expect interrupt sharing. From an upstream perspective, we also prefer not to introduce additional constraints or modify common code solely to accommodate our driver. For example, we do not think users should be required to perform interrupt operations on a datapath lcore, nor should they have to follow a specific sequence of operations for the driver to work correctly. Currently, our driver can be supported without any changes to the common/example code.
Regarding the resource concern you mentioned, we can certainly provide other options to address it. My point is simply that not all users require so many resources, so the implementation should not impose such requirements on everyone. However, we will certainly provide an upstreaming alternative option to accommodate your use case.
From: Vincent JARDIN <vjardin at iliad-free.fr<mailto:vjardin at iliad-free.fr>>
Sent: Thursday, November 19, 2026 6:07 PM
To: Maxime Leroy <maxime at leroys.fr<mailto:maxime at leroys.fr>>; Jun Yang <jun.yang at nxp.com<mailto:jun.yang at nxp.com>>
Cc: Prashant Gupta <prashant.gupta_3 at nxp.com<mailto:prashant.gupta_3 at nxp.com>>; Hemant Agrawal <hemant.agrawal at nxp.com<mailto:hemant.agrawal at nxp.com>>; Francois Massot-Pellet <francois.massot-pellet at nxp.com<mailto:francois.massot-pellet at nxp.com>>
Subject: Re: [EXT] Re: [PATCH v4-S3 52/65] net/dpaa2: bind per-queue DPIO for rx-queue interrupts
Caution: This is an external email. Please take care when clicking links or opening attachments. When in doubt, report the message using the 'Report this email' button
+Francois for information
________________________________
From: Maxime Leroy <maxime at leroys.fr<mailto:maxime at leroys.fr>>
Sent: Thursday, 19 November 2026 10:49:52
To: Jun Yang <jun.yang at nxp.com<mailto:jun.yang at nxp.com>>
Cc: Prashant Gupta <prashant.gupta_3 at nxp.com<mailto:prashant.gupta_3 at nxp.com>>; Hemant Agrawal <hemant.agrawal at nxp.com<mailto:hemant.agrawal at nxp.com>>; Vincent JARDIN <vjardin at iliad-free.fr<mailto:vjardin at iliad-free.fr>>
Subject: Re: [EXT] Re: [PATCH v4-S3 52/65] net/dpaa2: bind per-queue DPIO for rx-queue interrupts
Hi Jun,
My review was sent to dev at dpdk.org<mailto:dev at dpdk.org>, and your reply came to me only.
Please answer on the mailing list, not in private: the other reviewers
and the maintainers need to see your answers. If you cannot disclose
something, for example the maximum number of DPIOs, just leave it out
of your public answer.
On Fri, Nov 13, 2026 at 2:34 PM Jun Yang <jun.yang at nxp.com<mailto:jun.yang at nxp.com>> wrote:
>
> Hi Maxime,
> We are still working on categorizing and improving our upstream patches. Regarding the issue you raised, please see my comments below:
> 1)The PMD changes the CPU frequency governor
> The CPU frequency governor is controlled through a devarg and is disabled by default. Users must explicitly enable this feature; otherwise, the driver does not modify the CPU frequency governor.
I do not find this devarg in v4-S3. 47/65 changes the governor from
dpaa2_event_portal_io_prepare() in the eventdev driver. 52/65 moves the
call to dpaa2_dpio_intr_thread_affinity() and also calls it from the Rx
bursts. There is no condition:
ret = dpaa2_dpio_cpufreq_governor(dpio_dev->cpu_affine, false);
A devarg does not fix the problem anyway. Please remove this code. The
design is wrong:
- The governor belongs to the CPU, not to the port. The admin and
the application can change it too. On LX2160A it is even shared
by the two cores of a cluster.
- DPDK already has rte_power for this. It has no qoriq_cpufreq
backend today: adding one in drivers/power is the right fix.
- A user who wants "performance" can set it with one command
before starting the application.
And the implementation is broken:
1. cpu_affine is a DPDK lcore id, not a Linux CPU number (see
point 2), so the PMD can change the governor of another CPU.
2. The save is per CPU, but the policy is per cluster. Queue 0 on
CPU 2 saves "schedutil". Queue 1 on CPU 3 then reads
"performance" and saves that. After close, the cluster stays
in "performance".
3. No reference count: two ports polled by the same CPU, closing
the first one restores the governor under the second one.
4. A queue moves from CPU A to CPU B: only B is restored.
5. A queue that was never polled has cpu_affine = -1: closing the
port restores every CPU that the PMD changed, including those
used by other ports.
6. The governor is written through sysfs inside rte_eth_rx_burst().
>
> 2)The Rx burst uses the lcore ID as a Linux CPU number
> Rx queues can be migrated dynamically between cores. After each migration, the driver updates the interrupt affinity and the stash core accordingly.
My concern is not the migration. It is the number given to the kernel:
int ret, num_rx = 0, cpu = rte_lcore_id();
[...]
dpio->cpu_affine = cpu;
dpaa2_dpio_intr_thread_affinity(dpio);
cpu_affine is a DPDK lcore id, but it is used as a Linux CPU number for
the IRQ affinity, the stashing destination and the governor. Example
with --lcores '1 at 12': lcore 1 runs on CPU 12, and all three are set for
CPU 1.
grout is in this case: each worker is a thread pinned to its CPU, then
it calls rte_thread_register(), and EAL gives it the first free lcore
id, whatever its CPU.
dpaa2_get_core_id() is in the same file and returns the CPU of the
calling thread. Please use it in dpaa2_dpio_intr_thread_affinity().
This code also still runs inside rte_eth_rx_burst(): it parses
/proc/interrupts, writes /proc/irq/N/smp_affinity_list and sends an MC
command. Please set the IRQ affinity in
dpaa2_dev_rx_queue_intr_enable() instead
>
> The implementation also drains any frames being pulled on the previous core before continuing processing on the new one.
I do not find this drain in v4-S3. I only see the loop that was already
in 26.07, now in dpaa2_dev_rx_active_dq_storage(). It waits for the VDQ
of the previous core to complete. It does not read the frames.
Those frames are in q_storage[A]. B pulls into q_storage[B] and reads
only q_storage[B]. Can you point me to the code that reads them?
> 3)One DPIO per Rx queue is expensive
> The hardware currently supports up to 67 DPIO portals in total. If your deployment requires more portals than the hardware can provide under this model, we can evaluate and discuss possible compatibility or scalability alternatives.
On our LX2160A board I can create 65 DPIOs, not 67: 50 from the DPL,
then 15 more with restool before the MC refuses.
The two models do not spend them the same way:
- 26.07: two DPIOs per worker, whatever the number of ports and Rx
queues. Only the DPCONs follow the queues, and they are not
scarce: I can create 174 of them on the same board.
- this patch: one DPIO per worker, plus one DPIO per Rx queue.
Our case: 12 workers, 5 ports, 12 Rx queues per port.
26.07 this patch
workers 24 12
main thread 1 1
per Rx queue 0 60
total for DPDK 25 73
73 is more than the 65 of the board, before the kernel takes its own.
The limits are probably lower on the other DPAA2 SoCs, so the new
approach could be even worse there.
The only gain I see in the code is that the fd of the queue is known at
rx_queue_setup(), so the application can call
rte_eth_dev_rx_intr_ctl_q() before rte_eth_dev_rx_intr_enable(). My
l3fwd-power series already handles that in two patches (3/6 and 4/6).
The rest of the series still applies to your model: for example the
enable still returns -EAGAIN, and l3fwd-power does not check it today.
Without a benefit, I do not understand the point of this change. Please
tell me what the benefit is, so that we can weigh the trade-off between
the two approaches.
Thanks,
-------------------------------
Maxime Leroy
maxime at leroys.fr<mailto:maxime at leroys.fr>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mails.dpdk.org/archives/dev/attachments/20261119/776af715/attachment-0001.htm>
More information about the dev
mailing list