Proposal using RCU for callbacks

[Konstantin Ananyev]

> On Mon, 12 Oct 2026 21:27:08 +0100
> Marat Khalili <qm2k21 at gmail.com> wrote:
>
>> On 12/10/2026 19:13, Stephen Hemminger wrote:
>>
>>> The BPF and packet capture Rx/Tx callbacks use a per-queue use counter
>>> so that removal can wait for the datapath. In BPF that costs a locked
>>> instruction per burst. The code would be faster and cleaner if RCU
>>> could be used, but DPDK RCU is limited to the QSBR model, which
>>> requires application opt-in.
>>>
>>> The userspace RCU library (liburcu) supports other models, so I did a
>>> brief investigation into whether something similar could be done in
>>> DPDK. As expected, it also turned up pre-existing issues: the counters
>>> are wrong with MT_LOCKFREE Tx, the pdump one is not safe as written,
>>> and cryptodev callback removal never waits.
>>>
>>> I used Claude to write it up in more detail (attached). It is all from
>>> code reading; nothing has been reproduced or prototyped yet.
>>> This is too big to go into 26.11 at this point, will try and have something
>>> for 27.03
>> Maybe it's just me, but the document is a little hard to read without
>> knowing all of the context. Nevertheless, there is no doubt the sync
>> mechanism in bpf_pkt.c can be improved. It done today, it would probably
>> be written differently, very likely with RCU.
>>
>> I would argue for incremental approach however. I'm currently aware of
>> the following problems, in the order of decreasing priority and gain/effort:
>> * Deprecated rte_smp barrier functions being used.
>> * BPF inuse counter is not safe when using lockfree transmit (Bug 2051)
>> * Code needs cleanup and documenting to make discussions more productive.
>> * Possibly additional safety issues listed above (which I don't yet
>> fully understand).
>> * Missing features: multi-threaded data planes, multiple callbacks, BPF
>> program reuse etc.
>> * On the very last place, performance; as usual, benchmarks are
>> mandatory for it to have any meaning.
>>
>> I think we need to resist the temptation to just rewrite everything,
>> especially with AI. Some recent research shows that productivity gains
>> from AI are very limited particularly because code review becomes
>> bottleneck (tbh it always was). Test coverage is also clearly
>> insufficient for any major rewrites of this code. A patch solving
>> everything at once risks getting stuck in review for very long time.
>> Perhaps we arrive at RCU solution eventually, but I'd start with first
>> items in the list.
>>
> Short form: BPF and pdump could use RCU to manage callback lifetime;
> cryptodev already uses QSBR, but incorrectly. I would like to see
> one usage pattern across all of DPDK that is fast and correct.
Yep, in general I do agree with your analysis
Yes cryptodev CB add/remove uses RCU incorrectly and needs to be fixed.
ethdev CB add/remove doesn't contain any sync mechanism at all,
which is a problem.
Agree that both should use the same mechanism and then we can simply
remove current one in bpf_pkt.c and rely on one in ethdev.

About further enhancements, i.e. using something better/smarter then our
current QSBR-based RCU ny position is simple: let's fix the problems
above first before starting to do any further enhancements.

In particular, regarding membarrier() - consider using it for BPF CBs
instead of smb_mb, but decided not to. The main reasons:
- with membarrier() - all threads in the process will be affected.
   Also, AFAIK it is implemented via IPI - which is quite heavy.
- current smp_mb (at least on modern x86_64) is quite fast
(it uses lock-ed instruction on thread's stack, and last time I measured it,
takes ¬30-50 cycles). As we usually doing it for a bulk (32+) of packets,
the overhead is quite small: 1-1.5 cycle per packet.

But again, if you have some smart ideas how to improve our RCU lib with 
membarrier() -
would be really interested to hear.
>
> DPDK RCU only does QSBR, which needs the application to report when
> each thread has passed a quiescent state. That restriction is why it is underused.
>
> Other RCU flavors avoid this. With membarrier, the reader side needs
> no fence, only a compiler barrier, and the writer pays with a syscall.
> Reader state can hang off EAL thread registration, so no application
> change is needed.
>
> Today the BPF callback does a full memory barrier per burst. RCU would
> replace that with a compiler barrier. Benchmarks will be needed to
> show it matters.
>
> This is not "let's use AI to rewrite it". AI was useful for the what-if
> analysis; the real fixes will be real code, done incrementally.
> Agree the barrier conversion and Bug 2051 come first.
>
> See:
>   - https://man7.org/linux/man-pages/man2/membarrier.2.html
>   - https://liburcu.org/
>   - https://github.com/nmraz/simplercu (toy implementation)
>   - https://lwn.net/Articles/369018/ (original 2010 RFC)