Issues around packet capture when secondary process is doing rx/tx

[Konstantin Ananyev]

> >>>> -----Original Message-----
> >>>> From: Stephen Hemminger <stephen at networkplumber.org>
> >>>> Sent: Tuesday, January 9, 2024 11:07 PM
> >>>> To: Konstantin Ananyev <konstantin.ananyev at huawei.com>
> >>>> Cc: dev at dpdk.org; arshdeep.kaur at intel.com; Gowda, Sandesh <sandesh.gowda at intel.com>; Reshma Pattan
> >>>> <reshma.pattan at intel.com>
> >>>> Subject: Re: Issues around packet capture when secondary process is doing rx/tx
> >>>>
> >>>> On Mon, 8 Jan 2024 15:13:25 +0000
> >>>> Konstantin Ananyev <konstantin.ananyev at huawei.com> wrote:
> >>>>
> >>>>>> I have been looking at a problem reported by Sandesh
> >>>>>> where packet capture does not work if rx/tx burst is done in secondary process.
> >>>>>>
> >>>>>> The root cause is that existing rx/tx callback model just doesn't work
> >>>>>> unless the process doing the rx/tx burst calls is the same one that
> >>>>>> registered the callbacks.
> >>>>>>
> >>>>>> An example sequence would be:
> >>>>>> 	1. dumpcap (or pdump) as secondary tells pdump in primary to register callback
> >>>>>> 	2. secondary process calls rx_burst.
> >>>>>> 	3. rx_burst sees the callback but it has pointer pdump_rx which is not necessarily
> >>>>>> 	   at same location in primary and secondary process.
> >>>>>> 	4. indirect function call in secondary to bad location likely causes crash.
> >>>>>
> >>>>> As I remember, RX/TX callbacks were never intended to work over multiple processes.
> >>>>> Right now RX/TX callbacks are private for the process, different process simply should not
> >>>>> see/execute them.
> >>>>> I.E. it callbacks list is part of 'struct rte_eth_dev' itself, not the rte_eth_dev.data that is shared
> >>>>> between processes.
> >>>>> It should be normal, wehn for the same port/queue you will end-up with different list of callbacks
> >>>>> for different processes.
> >>>>> So, unless I am missing something, I don't see how we can end-up with 3) and 4) from above:
> >>>>> From my understanding secondary process will never see/call primary's callbacks.
> >>>>>
> >>>>> About pdump itself, it was a while when I looked at it last time, but as I remember to start it to work,
> >>>>> server process has to call rte_pdump_init() which in terns register PDUMP_MP handler.
> >>>>> I suppose for the secondary process to act as a 'pdump server' it needs to call rte_pdump_init() itself,
> >>>>> though I am not sure such option is supported right now.
> >>>>>
> >>>>
> >>>> Did some more tests with modified testpmd, and reached some conclusions:
> >>>>
> >>>> The logical interface would be to allow rte_pdump_init() to be called by
> >>>>    the process that would be using rx/tx burst API's.
> >>>>
> >>>>   This doesn't work as it should because the multi-process socket API
> >>>>   assumes that the it only runs the server in primary.  The secondary
> >>>>   can start its own MP thread, but it won't work:
> >>>>
> >>>>   Primary EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> >>>>   Secondary: EAL: Multi-process socket /var/run/dpdk/rte/mp_socket_6057_1ccd4157fd5
> >>>>
> >>>>   The problem is when client (pdump or dumpcap) tries to run, it uses the mp_socket
> >>>>   in the primary which causes: EAL: Cannot find action: mp_pdump
> >>>>
> >>>>   Looks like the whole MP socket mechanism is just not up to this.
> >>>>
> >>>> Maybe pdump needs to have its own socket and control thread?
> >>>> Or MP socket needs to have some multicast fanout to all secondaries?
> >>>
> >>> Might be we can do something simpler: pass to pdump_enable(), where we want to enable it:
> >>> on primary (remote_ process or secondary (local) process?
> >>> And then for primary send a message over MP socket (as we doing now), and for secondary (itself)
> >>> just do actual pdump enablement on it's own (install callbacks, etc.).
> >>> Yes, in that way, one secondary would not be able to enable/idable pdump on another secondary,
> >>> only on itself, but might be it is not needed?
> >>>
> >>>
> >>
> >> How secondary, lets say testpmd secondary, install callbacks without
> >> getting 'mp' & 'ring' info from pdump secondary process?
> >
> > Please see my comment above (I copied it here too):
> >> Yes, in that way, one secondary would not be able to enable/disable pdump on another secondary, only on itself, but might be it is
> not needed?
> >
> 
> I saw it Konstantin, but it wasn't clear to me what you are suggesting,
> that is why I am asking more.
> 
> Do you suggest when testpmd run as secondary process and doing
> forwarding, it should do the tasks of pdump itself and we don't use
> pdump at all?

Sort of - we can still use pdump API, but under the hood instead of sending request to primary,
secondary would just install an RX/TX callback for itself.
Again, with that schema secondary<->secondary would not be supported.