One transmitter worker stop after "some time" (Was: Performance "problem" or how to get power of the DPDK )

[Stephen Hemminger]

On Sun, 08 Jan 2023 16:23:13 +0300
Ruslan R. Laishev <zator at yandex.ru> wrote:

> Hello!
>  
> According the advices (from previous mails)  I recoded my little app to use several lcore-queue pairs to generate traffic. Thanks it's works fine, I see 8Gbps+ now with 2 workers .
> But! Now I have some other situation which I cannot to resolve. 2 workers  (every worker run on assigned lcore and put packets to separated tx queue) :
> after start of the app - both worker works some time, but at "some moment" one worker cannot get mbufs by rte_pktmbuf_alloc_bulk() .  Juts for demonstration a piece of stats:
>  
> At start :
> 08-01-2023 16:03:20.065  58628 [CPPROC\s_proc_auxilary:822] %TTR2CP-I:  [LCore:#001] TX/NoMbufs/Flush:1981397/0/1981397
> 08-01-2023 16:03:20.065  58628 [CPPROC\s_proc_auxilary:822] %TTR2CP-I:  [LCore:#002] TX/NoMbufs/Flush:1989108/0/1989108
>  
> Since "some moment"
> 08-01-2023 16:15:20.110  58628 [CPPROC\s_proc_auxilary:822] %TTR2CP-I:  [LCore:#001] TX/NoMbufs/Flush:2197615/5778976181/2197631
> 08-01-2023 16:15:20.110  58628 [CPPROC\s_proc_auxilary:822] %TTR2CP-I:  [LCore:#002] TX/NoMbufs/Flush:3952732/0/3952732
>  
> 08-01-2023 16:15:30.111  58628 [CPPROC\s_proc_auxilary:822] %TTR2CP-I:  [LCore:#001] TX/NoMbufs/Flush:2197615/5869388078/2197631
> 08-01-2023 16:15:30.111  58628 [CPPROC\s_proc_auxilary:822] %TTR2CP-I:  [LCore:#002] TX/NoMbufs/Flush:3980054/0/3980054
>  
> 08-01-2023 16:15:40.111  58628 [CPPROC\s_proc_auxilary:822] %TTR2CP-I:  [LCore:#001] TX/NoMbufs/Flush:2197615/5959777107/2197631
> 08-01-2023 16:15:40.111  58628 [CPPROC\s_proc_auxilary:822] %TTR2CP-I:  [LCore:#002] TX/NoMbufs/Flush:4007378/0/4007378
>  
> 08-01-2023 16:15:50.112  58628 [CPPROC\s_proc_auxilary:822] %TTR2CP-I:  [LCore:#001] TX/NoMbufs/Flush:2197615/6050173812/2197631
> 08-01-2023 16:15:50.112  58628 [CPPROC\s_proc_auxilary:822] %TTR2CP-I:  [LCore:#002] TX/NoMbufs/Flush:4034699/0/4034699
>  
> 08-01-2023 16:16:00.112  58628 [CPPROC\s_proc_auxilary:822] %TTR2CP-I:  [LCore:#001] TX/NoMbufs/Flush:2197615/6140583818/2197631
> 08-01-2023 16:16:00.112  58628 [CPPROC\s_proc_auxilary:822] %TTR2CP-I:  [LCore:#002] TX/NoMbufs/Flush:4062021/0/4062021
>  
> So one worker works fine and as expected, second worker - permanently don't getting mbufs .
> Is there what I have to check ?
> Thanks in advance!
>  
> --- 
> С уважением,
> Ruslan R. Laishev
> OpenVMS bigot, natural born system/network progger, C contractor.
> +79013163222
> +79910009922
>  
> 

Two thing to look at. First is the allocated mbuf pool big enough to handle the maximum
number of mbufs in flight in your application. For Tx, that is the number of transmit
queues multiplied by the number of transmit descriptors per ring. With some additional
buffers for staging.  Similar for receive side.

Second, transmit mbufs need to get cleaned up by the device driver after they
are sent. Depending on the the device, this maybe triggered by the receive path.
So polling for receive data may be needed even if you aren't doing any receives.