<html> <head> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> <style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style> </head> <body dir="ltr"> <div id="divRplyFwdMsg"> <div style="direction: ltr; font-family: Calibri, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);" class="elementToProof"> <b>From:</b> Stephen Hemminger <stephen@networkplumber.org></div> <div style="direction: ltr; font-family: Calibri, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);"> <b>Sent:</b> 08 September 2025 23:08<br> <b>To:</b> Serhii Iliushyk <sil-plv@napatech.com><br> <b>Cc:</b> dev@dpdk.org <dev@dpdk.org>; Mykola Kostenok <mko-plv@napatech.com>; Christian Koue Muf <ckm@napatech.com><br> <b>Subject:</b> Re: [PATCH v1 0/7] migrate threads to DPDK service framework</div> <div style="direction: ltr;"> </div> </div> <div style="font-size: 11pt;" class="elementToProof">On Mon, 8 Sep 2025 13:04:38 +0200<br> Serhii Iliushyk <sil-plv@napatech.com> wrote:<br> <br> > This modification provides better resource (CPU) management for NTNIC PMD.<br> ><br> > The following threads are migrated:<br> > * FLM update thread<br> > * Statistic thread<br> > * Port event thread<br> > * Adapter monitoring thread<br> > Additionally, a warning is added to inform users about the importance of<br> > dedicating lcores to the DPDK service framework when using the NTNIC PMD.<br> > The code is also cleaned up to use pthreads and rte_thread APIs.<br> ><br> > After this patch series, an each application using NTNIC PMD should<br> > dedicate at least five(5) cores for DPDK service framework to ensure<br> > proper operation of the NTNIC PMD.<br> <br> I was concerned with excessive control thread usage before, and this<br> seems to be worse not better.<br> <br> There are conflicting use cases here:<br> 1. The original DPDK goal was to make effective use of multiple cores<br> with no locking. Intel customers often had idle lcore's and some CPU's<br> had lots of inactive lcores that could be used to get more work done.<br> Dedicating some to service tasks etc was a natural outcome.<br> <br> 2. DPDK applications (OVS, Grout, VPP) usually want to know about lcores<br> at least in the documentation and examples. They don't cover the case<br> of service lcores.<br> <br> 3. Dedicated low core count smart NIC's using DPDK. In this case it<br> makes sense to be frugal with lcores since the point of the smart NIC<br> is to be able to run other control services. For example, the MS<br> NIC had hard limit on the DPDK part (via cgroups) of only 4 + main<br> lcores.<br> <br> Granted NTNIC is likely only being used for a specific application on<br> a specific set of hardware.<br> <br> The ideal would be to have better control event management in EAL.<br> Something like "libevent" style API. This would reduce control core<br> needs, and avoid any potential resource conflict overlap between control<br> threads. </div> <div style="font-size: 11pt;" class="elementToProof"><br> </div> <p style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0cm 0cm 12pt;" class="elementToProof"> <span style="font-family: Aptos, sans-serif; font-size: 11pt; color: black;">Hi Stephen!</span></p> <div style="background-color: rgb(255, 255, 255); margin: 12pt 0px;" class="elementToProof"> <p style="text-align: left; text-indent: 0px; margin: 0cm; font-family: Aptos, sans-serif; font-size: 12pt;" class="elementToProof"> <span style="font-size: 11pt; color: black;">Thanks for the detailed feedback.</span></p> </div> <div style="background-color: rgb(255, 255, 255); margin: 12pt 0px;" class="elementToProof"> <p style="text-align: left; text-indent: 0px; margin: 0cm; font-family: Aptos, sans-serif; font-size: 12pt;" class="elementToProof"> <span style="font-size: 11pt; color: black;">The migration of the rte_service actually improves the performance of the ntnic. That is an obvious result, since we dedicate the entire core to a single service when the default (1-to-1) mapping is used. <br> A similar performance and stability can be achieved by using threads and mutexes (The rte_spinlocks introduce instability when used with threads)</span></p> </div> <div style="background-color: rgb(255, 255, 255); margin: 12pt 0px;" class="elementToProof"> <p style="text-align: left; text-indent: 0px; margin: 0cm; font-family: Aptos, sans-serif; font-size: 12pt;" class="elementToProof"> <span style="font-size: 11pt; color: black;">The ntnic service API provided with this patch series enables users to map NTNIC services to lcores as necessary.</span></p> <p style="text-align: left; text-indent: 0px; margin: 0cm; font-family: Aptos, sans-serif; font-size: 12pt;" class="elementToProof"> <span style="font-size: 11pt; color: black;">All NTNIC services can be mapped to separate lcores or to a single lcore; however, this approach has a significant impact on performance. </span></p> <p style="text-align: left; text-indent: 0px; margin: 0cm; font-family: Aptos, sans-serif; font-size: 12pt;" class="elementToProof"> <span style="font-size: 11pt; color: black;">The mapping of all services to a single lcore is similar to a "libevent" style API. However, there are no events; instead, there are continuous calling services in a predefined order. </span></p> <p style="text-align: left; text-indent: 0px; margin: 0cm; font-family: Aptos, sans-serif; font-size: 12pt;" class="elementToProof"> <span style="font-size: 11pt; color: black;">The one core cannot process all the services due to a negative performance impact. Following this approach, we have to spawn a thread (or start the service by looking for a free lcore) every time an event occurs. If you have any other vision about it, please share your opinion.</span></p> </div> <div style="background-color: rgb(255, 255, 255); margin: 12pt 0px;" class="elementToProof"> <p style="text-align: left; text-indent: 0px; margin: 0cm; font-family: Aptos, sans-serif; font-size: 12pt;" class="elementToProof"> <span style="font-size: 11pt; color: black;">In the scope of complex DPDK-based applications (OVS, Grout, VPP), we still have options to use (1-to-1) mapping by passing raw EAL options -(s SERVICE COREMASK or -S SERVICE CORELIST).</span></p> </div> <div style="background-color: rgb(255, 255, 255); margin: 12pt 0px;" class="elementToProof"> </div> <div style="background-color: rgb(255, 255, 255); margin: 12pt 0px;" class="elementToProof"> <p style="text-align: left; text-indent: 0px; margin: 0cm; font-family: Aptos, sans-serif; font-size: 12pt;" class="elementToProof"> <span style="font-size: 11pt; color: black;">Thanks, <br> Serhii</span></p> </div> <div style="font-size: 11pt;" class="elementToProof"><br> </div> <div style="font-size: 11pt;" class="elementToProof"><br> </div> <div style="font-size: 11pt;" class="elementToProof"><br> </div> <div style="font-size: 11pt;" class="elementToProof"><br> </div> <div style="font-size: 11pt;" class="elementToProof"><br> </div> </body> </html>