[dpdk-dev] [ RFC ] performance thread example

[Betts, Ian]

Hi Folks,

Would welcome feedback on this proposal for an example app to explore performance
with different threading models.

Thanks,
Ian

Performance thread example application

This example will comprise a simple layer 3 forwarding application distributed across multiple threads.
It will be possible to configure the application for, and contrast forwarding performance of different threading models,
More specifically when the different threads are:-


1.       EAL threads running on different physical cores

2.       EAL threads running on the same physical core ( multiple lcore per physical core)

3.       Lightweight threads running in an EAL thread on one more physical cores.


Purpose and justification

Since dpdk 2.0 it has been possible to assign multiple EAL threads to a physical core ( case 2 above ).
Currently no example application has focused on demonstrating the performance constraints of differing threading models.

Whilst purpose built applications that fully comprehend the DPDK single threaded programming model will always yield superior performance,
the desire to preserve ROI in legacy code written for multithreaded operating environments  makes lightweight threads ( case 3 above )
an option worthy of consideration.

As well as aiding with legacy code reuse, it is anticipated that lightweight threads will make it possible to scale a multithreaded
application with fine granularity allowing an application  to more easily take advantage of headroom on EAL cores,
or conversely occupy more cores, as dictated by system load.

To explore performance with lightweight threads a simple cooperative scheduler subsystem is being included in this example application.
If the expected benefits and use cases prove to be of value, it is anticipated that this lightweight thread subsystem would become a library
in some future DPDK release.

Proposed High-Level Solution(s)
A layer 3 forwarding application will comprise a number of threads from rx through  tx stages, and will accept command line parameters enabling
the assignment of queues to threads, threads to lcores, and lcores to physical cores.

In the case that the threads are to be lightweight threads then a lightweight thread scheduler will be run as the main loop of one or more EAL threads.
The lightweight thread subsystem will have an API comparable in purpose to a subset of the equivalent POSIX P-thread functions.

Main APIs

*         Thread control APIs

o   Lthread_create

o   Lthread_cancel

o   Lthread_join

o   Lthread_detach

o   Lthread_exit

o   Lthread_sleep

o   Lthread_yield

o   Lthread_set_affinity

o   Lthread_current



*         Thread local storage APIs

o   Lthread_set_data

o   Lthread_get_data

o   Lthread_key_create

o   Lthread_key_delete

o   Lthread_get_specific

o   Lthread_set_specific

o   PER_LTHREAD macros



*         Sychronization APIs

o   Lthread_mutex_init

o   Lthread_mutex_destroy

o   Lthread_mutex_lock

o   Lthread_mutex_trylock

o   Lthread_mutex_unlock

o   Lthread_cond_init

o   Lthread_cond_destroy

o   Lthread_cond_wait

o   Lthread_cond_signal

o   Lthread_cond_broadcast



*         Scheduler control APIs

o   Lthread_num_schedulers_set

o   Lthread_active_schedulers

o   Lthread_run


Proposed example format
The L-thread packet forwarding application, (and test code) , will be example applications under RTE_SDK/examples/. The L-thread subsystem will be a common source folder included by these applications.
RTE_SDK/examples/lthread
RTE_SDK/examples/lthread/common/
RTE_SDK/examples/lthread/l3fwd-lthread/
RTE_SDK/examples/lthread/test/



Ian Betts
Intel Shannon Limited
Registered in Ireland
Registered Office Collinstown Industrial Park, Leixlip, County Kildare
Registered Number:308263
Business address: Dromore House, East Park, Shannon, Co. Clare

--------------------------------------------------------------
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263
Business address: Dromore House, East Park, Shannon, Co. Clare

This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.