[PATCH] eal: add worker threads cleanup in rte_eal_cleanup()

[Gagandeep Singh]

Hi,

> -----Original Message-----
> From: Stephen Hemminger <stephen at networkplumber.org>
> Sent: Monday, January 13, 2025 10:10 PM
> To: Gagandeep Singh <G.Singh at nxp.com>
> Cc: dev at dpdk.org
> Subject: Re: [PATCH] eal: add worker threads cleanup in rte_eal_cleanup()
> 
> On Mon, 13 Jan 2025 05:13:01 +0000
> Gagandeep Singh <G.Singh at nxp.com> wrote:
> 
> > Hi,
> >
> > > -----Original Message-----
> > > From: Stephen Hemminger <stephen at networkplumber.org>
> > > Sent: Friday, January 10, 2025 10:49 PM
> > > To: Gagandeep Singh <G.Singh at nxp.com>
> > > Cc: dev at dpdk.org
> > > Subject: Re: [PATCH] eal: add worker threads cleanup in
> > > rte_eal_cleanup()
> > >
> > > On Fri, 10 Jan 2025 12:17:17 +0530
> > > Gagandeep Singh <g.singh at nxp.com> wrote:
> > >
> > > > This patch introduces a worker thread cleanup function in the EAL
> > > > library, ensuring proper termination of created pthreads and
> > > > invocation of registered pthread destructors.
> > > > This guarantees the correct cleanup of thread-specific resources,
> > > > used by drivers or applications.
> > > >
> > > > Signed-off-by: Gagandeep Singh <g.singh at nxp.com>
> > > > ---
> > >
> > > What problem is this trying to solve?
> > >
> > > Canceling threads sends signals and can be problematic.
> > > Many of the operations done in drivers are not signal safe.
> >
> > To ensure the proper cleanup of thread-specific resources, the DPAA driver
> initializes pthread-specific destructors using pthread_key_create(). These
> destructors are executed only when a thread terminates or the key is deleted.
> However, since threads are not terminated when the application is killed, these
> destructors are not executed, resulting in resource leaks.
> > To address this issue, we propose adding thread termination code to
> > rte_eal_cleanup() to ensure that threads are properly terminated,
> > thereby triggering the execution of pthread-specific destructors
> >
> > Any alternate suggestion in case pthread_cancel is not a better
> > solution? We can add pthread join timeout to avoid blocking on thread stuck or
> May be any way to call pthread_exit?
> 
> The DPAA driver is the problem here. It should not be using pthread_key.
> Other drivers don't do this. Other drivers do setup on probe and cleanup on close.
> 
> An application doing a clean shutdown should do what existing testpmd, l3fwd, do
> 	- wait for worker threads to go idle
> 	- stop all ports
> 	- close all ports
> 	- call eal cleanup
> 
> If DPAA driver needs pthread_key it should handling that in the close.
> But it really should be using DPDK thread local storage for this.
> 
This is fine for graceful application termination, but what about non-graceful application termination?
I have a DPAA bus cleanup patch ready, and will submit it soon, But we still need
destructor in case application exit without calling the rte_eal_cleanup().

I can see DPDK is giving an API rte_thread_key_create() which is defined only in UNIX and windows.
Why not for Linux?

I know DPAA driver is using pthread_key which it should not use but instead use
rte_thread_key_create() provided by DPDK which we can replace.
Having pthread destructor is still a valid case in my opinion.