[dpdk-dev] eal: DPDK: 18.11.6 version rte_eal_init() function cleans the runtime directory in 5.4.35 kernel

[Mohakud, Amiya Ranjan]

Hi Anatoly -  I raised a bug https://bugs.dpdk.org/show_bug.cgi?id=561 in DPDK for tracking purpose.

Regards
Amiya

From: Mohakud, Amiya Ranjan
Sent: 16 October 2020 11:04
To: Burakov, Anatoly <anatoly.burakov at intel.com>; dpdk-dev <dev at dpdk.org>
Subject: RE: [dpdk-dev] eal: DPDK: 18.11.6 version rte_eal_init() function cleans the runtime directory in 5.4.35 kernel

Hi Anatoly,
Thanks for your confirmation. Yes, you got it right now. The primary process removing *its own* own files for some locking issue, which should not be the case.

Just to clarify:
Removing stuff at the end of the init process is intended process. This
is because every process will create their own shadow page tables, and
defunct processes will not remove them afterwards. This is done because
otherwise the tmpfs will slowly fill up with unused processes' fbarrays.

What is *not* intended behavior is primary process removing *its own*
filesystem entries - this shouldn't happen, and in fact wouldn't have
happened if the file locking was working as it was intended. Normally,
when seeing files that are locked (in use), EAL will skip them, so that
only files that are not in use would be deleted. It looks like you're
observing exactly that - primary process removing *its own* files for
some reason.
Regards
Amiya

From: Burakov, Anatoly <anatoly.burakov at intel.com<mailto:anatoly.burakov at intel.com>>
Sent: 16 October 2020 00:04
To: Mohakud, Amiya Ranjan <amohakud at rbbn.com<mailto:amohakud at rbbn.com>>; dpdk-dev <dev at dpdk.org<mailto:dev at dpdk.org>>
Cc: Dey, Souvik <sodey at rbbn.com<mailto:sodey at rbbn.com>>
Subject: Re: [dpdk-dev] eal: DPDK: 18.11.6 version rte_eal_init() function cleans the runtime directory in 5.4.35 kernel

________________________________
NOTICE: This email was received from an EXTERNAL sender
________________________________

On 15-Oct-20 7:01 PM, Burakov, Anatoly wrote:
> On 15-Oct-20 5:14 PM, Mohakud, Amiya Ranjan wrote:
>> + Souvik
>>
>> *From:*Mohakud, Amiya Ranjan
>> *Sent:* 15 October 2020 21:38
>> *To:* Burakov, Anatoly <anatoly.burakov at intel.com<mailto:anatoly.burakov at intel.com>>; dpdk-dev
>> <dev at dpdk.org<mailto:dev at dpdk.org>>
>> *Subject:* RE: [dpdk-dev] eal: DPDK: 18.11.6 version rte_eal_init()
>> function cleans the runtime directory in 5.4.35 kernel
>>
>> Hi Anatoly - Thanks for helping on this.
>>
>> I am not aware, where the primary process re-creates the files. Can
>> you please point me to that? As per my code browsing and
>> understanding, I can see, fbarray_memzone file gets created in
>> rte_eal_memzone_init()->rte_fbarray_init() and it stays there till
>> eal_clean_runtime_dir() gets called towards end of rte_eal_init().
>> This does not get deleted in 4.19 kernel, but in 5.4, it does.
>>
>> /I'm not sure i understand. Primary process is supposed to clear the
>> files. It will then recreate them. Are you suggesting that it's clearing
>> them *after* it has created them?/
>>
>> /
>> /Going by my observation, the file highlighted below gets deleted by
>> the time rte_eal_init() is over.
>>
>> srwxr-xr-x 1 root root      0 Oct 15 11:24 mp_socket
>>
>> -rw------- 1 root root  12432 Oct 15 11:24 hugepage_info
>>
>> -rw------- 1 root root 188416 Oct 15 11:24 fbarray_memzone
>>
>> -rw------- 1 root root 397312 Oct 15 11:24 fbarray_memseg-2048k-0-1
>>
>> -rw------- 1 root root 397312 Oct 15 11:24 fbarray_memseg-2048k-0-0
>>
>> -rw------- 1 root root 397312 Oct 15 11:24 fbarray_memseg-2048k-0-3
>>
>> -rw------- 1 root root 397312 Oct 15 11:24 fbarray_memseg-2048k-0-2
>>
>> -rw------- 1 root root  16529 Oct 15 11:24 config
>>
>> Please reach out to me for further clarification.
>>
>> Regards
>>
>> Amiya
>
> Hi,
>
> Sorry, yes, you're right (it's been a while since i looked at the code),
> it removes unused stuff at the end of init. There's even a comment
> explaining why that's done :D
>
> It sounds like closing the file descriptor also drops the lock. This
> locking business is a huge pain because we have to support old kernels
> which don't have the only sane file locking implementation that Linux has.
>
> While i wouldn't go as far as to say "this is a kernel regression" as
> most likely it's me who's at fault here, but this definitely shouldn't
> happen. Unfortunately, i won't be online for the next two weeks, but
> i'll definitely look into this after i'm back, so thanks for your report.
>

Hi,

Just to clarify:

Removing stuff at the end of the init process is intended process. This
is because every process will create their own shadow page tables, and
defunct processes will not remove them afterwards. This is done because
otherwise the tmpfs will slowly fill up with unused processes' fbarrays.

What is *not* intended behavior is primary process removing *its own*
filesystem entries - this shouldn't happen, and in fact wouldn't have
happened if the file locking was working as it was intended. Normally,
when seeing files that are locked (in use), EAL will skip them, so that
only files that are not in use would be deleted. It looks like you're
observing exactly that - primary process removing *its own* files for
some reason.

--
Thanks,
Anatoly