[dpdk-dev] [dpdk-users] Should ''shmget" not be used to consume hugepages in DPDK?

[Byonggon Chun]

x-posting to dev mailing list.

Hi all.
I'm Kubernetes contributors and I'm working to make container isolation of
hugepages that allows us to set hugepages limit per container cgroup.
(At this point, limits are set on pod level cgroup even though we asked
hugepages as the container level resource)

I tore down testPMD and some parts of DPDK lib and I got a question after i
found there is no usage of "shmget" in DPDK.

My question is that Should "shmget" not be used to consume hugepages in
DPDK?
And here is following questions:
1) If we don't have to use "shmget", Why? Does it affect performance?
2) If I use "shmget" to get hugepages, should I call "mlock" syscall for it?

For more details, as I know there are three ways to consume hugepages in
kubernetes.
1) shmget with SHM_HUGETLB
2) mmap with hugetlbs filebacking
3) mmap with MAP_ANONYMOUS | MAP_HUGETLB

And I found that testPMD calls mlock syscall when it maps an anonymous
hugepages or external allocated
hugepages.https://github.com/DPDK/dpdk/blob/924e55fb340623f03fdf2ff7fbcfd78819d1db25/app/test-pmd/testpmd.c#L896https://github.com/DPDK/dpdk/blob/924e55fb340623f03fdf2ff7fbcfd78819d1db25/app/test-pmd/testpmd.c#L916

Thanks.





On Fri, Dec 20, 2019 at 9:42 PM Byonggon Chun <byonggonchun at gmail.com>
wrote:

> > shmget is a legacy Unix API and there is no point in using it.
>
> Yeah, I agree with it,
> I also prefer to use mmap with hugetlbfs in a DPDK container.
>
> The reason why I started this mail thread is some DPDK users still use
> shmget to consume hugepages, and I just wanted to find a good rationale to
> convince them to use mmap.
>
> But, at this point, I have only one rationale : shmget is a legacy UINIX
> API.
>
> On Fri, Dec 20, 2019 at 6:06 AM Stephen Hemminger <
> stephen at networkplumber.org> wrote:
>
>> On Fri, 20 Dec 2019 01:23:50 +0900
>> Byonggon Chun <byonggonchun at gmail.com> wrote:
>>
>> > Hi all.
>> > I'm Kubernetes contributors and I'm working to make container isolation
>> of
>> > hugepages that allows us to set hugepages limit per container cgroup.
>> > (At this point, limits are set on pod level cgroup even though we asked
>> > hugepages as the container level resource)
>> >
>> > I tore down testPMD and some parts of DPDK lib and I got a question
>> after i
>> > found there is no usage of "shmget" in DPDK.
>> >
>> > My question is that Should "shmget" not be used to consume hugepages in
>> > DPDK?
>> > And here is following questions:
>> > 1) If we don't have to use "shmget", Why? Does it affect performance?
>> > 2) If I use "shmget" to get hugepages, should I call "mlock" syscall
>> for it?
>> >
>> > For more details, as I know there are three ways to consume hugepages in
>> > kubernetes.
>> > 1) shmget with SHM_HUGETLB
>> > 2) mmap with hugetlbs filebacking
>> > 3) mmap with MAP_ANONYMOUS | MAP_HUGETLB
>> >
>> > And I found that testPMD calls mlock syscall when it maps an anonymous
>> > hugepages or external allocated hugepages.
>> >
>> https://github.com/DPDK/dpdk/blob/924e55fb340623f03fdf2ff7fbcfd78819d1db25/app/test-pmd/testpmd.c#L896
>> >
>> https://github.com/DPDK/dpdk/blob/924e55fb340623f03fdf2ff7fbcfd78819d1db25/app/test-pmd/testpmd.c#L916
>> >
>> > Thanks.
>>
>> shmget is a legacy Unix API and there is no point in using it.
>> For new applications libhugetlbfs is preferable.
>>
>