Shared memory between two primary DPDK processes

[Antonio Di Bacco]

Another info to add:

The process that allocates the 1GB page has this map:
antodib at Ubuntu-20.04-5:: /proc> sudo cat /proc/27812/maps | grep huge
140000000-180000000 rw-s 00000000 00:46 97193
 /dev/huge1G/rtemap_0

while the process that maps the 1GB page (--file-prefix p2) has this maps,
is stealing a new page?
antodib at Ubuntu-20.04-5:: /proc> sudo cat /proc/27906/maps | grep huge
140000000-180000000 rw-s 00000000 00:46 113170
/dev/huge1G/p2map_0
7f7bc0000000-7f7c00000000 rw-s 00000000 00:46 97193
 /dev/huge1G/rtemap_0

Il giorno lun 18 apr 2022 alle ore 19:34 Antonio Di Bacco <
a.dibacco.ks at gmail.com> ha scritto:

> At the end I tried the pidfd_getfd syscall that is working really fine and
> giving me back a "clone" fd of an fd in that was opened from another
> process. I tested it opening a text file in the first process  and after
> cloning the fd , I could really read the file also in the second process.
> Now the weird thing:
> 1) In the first process I allocate- a huge page, then get the fd
> 2) In the second process I get my "clone" fd and do an mmap, it works but
> if I write on that memory, the first process cannot see what I wrote
>
> int second_process(int remote_pid, int remote_mem_fd) {
>
>         printf("remote_pid %d remote_mem_fd %d\n", remote_pid,
> remote_mem_fd);
>         int pidfd = syscall(__NR_pidfd_open, remote_pid, 0);
>
>         int my_mem_fd = syscall(438, pidfd, remote_mem_fd, 0);
>         printf("my_mem_fd %d\n", my_mem_fd);   // This is nice
>
>         int flags = MAP_SHARED | MAP_HUGETLB | (30 << MAP_HUGE_SHIFT);
>         uint64_t* addr = (uint64_t*) mmap(NULL, 1024 * 1024 * 1024,
> PROT_READ|PROT_WRITE, flags, my_mem_fd, 0);
>         if (addr == -1)
>             perror("mmap");
>         *addr = 0x0101010102020202;
> }
>
>
> Il giorno gio 14 apr 2022 alle ore 21:51 Antonio Di Bacco <
> a.dibacco.ks at gmail.com> ha scritto:
>
>>
>>
>> Il giorno gio 14 apr 2022 alle ore 21:01 Dmitry Kozlyuk <
>> dmitry.kozliuk at gmail.com> ha scritto:
>>
>>> 2022-04-14 10:20 (UTC+0200), Antonio Di Bacco:
>>> [...]
>>> > Ok, after having a look to memif I managed to exchange the fd  between
>>> the
>>> > two processes and it works.
>>> > Anyway the procedure seems a little bit clunky and I think I'm going
>>> to use
>>> > the new SYSCALL pidfd_getfd
>>> > to achieve the same result.  In your opinion this method (getfd_pidfd)
>>> > could also work if the two DPDK processes
>>> > are inside different docker containers?
>>>
>>> Honestly, I've just learned about pidfd_getfd() from you.
>>> But I know that containers use PID namespaces, so there's a question
>>> how you will obtain the pidfd of a process in another container.
>>>
>>> In general, any method of sharing FD will work.
>>> Remember that you also need offset and size.
>>> Given that some channel is required to share those,
>>> I think Unix domain socket is still the preferred way.
>>>
>>> > Or is there another mechanims like using handles to hugepages present
>>> in
>>> > the filesystem to share between two
>>> > different containers?
>>>
>>> FD is needed for mmap().
>>> You need to either pass the FD or open() the same hugepage file by path.
>>> I advise against using paths because they are not a part of DPDK API
>>> contract.
>>>
>>
>> Thank you very much Dmitry, your answers are always enlightening.
>> I'm going to ask a different question on the dpdk.org about the best
>> practice to share memory between two dpdk processes running in different
>> containers.
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mails.dpdk.org/archives/users/attachments/20220418/101f4494/attachment.htm>