[dpdk-dev] Huge mapping secondary process linux

[Jonas Pfefferle1]

"Burakov, Anatoly" <anatoly.burakov at intel.com> wrote on 10/27/2017 04:06:44
PM:

> From: "Burakov, Anatoly" <anatoly.burakov at intel.com>
> To: Jonas Pfefferle1 <JPF at zurich.ibm.com>, dev at dpdk.org
> Cc: chaozhu at linux.vnet.ibm.com, bruce.richardson at intel.com
> Date: 10/27/2017 04:06 PM
> Subject: Re: [dpdk-dev] Huge mapping secondary process linux
>
> On 27-Oct-17 1:43 PM, Jonas Pfefferle1 wrote:
> >
> >
> > Hi @all,
> >
> > I'm trying to make sense of the hugepage memory mappings in
> > librte_eal/linuxapp/eal/eal_memory.c:
> > * In rte_eal_hugepage_attach (line 1347) when we try to do a private
> > mapping on /dev/zero (line 1393) why do we not use MAP_FIXED if we need
the
> > addresses to be identical with the primary process?
> > * On POWER we have this weird business going on where we use
MAP_HUGETLB
> > because according to this commit:
> >
> > commit 284ae3e9ff9a92575c28c858efd2c85c8de6d440
> > Author: Chao Zhu <chaozhu at linux.vnet.ibm.com>
> > Date:   Thu Apr 6 15:36:09 2017 +0530
> >
> >      eal/ppc: fix mmap for memory initialization
> >
> >      On IBM POWER platform, when mapping /dev/zero file to hugepage
memory
> >      space, mmap will not respect the requested address hint. This will
> > cause
> >      the memory initialization for the second process fails. This patch
adds
> >      the required mmap flags to make it work. Beside this, users need
to set
> >      the nr_overcommit_hugepages to expand the VA range. When
> >      doing the initialization, users need to set both nr_hugepages and
> >      nr_overcommit_hugepages to the same value, like 64, 128, etc.
> >
> > mmap address hints are not respected. Looking at the mmap code in the
> > kernel this is not true entirely however under some circumstances the
hint
> > can be ignored (
> > https://urldefense.proofpoint.com/v2/url?
>
u=http-3A__elixir.free-2Delectrons.com_linux_latest_source_arch_powerpc_mm_mmap.c-23L103&d=DwICaQ&c=jf_iaSHvJObTbx-

> siA1ZOg&r=rOdXhRsgn8Iur7bDE0vgwvo6TC8OpoDN-
> pXjigIjRW0&m=cttQcHlAYixhsYS3lz-
> BAdEeg4dpbwGdPnj2R3I8Do0&s=Gp0TIjUtIed05Jgb7XnlocpCYZdFXZXiH0LqIWiNMhA&e=
> > ). However I believe we can remove the extra case for PPC if we use
> > MAP_FIXED when doing the secondary process mappings because we need
them to
> > be identical anyway. We could also use MAP_FIXED when doing the primary
> > process mappings resp. get_virtual_area if we want to have any
guarantees
> > when specifying a base address. Any thoughts?
> >
> > Thanks,
> > Jonas
> >
> hi Jonas,
>
> MAP_FIXED is not used because it's dangerous, it unmaps anything that is
> already mapped into that space. We would rather know that we can't map
> something than unwittingly unmap something that was mapped before.

Ok, I see. Maybe we can add a check to the primary process's memory
mappings whether the hint has been respected or not? At least warn if it
hasn't.

>
> --
> Thanks,
> Anatoly
>