[dpdk-dev] DPDK 18.05 only works with up to 4 NUMAs systems

[Burakov, Anatoly]

On 22-Jun-18 5:37 PM, Kumar, Ravi1 wrote:
> Hi,
> 
> As the memory subsystem in DPDK 18.05 is reworked, it has introduced a problem for AMD EPYC 2P platforms.
> The issue is that DPDK 18.05 only works with up to 4 NUMAs. For AMD EPYC 2P platforms, DPDK now only works with P0 (NUMA 0-3) and does not work with P1 (NUMA 4-7).
> 
> The problem can be fixed by reducing some of the default settings of the memory subsystem.
> 
> To solve this issue:
> -              We can create our own config file for our integrated 10G NIC, that is for amd_xgbe PMD. This will make amd_xgbe immune to this problem.
> -              However, when any other NIC (Intel, Mellanox, Cavium or Broadcom etc.) is plugged into NUMA 4-7, the problem will still be exposed.
> -              If we only fix it in "config/common_base", it will cover all cases.
> 
> Our current workaround is:
> Edit config file "./config/common_base" and change the following line
>                  CONFIG_RTE_MAX_MEM_MB_PER_TYPE=131072
> TO
>                  CONFIG_RTE_MAX_MEM_MB_PER_TYPE=65536
> 
> Any better solution for this issue is welcome.
> 
> We would appreciate if this issue can be fixed in the next release (18.08) so the STOCK version of DPDK works on AMD EPYC 2P platforms.
> 
> Regards,
> Ravi
> 

Hi Ravi,

What is the reason behind this limitation? Is it too much virtual memory 
being preallocated?

-- 
Thanks,
Anatoly