[dpdk-dev] [PATCH v3 00/10] Windows basic memory management

[Kadam, Pallavi]

On 4/14/2020 12:44 PM, Dmitry Kozlyuk wrote:
> Note: no changes in cover letter since v2.
> 
> This patchset implements basic MM with the following features:
> 
> * Hugepages are dynamically allocated in user-mode.
> * Only 2MB hugepages are supported.
> * IOVA is always PA, obtained through kernel-mode driver.
> * No 32-bit support (presumably not demanded).
> * Ni multi-process support (it is forcefully disabled).
> * No-huge mode for testing with IOVA unavailable.
> 
> 
> The first commit introduces a new kernel-mode driver, virt2phys.
> It translates user-mode virtual addresses into physical addresses.
> On Windows community call 2020-04-01 it was decided this driver can be
> used for now, later netUIO may pick up its code/interface or not.
> 
> 
> New EAL public functions for memory mapping are introduced
> to mitigate OS differences in DPDK libraries and applications:
> 
> * rte_mem_map
> * rte_mem_unmap
> * rte_mem_lock
> 
> To support common MM routines, internal wrappers for low-level
> memory reservation and file management are introduced. These changes
> affect Linux and FreeBSD EAL. Shared code is placed unded /unix/
> subdirectory (suggested by Thomas).
> 
> Also, entire <sys/queue.h> is imported from FreeBSD, replacing existing
> partial import. There is already a license exception for this file.
> 
> 
> Windows MM duplicates quite a lot of code from Linux EAL:
> 
> * eal_memalloc_alloc_seg_bulk
> * eal_memalloc_free_seg_bulk
> * calc_num_pages_per_socket
> * rte_eal_hugepage_init
> 
> Perhaps this should be left as-is until Windows MM evolves into having
> some specific requirements for these parts.
> 
> 
> Notes on checkpatch warnings:
> 
> * No space after comma / no space before closing parent in macros---
>    definitely a false-positive, unclear how to suppress this.
> 
> * Issues from imported BSD code---probably should be ignored?
> 
> * Checkpatch is not run against dpdk-kmods (Windows drivers).
> 
> ---
> 
> v3:
> 
>      * Fix Linux build on and aarch64 and 32-bit x86 (reported by CI).
>      * Fix logic and error handling while allocating segments.
>      * Fix Unix rte_mem_map(): return NULL on failure.
>      * Fix some checkpatch.sh issues:
>          * Do not return positive errno, use DWORD for GetLastError().
>          * Make dpdk-kmods source files non-executable.
>      * Improve GSG for Windows Server (suggested by Ranjit Menon).
> 
> v2:
> 
>      * Rebase on ToT. Move all new code shared between Linux and FreeBSD
>        to /unix/ subdirectory, also factor out some existing code there.
>      * Improve description of Clang issue with rte_page_sizes on Windows.
>        Restore -fstrict-enum for EAL. Check running, not target compiler.
>      * Use EAL prefix for private facilities instead if RTE.
>      * Improve documentation comments for new functions.
>      * Remove co-installer for virt2phys. Add a typecast for clarity.
>      * Document virt2phys in user guide, improve its own README.
>      * Explicitly and forcefully disable multi-process.
> 
> Dmitry Kozlyuk (9):
>    eal/windows: do not expose private EAL facilities
>    eal/windows: improve CPU and NUMA node detection
>    eal/windows: initialize hugepage info
>    eal: introduce internal wrappers for file operations
>    eal: introduce memory management wrappers
>    eal: extract common code for memseg list initialization
>    eal/windows: fix rte_page_sizes with Clang on Windows
>    eal/windows: replace sys/queue.h with a complete one from FreeBSD
>    eal/windows: implement basic memory management
> 
>   config/meson.build                            |   12 +-
>   doc/guides/windows_gsg/build_dpdk.rst         |   20 -
>   doc/guides/windows_gsg/index.rst              |    1 +
>   doc/guides/windows_gsg/run_apps.rst           |   84 ++
>   lib/librte_eal/common/eal_common_fbarray.c    |   57 +-
>   lib/librte_eal/common/eal_common_memory.c     |  104 +-
>   lib/librte_eal/common/eal_private.h           |  134 +-
>   lib/librte_eal/common/malloc_heap.c           |    1 +
>   lib/librte_eal/common/meson.build             |    9 +
>   lib/librte_eal/freebsd/eal_memory.c           |   55 +-
>   lib/librte_eal/include/rte_memory.h           |   74 ++
>   lib/librte_eal/linux/eal_memory.c             |   68 +-
>   lib/librte_eal/meson.build                    |    4 +
>   lib/librte_eal/rte_eal_exports.def            |  119 ++
>   lib/librte_eal/rte_eal_version.map            |    4 +
>   lib/librte_eal/unix/eal.c                     |   47 +
>   lib/librte_eal/unix/eal_memory.c              |  113 ++
>   lib/librte_eal/unix/meson.build               |    7 +
>   lib/librte_eal/windows/eal.c                  |  160 +++
>   lib/librte_eal/windows/eal_hugepages.c        |  108 ++
>   lib/librte_eal/windows/eal_lcore.c            |  187 ++-
>   lib/librte_eal/windows/eal_memalloc.c         |  418 ++++++
>   lib/librte_eal/windows/eal_memory.c           | 1141 +++++++++++++++++
>   lib/librte_eal/windows/eal_mp.c               |  103 ++
>   lib/librte_eal/windows/eal_thread.c           |    1 +
>   lib/librte_eal/windows/eal_windows.h          |  129 ++
>   lib/librte_eal/windows/include/meson.build    |    2 +
>   lib/librte_eal/windows/include/pthread.h      |    2 +
>   lib/librte_eal/windows/include/rte_os.h       |   48 +-
>   .../windows/include/rte_virt2phys.h           |   34 +
>   lib/librte_eal/windows/include/rte_windows.h  |   43 +
>   lib/librte_eal/windows/include/sys/queue.h    |  663 +++++++++-
>   lib/librte_eal/windows/include/unistd.h       |    3 +
>   lib/librte_eal/windows/meson.build            |    6 +
>   34 files changed, 3611 insertions(+), 350 deletions(-)
>   create mode 100644 doc/guides/windows_gsg/run_apps.rst
>   create mode 100644 lib/librte_eal/unix/eal.c
>   create mode 100644 lib/librte_eal/unix/eal_memory.c
>   create mode 100644 lib/librte_eal/unix/meson.build
>   create mode 100644 lib/librte_eal/windows/eal_hugepages.c
>   create mode 100644 lib/librte_eal/windows/eal_memalloc.c
>   create mode 100644 lib/librte_eal/windows/eal_memory.c
>   create mode 100644 lib/librte_eal/windows/eal_mp.c
>   create mode 100644 lib/librte_eal/windows/eal_windows.h
>   create mode 100644 lib/librte_eal/windows/include/rte_virt2phys.h
>   create mode 100644 lib/librte_eal/windows/include/rte_windows.h
> 
Tested-by: Pallavi Kadam <pallavi.kadam at intel.com>