[PATCH v5 01/10] memarea: introduce memarea library
Mattias Rönnblom
hofors at lysator.liu.se
Thu Oct 6 22:15:34 CEST 2022
On 2022-10-05 06:09, datshan wrote:
> From: Chengwen Feng <fengchengwen at huawei.com>
>
> The memarea library is an allocator of variable-size object which based
> on a memory region.
>
> This patch provides create/destroy API.
>
> Signed-off-by: Chengwen Feng <fengchengwen at huawei.com>
> ---
> MAINTAINERS | 5 +
> doc/api/doxy-api-index.md | 3 +-
> doc/api/doxy-api.conf.in | 1 +
> doc/guides/prog_guide/index.rst | 1 +
> doc/guides/prog_guide/memarea_lib.rst | 39 ++++++
> doc/guides/rel_notes/release_22_11.rst | 6 +
> lib/eal/common/eal_common_log.c | 1 +
> lib/eal/include/rte_log.h | 1 +
> lib/memarea/memarea_private.h | 30 +++++
> lib/memarea/meson.build | 16 +++
> lib/memarea/rte_memarea.c | 157 +++++++++++++++++++++++++
> lib/memarea/rte_memarea.h | 145 +++++++++++++++++++++++
> lib/memarea/version.map | 12 ++
> lib/meson.build | 1 +
> 14 files changed, 417 insertions(+), 1 deletion(-)
> create mode 100644 doc/guides/prog_guide/memarea_lib.rst
> create mode 100644 lib/memarea/memarea_private.h
> create mode 100644 lib/memarea/meson.build
> create mode 100644 lib/memarea/rte_memarea.c
> create mode 100644 lib/memarea/rte_memarea.h
> create mode 100644 lib/memarea/version.map
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index a55b379d73..b9c638221d 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1550,6 +1550,11 @@ F: app/test/test_lpm*
> F: app/test/test_func_reentrancy.c
> F: app/test/test_xmmt_ops.h
>
> +Memarea - EXPERIMENTAL
> +M: Chengwen Feng <fengchengwen at huawei.com>
> +F: lib/memarea
> +F: doc/guides/prog_guide/memarea_lib.rst
> +
> Membership - EXPERIMENTAL
> M: Yipeng Wang <yipeng1.wang at intel.com>
> M: Sameh Gobriel <sameh.gobriel at intel.com>
> diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
> index de488c7abf..24456604f8 100644
> --- a/doc/api/doxy-api-index.md
> +++ b/doc/api/doxy-api-index.md
> @@ -62,7 +62,8 @@ The public API headers are grouped by topics:
> [memzone](@ref rte_memzone.h),
> [mempool](@ref rte_mempool.h),
> [malloc](@ref rte_malloc.h),
> - [memcpy](@ref rte_memcpy.h)
> + [memcpy](@ref rte_memcpy.h),
> + [memarea](@ref rte_memarea.h)
>
> - **timers**:
> [cycles](@ref rte_cycles.h),
> diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in
> index f0886c3bd1..8334ebcbd6 100644
> --- a/doc/api/doxy-api.conf.in
> +++ b/doc/api/doxy-api.conf.in
> @@ -53,6 +53,7 @@ INPUT = @TOPDIR@/doc/api/doxy-api-index.md \
> @TOPDIR@/lib/latencystats \
> @TOPDIR@/lib/lpm \
> @TOPDIR@/lib/mbuf \
> + @TOPDIR@/lib/memarea \
> @TOPDIR@/lib/member \
> @TOPDIR@/lib/mempool \
> @TOPDIR@/lib/meter \
> diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst
> index 8564883018..e9015d65e3 100644
> --- a/doc/guides/prog_guide/index.rst
> +++ b/doc/guides/prog_guide/index.rst
> @@ -37,6 +37,7 @@ Programmer's Guide
> hash_lib
> toeplitz_hash_lib
> efd_lib
> + memarea_lib
> member_lib
> lpm_lib
> lpm6_lib
> diff --git a/doc/guides/prog_guide/memarea_lib.rst b/doc/guides/prog_guide/memarea_lib.rst
> new file mode 100644
> index 0000000000..b96dad15f6
> --- /dev/null
> +++ b/doc/guides/prog_guide/memarea_lib.rst
> @@ -0,0 +1,39 @@
> +.. SPDX-License-Identifier: BSD-3-Clause
> + Copyright(c) 2022 HiSilicon Limited
> +
> +Memarea Library
> +===============
> +
> +Introduction
> +------------
> +
> +The memarea library provides an allocator of variable-size objects, it is
> +oriented towards the application layer, which could provides 'region-based
> +memory management' function [1].
> +
> +The main features are as follows:
> +
> +* The default aligement size is ``RTE_CACHE_LINE_SIZE``.
> +
> +* The memory region can be initialized from the following memory sources:
> + a) RTE memory: e.g. invoke ``rte_malloc_socket`` to obtain. b) System API:
> + e.g. invoke posix_memalign to obtain. c) User provided address: it can be from
> + extendedd memory as long as it is available. d) User provided memarea: it can
> + be from another memarea.
> +
> +* It provides refcnt feature which could be useful in multi-reader scenario.
> +
> +* It supports MT-safe as long as it's specified at creation time.
> +
> +Library API Overview
> +--------------------
> +
> +The ``rte_memarea_create()`` function is used to create a memarea, the function
> +returns the pointer to the created memarea or ``NULL`` if the creation failed.
> +
> +The ``rte_memarea_destroy()`` function is used to destroy a memarea.
> +
> +Reference
> +---------
> +
> +[1] https://en.wikipedia.org/wiki/Region-based_memory_management
> diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst
> index 5d8ef669b8..4c1f760b98 100644
> --- a/doc/guides/rel_notes/release_22_11.rst
> +++ b/doc/guides/rel_notes/release_22_11.rst
> @@ -55,6 +55,12 @@ New Features
> Also, make sure to start the actual text at the margin.
> =======================================================
>
> +* **Added memarea library.**
> +
> + The memarea library is an allocator of variable-size objects, it is oriented
> + towards the application layer, which could provides 'region-based memory
> + management' function.
> +
> * **Added configuration for asynchronous flow connection tracking.**
>
> Added connection tracking action number hint to ``rte_flow_configure``
> diff --git a/lib/eal/common/eal_common_log.c b/lib/eal/common/eal_common_log.c
> index bd7b188ceb..3d62af59c6 100644
> --- a/lib/eal/common/eal_common_log.c
> +++ b/lib/eal/common/eal_common_log.c
> @@ -369,6 +369,7 @@ static const struct logtype logtype_strings[] = {
> {RTE_LOGTYPE_EFD, "lib.efd"},
> {RTE_LOGTYPE_EVENTDEV, "lib.eventdev"},
> {RTE_LOGTYPE_GSO, "lib.gso"},
> + {RTE_LOGTYPE_MEMAREA, "lib.memarea"},
> {RTE_LOGTYPE_USER1, "user1"},
> {RTE_LOGTYPE_USER2, "user2"},
> {RTE_LOGTYPE_USER3, "user3"},
> diff --git a/lib/eal/include/rte_log.h b/lib/eal/include/rte_log.h
> index 25ce42cdfc..708f3a39dd 100644
> --- a/lib/eal/include/rte_log.h
> +++ b/lib/eal/include/rte_log.h
> @@ -48,6 +48,7 @@ extern "C" {
> #define RTE_LOGTYPE_EFD 18 /**< Log related to EFD. */
> #define RTE_LOGTYPE_EVENTDEV 19 /**< Log related to eventdev. */
> #define RTE_LOGTYPE_GSO 20 /**< Log related to GSO. */
> +#define RTE_LOGTYPE_MEMAREA 21 /**< Log related to memarea. */
>
> /* these log types can be used in an application */
> #define RTE_LOGTYPE_USER1 24 /**< User-defined log type 1. */
> diff --git a/lib/memarea/memarea_private.h b/lib/memarea/memarea_private.h
> new file mode 100644
> index 0000000000..c76392d3e6
> --- /dev/null
> +++ b/lib/memarea/memarea_private.h
> @@ -0,0 +1,30 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2022 HiSilicon Limited
> + */
> +
> +#ifndef MEMAREA_PRIVATE_H
> +#define MEMAREA_PRIVATE_H
> +
> +#include <rte_memarea.h>
> +
> +#define MEMAREA_FREE_ELEM_COOKIE 0xFFFFFFFF
> +
> +struct memarea_elem {
> + size_t size;
> + uint32_t cookie;
> + int32_t refcnt; /* Non-zero indicates that it has been allocated */
> + TAILQ_ENTRY(memarea_elem) elem_node;
> + TAILQ_ENTRY(memarea_elem) free_node;
> +} __rte_cache_aligned;
> +
Why is the elem type cache line aligned? Need the elem data start be
cache line aligned?
> +TAILQ_HEAD(memarea_elem_list, memarea_elem);
> +
> +struct rte_memarea {
> + struct rte_memarea_param init;
> + rte_spinlock_t lock;
> + void *area_addr;
> + struct memarea_elem_list elem_list;
> + struct memarea_elem_list free_list;
> +} __rte_cache_aligned;
> +
> +#endif /* MEMAREA_PRIVATE_H */
> diff --git a/lib/memarea/meson.build b/lib/memarea/meson.build
> new file mode 100644
> index 0000000000..0a74fb4cd1
> --- /dev/null
> +++ b/lib/memarea/meson.build
> @@ -0,0 +1,16 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2022 HiSilicon Limited
> +
> +if is_windows
> + build = false
> + reason = 'not supported on Windows'
> + subdir_done()
> +endif
> +
> +sources = files(
> + 'rte_memarea.c',
> +)
> +headers = files(
> + 'rte_memarea.h',
> +)
> +deps += []
> diff --git a/lib/memarea/rte_memarea.c b/lib/memarea/rte_memarea.c
> new file mode 100644
> index 0000000000..868da7661d
> --- /dev/null
> +++ b/lib/memarea/rte_memarea.c
> @@ -0,0 +1,157 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2022 HiSilicon Limited
> + */
> +
> +#include <stdio.h>
> +#include <stdlib.h>
> +
> +#include <rte_common.h>
> +#include <rte_log.h>
> +#include <rte_malloc.h>
> +#include <rte_spinlock.h>
> +
> +#include "rte_memarea.h"
> +#include "memarea_private.h"
> +
> +static int
> +memarea_check_param(const struct rte_memarea_param *init)
> +{
> + size_t len;
> +
> + if (init == NULL) {
> + RTE_LOG(ERR, MEMAREA, "memarea init param is NULL!\n");
> + return -EINVAL;
> + }
> +
> + len = strnlen(init->name, RTE_MEMAREA_NAMESIZE);
> + if (len == 0 || len >= RTE_MEMAREA_NAMESIZE) {
> + RTE_LOG(ERR, MEMAREA, "memarea name size %zu invalid!\n", len);
> + return -EINVAL;
> + }
> +
> + if (init->source != RTE_MEMAREA_SOURCE_RTE_MEMORY &&
> + init->source != RTE_MEMAREA_SOURCE_SYSTEM_API &&
> + init->source != RTE_MEMAREA_SOURCE_USER_ADDR &&
> + init->source != RTE_MEMAREA_SOURCE_USER_MEMAREA) {
> + RTE_LOG(ERR, MEMAREA, "memarea: %s source: %d not supported!\n",
> + init->name, init->source);
> + return -EINVAL;
> + }
> +
> + if (init->total_sz <= sizeof(struct memarea_elem)) {
> + RTE_LOG(ERR, MEMAREA, "memarea: %s total-size: %zu too small!\n",
> + init->name, init->total_sz);
> + return -EINVAL;
> + }
> +
> + if (init->source == RTE_MEMAREA_SOURCE_USER_ADDR && init->user_addr == NULL) {
> + RTE_LOG(ERR, MEMAREA, "memarea: %s user provided addr is NULL!\n", init->name);
> + return -EINVAL;
> + }
> +
> + if (init->source == RTE_MEMAREA_SOURCE_USER_ADDR &&
> + ((uintptr_t)init->user_addr & (RTE_CACHE_LINE_SIZE - 1))) {
> + RTE_LOG(ERR, MEMAREA, "memarea: %s user provided addr should align: %d!\n",
> + init->name, RTE_CACHE_LINE_SIZE);
> + return -EINVAL;
> + }
> +
> + if (init->source == RTE_MEMAREA_SOURCE_USER_MEMAREA && init->user_memarea == NULL) {
> + RTE_LOG(ERR, MEMAREA, "memarea: %s user provided memarea is NULL!\n", init->name);
> + return -EINVAL;
> + }
> +
> + if (init->alg != RTE_MEMAREA_ALG_DEFAULT) {
> + RTE_LOG(ERR, MEMAREA, "memarea: %s alg: %d not supported!\n",
> + init->name, init->alg);
> + return -EINVAL;
> + }
> +
> + return 0;
> +}
> +
> +static void *
> +memarea_alloc_from_system_api(size_t size)
> +{
> + void *ptr = NULL;
> + int ret;
> +
> + ret = posix_memalign(&ptr, RTE_CACHE_LINE_SIZE, size);
> + if (ret)
> + return NULL;
> + return ptr;
> +}
> +
> +static void *
> +memarea_alloc_area(const struct rte_memarea_param *init)
> +{
> + void *ptr = NULL;
> +
> + if (init->source == RTE_MEMAREA_SOURCE_RTE_MEMORY)
Delete MEMORY. Of course it's memory. What else? If you want to make it
clear it's from the RTE heap, it should spell out HEAP. Or MALLOC.
> + ptr = rte_malloc_socket(NULL, init->total_sz, RTE_CACHE_LINE_SIZE,
> + init->numa_socket);
> + else if (init->source == RTE_MEMAREA_SOURCE_SYSTEM_API)
> + ptr = memarea_alloc_from_system_api(init->total_sz);
"SYSTEM_API" doesn't strike me as a good name.
RTE_MEMAREA_SOURCE_LIBC
RTE_MEMAREA_SOURCE_STD_HEAP
or at least remove API so it'll be
RTE_MEMAREA_SOURCE_SYSTEM
> + else if (init->source == RTE_MEMAREA_SOURCE_USER_ADDR)
I would delete "ADDR".
> + ptr = init->user_addr;
> +
> + if (ptr == NULL)
> + RTE_LOG(ERR, MEMAREA, "memarea: %s alloc memory area fail!\n", init->name);
> +
> + return ptr;
> +}
> +
> +struct rte_memarea *
> +rte_memarea_create(const struct rte_memarea_param *init)
> +{
> + struct memarea_elem *elem;
> + struct rte_memarea *ma;
> + void *addr;
> + int ret;
> +
> + ret = memarea_check_param(init);
> + if (ret)
> + return NULL;
> +
> + addr = memarea_alloc_area(init);
> + if (addr == NULL)
> + return NULL;
> +
> + ma = rte_zmalloc(NULL, sizeof(struct rte_memarea), RTE_CACHE_LINE_SIZE);
> + if (ma == NULL) {
> + RTE_LOG(ERR, MEMAREA, "malloc memarea: %s management obj fail!\n", init->name);
> + return NULL;
> + }
> +
> + ma->init = *init;
> + rte_spinlock_init(&ma->lock);
> + TAILQ_INIT(&ma->elem_list);
> + TAILQ_INIT(&ma->free_list);
> + ma->area_addr = addr;
> + elem = addr;
> + elem->size = init->total_sz - sizeof(struct memarea_elem);
> + elem->cookie = MEMAREA_FREE_ELEM_COOKIE;
> + elem->refcnt = 0;
> + TAILQ_INSERT_TAIL(&ma->elem_list, elem, elem_node);
> + TAILQ_INSERT_TAIL(&ma->free_list, elem, free_node);
> +
> + return ma;
> +}
> +
> +static void
> +memarea_free_area(struct rte_memarea *ma)
> +{
> + if (ma->init.source == RTE_MEMAREA_SOURCE_RTE_MEMORY)
> + rte_free(ma->area_addr);
> + else if (ma->init.source == RTE_MEMAREA_SOURCE_SYSTEM_API)
> + free(ma->area_addr);
> +}
> +
> +void
> +rte_memarea_destroy(struct rte_memarea *ma)
> +{
> + if (ma == NULL)
> + return;
> + memarea_free_area(ma);
> + rte_free(ma);
> +}
> diff --git a/lib/memarea/rte_memarea.h b/lib/memarea/rte_memarea.h
> new file mode 100644
> index 0000000000..543cda4cac
> --- /dev/null
> +++ b/lib/memarea/rte_memarea.h
> @@ -0,0 +1,145 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2022 HiSilicon Limited
> + */
> +
> +#ifndef RTE_MEMAREA_H
> +#define RTE_MEMAREA_H
> +
> +/**
> + * @file
> + * RTE Memarea.
> + *
> + * The memarea is an allocator of variable-size object which based on a memory
> + * region. It has the following features:
> + *
> + * - The default alignment size is RTE_CACHE_LINE_SIZE.
This can be read as two things: the object size is aligned, or the start
address is aligned.
> + * - The memory region can be initialized from the following memory sources:
> + * 1. RTE memory: e.g. invoke rte_malloc_socket to obtain.
Remove "to obtain", or add "memory" after "obtain". Do you really mean
"e.g.", and not "i.e."?
> + * 2. System API: e.g. invoke posix_memalign to obtain.
> + * 3. User provided address: it can be from extended memory as long as it is
> + * available. The address must be aligned to RTE_CACHE_LINE_SIZE.
What is extended memory?
> + * 4) User provided memarea: it can be from another memarea. So we can build
> + * the following memory management structure:
> + * \code{.unparsed}
> + * -------------
> + * | memarea-1 |
> + * -------------
> + * |
> + * v
> + * -------------------------------
> + * | | |
> + * v v v
> + * ------------- ------------- ----------
> + * | memarea-2 | | memarea-3 | | object |
> + * ------------- ------------- ----------
> + * \endcode
> + * As shown above, the memarea-2/3 both create from memarea-1's memory.
> + * - It provides refcnt feature which could be useful in multi-reader scenario.
> + * - It supports MT-safe as long as it's specified at creation time. If not
> + * specified, all the functions of the memarea API are lock-free, and assume
> + * to not be invoked in parallel on different logical cores to work on the
> + * same memarea.
> + */
> +
> +#include <stdbool.h>
> +#include <stdint.h>
> +#include <stdio.h>
> +
> +#include <rte_compat.h>
> +
> +#ifdef __cplusplus
> +extern "C" {
> +#endif
> +
> +#define RTE_MEMAREA_NAMESIZE 64
> +
> +/**
> + * Memarea memory source.
> + */
> +enum rte_memarea_source {
> + /** Memory source comes from rte memory. */
> + RTE_MEMAREA_SOURCE_RTE_MEMORY,
> + /** Memory source comes from system API. */
> + RTE_MEMAREA_SOURCE_SYSTEM_API,
> + /** Memory source comes from user-provided address. */
> + RTE_MEMAREA_SOURCE_USER_ADDR,
> + /** Memory source comes from user-provided memarea. */
> + RTE_MEMAREA_SOURCE_USER_MEMAREA,
> +};
> +
> +/**
> + * Memarea memory management algorithm.
> + */
> +enum rte_memarea_alg {
> + /* The default management algorithm is a variant of the next fit
> + * algorithm. It uses a free-list to apply for memory and uses an
> + * element-list in ascending order of address to support merging
> + * upon free.
> + */
> + RTE_MEMAREA_ALG_DEFAULT,
> +};
> +
> +struct rte_memarea;
> +
> +struct rte_memarea_param {
Does this struct need to be public?
> + char name[RTE_MEMAREA_NAMESIZE]; /**< Name of memarea. */
> + enum rte_memarea_source source; /**< Memory source of memarea. */
> + enum rte_memarea_alg alg; /**< Memory management algorithm. */
> + size_t total_sz; /**< total size (bytes) of memarea. */
> + /** Indicates whether the memarea API should be MT-safe. */
> + uint32_t mt_safe : 1;
Why not bool?
> + union {
> + /** Numa socket from which to apply for memarea's memory, this
> + * field is valid only when the source is set to be
> + * RTE_MEMAREA_SOURCE_RTE_MEMORY.
> + */
> + int numa_socket;
> + /** User provided address, this field is valid only when the
> + * source is set to be RTE_MEMAREA_SOURCE_USER_ADDR.
> + * Note: the provided address must align at least
> + * RTE_CACHE_LINE_SIZE.
> + */
> + void *user_addr;
> + /** User provided memarea, this field is valid only when the
> + * source is set to be RTE_MEMAREA_SOURCE_USER_MEMAREA.
> + */
> + struct rte_memarea *user_memarea;
> + };
> +};
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Create memarea.
> + *
> + * Create one new memarea.
> + *
> + * @param init
> + * The init parameter of memarea.
> + *
> + * @return
> + * Non-NULL on success. Otherwise NULL is returned.
> + */
> +__rte_experimental
> +struct rte_memarea *rte_memarea_create(const struct rte_memarea_param *init);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Destroy memarea.
> + *
> + * Destroy the memarea.
> + *
> + * @param ma
> + * The pointer of memarea.
> + */
> +__rte_experimental
> +void rte_memarea_destroy(struct rte_memarea *ma);
> +
> +#ifdef __cplusplus
> +}
> +#endif
> +
> +#endif /* RTE_MEMAREA_H */
> diff --git a/lib/memarea/version.map b/lib/memarea/version.map
> new file mode 100644
> index 0000000000..f36a04d7cf
> --- /dev/null
> +++ b/lib/memarea/version.map
> @@ -0,0 +1,12 @@
> +EXPERIMENTAL {
> + global:
> +
> + rte_memarea_create;
> + rte_memarea_destroy;
> +
> + local: *;
> +};
> +
> +INTERNAL {
> + local: *;
> +};
> diff --git a/lib/meson.build b/lib/meson.build
> index c648f7d800..521a25d6c0 100644
> --- a/lib/meson.build
> +++ b/lib/meson.build
> @@ -42,6 +42,7 @@ libraries = [
> 'kni',
> 'latencystats',
> 'lpm',
> + 'memarea',
> 'member',
> 'pcapng',
> 'power',
More information about the dev
mailing list