[dpdk-users] Memory requirement calculation to create mempools with external memory

[Bao-Long Tran]

Hi,

My goal is to create mbuf pools using external memory with rte_malloc_heap_*().
The initial implementation goes like this:
- Create an empty heap rte_malloc_heap_create()
- Calculate memory required
- Provision and add the calculated single memory chunk to 
  the heap rte_malloc_heap_memory_add()
- Create mbuf pool from the heap

The problematic step is the calculation step. AFAIK there's no generic way to 
know how much memory is required to create a particular mempool. The example 
I found in testpmd (see below) is quite pessimistic as stated, which leads to 
up to 1G overprovision if you're using 1G hugepages.

I wonder under what circumstances can I make this memory requirement 
calculation deterministic. I.e., let's say I have total control over the 
underlying hugepages, including page size, iova, alignment, contiguousness 
(both iova and va), etc, how can I know exactly how much memory to allocate?

Anatoly, I don't quite understand your comment in the code below about the 
hdr_mem provision in particular. Why do we need to give hdr_mem special 
treatment? Further explanation is much appreciated.

Thanks,
BL

/* extremely pessimistic estimation of memory required to create a mempool */
static int
calc_mem_size(uint32_t nb_mbufs, uint32_t mbuf_sz, size_t pgsz, size_t *out)
{
	unsigned int n_pages, mbuf_per_pg, leftover;
	uint64_t total_mem, mbuf_mem, obj_sz;

	/* there is no good way to predict how much space the mempool will
	 * occupy because it will allocate chunks on the fly, and some of those
	 * will come from default DPDK memory while some will come from our
	 * external memory, so just assume 128MB will be enough for everyone.
	 */
	uint64_t hdr_mem = 128 << 20;

	/* account for possible non-contiguousness */
	obj_sz = rte_mempool_calc_obj_size(mbuf_sz, 0, NULL);
	if (obj_sz > pgsz) {
		TESTPMD_LOG(ERR, "Object size is bigger than page size\n");
		return -1;
	}

	mbuf_per_pg = pgsz / obj_sz;
	leftover = (nb_mbufs % mbuf_per_pg) > 0;
	n_pages = (nb_mbufs / mbuf_per_pg) + leftover;

	mbuf_mem = n_pages * pgsz;

	total_mem = RTE_ALIGN(hdr_mem + mbuf_mem, pgsz);

	if (total_mem > SIZE_MAX) {
		TESTPMD_LOG(ERR, "Memory size too big\n");
		return -1;
	}
	*out = (size_t)total_mem;

	return 0;
}