<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"> <head> <meta http-equiv="Content-Type" content="text/html; charset=Windows-1252"> <meta name="Generator" content="Microsoft Word 15 (filtered medium)"> <style></style> </head> <body lang="EN-IN" link="blue" vlink="purple" style="word-wrap:break-word"> <div class="WordSection1"> Thanks <a id="OWAAMF9BEDABBDB271A48B5873C869C70D9D0" href="mailto:dmitry.kozliuk@gmail.com"> @Dmitry Kozlyuk</a> for your suggestions<o:p></o:p> <o:p> </o:p> I will try the following for DPDK pool creation <o:p></o:p> <o:p> </o:p> My logic of calculating MBUF’s remains same<o:p></o:p> <o:p> </o:p> Saw this code in DPDK testpmd where External heap memory is used<o:p></o:p> <o:p> </o:p> case MP_ALLOC_XMEM_HUGE:<o:p></o:p> {<o:p></o:p> int heap_socket;<o:p></o:p> bool huge = mp_alloc_type == MP_ALLOC_XMEM_HUGE;<o:p></o:p> <o:p> </o:p> if (setup_extmem(nb_mbuf, mbuf_seg_size, huge) < 0)<o:p></o:p> rte_exit(EXIT_FAILURE, "Could not create external memory\n");<o:p></o:p> <o:p> </o:p> heap_socket =<o:p></o:p> rte_malloc_heap_get_socket(EXTMEM_HEAP_NAME);<o:p></o:p> if (heap_socket < 0)<o:p></o:p> rte_exit(EXIT_FAILURE, "Could not get external memory socket ID\n");<o:p></o:p> <o:p> </o:p> TESTPMD_LOG(INFO, "preferred mempool ops selected: %s\n",<o:p></o:p> rte_mbuf_best_mempool_ops());<o:p></o:p> rte_mp = rte_pktmbuf_pool_create(pool_name, nb_mbuf,<o:p></o:p> mb_mempool_cache, 0, mbuf_seg_size,<o:p></o:p> heap_socket);<o:p></o:p> break;<o:p></o:p> }<o:p></o:p> <o:p> </o:p> So I will do the same<o:p></o:p> <o:p> </o:p> <ol style="margin-top:0cm" start="1" type="1"> <li class="MsoListParagraph" style="margin-left:0cm;mso-list:l0 level1 lfo1">EAL Init<o:p></o:p></li><li class="MsoListParagraph" style="margin-left:0cm;mso-list:l0 level1 lfo1">Calculate MBUF we need for our application<o:p></o:p></li><li class="MsoListParagraph" style="margin-left:0cm;mso-list:l0 level1 lfo1">Then create pool using MP_ALLOC_XEM_HUGE<o:p></o:p></li></ol> <o:p> </o:p> 1,2,3 should work right , that should avoid heap corruption issues right ?<o:p></o:p> <o:p> </o:p> We have total 3 types pool creation<o:p></o:p> <o:p> </o:p> /*<o:p></o:p> * Select mempool allocation type:<o:p></o:p> * - native: use regular DPDK memory<o:p></o:p> * - anon: use regular DPDK memory to create mempool, but populate using<o:p></o:p> * anonymous memory (may not be IOVA-contiguous)<o:p></o:p> * - xmem: use externally allocated hugepage memory<o:p></o:p> */<o:p></o:p> <o:p> </o:p> Instead of Freeing unused virtual memory in DPDK native way, we just create a pool with type XMEM_HUGE and add page by page to it like testpmd code<o:p></o:p> <o:p> </o:p> Please let me know 1,2,3 steps are good right, so we need to boot DPDK with socket mem<o:p></o:p> <o:p> </o:p> --socket-mem 2048 so that DPDK takes only 1 page natively and boots up right ?<o:p></o:p> <o:p> </o:p> Thanks<o:p></o:p> Umakiran<o:p></o:p> <o:p> </o:p> <o:p> </o:p> <div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm"> From: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com> Date: Friday, 23 September 2022 at 6:41 PM To: Umakiran Godavarthi (ugodavar) <ugodavar@cisco.com> Cc: anatoly.burakov@intel.com <anatoly.burakov@intel.com>, dev@dpdk.org <dev@dpdk.org>, stephen@networkplumber.org <stephen@networkplumber.org> Subject: Re: DPDK 19.11.5 Legacy Memory Design Query<o:p></o:p> </div> <div> 2022-09-23 12:12 (UTC+0000), Umakiran Godavarthi (ugodavar): > [Uma] : Yes I agree if free_hp = 400, nr_hp = 252, we are expecting DPDK takes only 252 and keep the remaining pages free in its heap. > As you have mentioned just boot DPDK with 1 page, and add pages we want later, is this the steps > > 1. NR_HP =1 , FREE_HP = 1 > 2. EAL INIT (DPDK boots up with 1 2 MB page) > 3. What is the API for adding later on pages ? (rte_extmem_*, can you please give the full API details and how to call it with arguments ) Guide: <a href="https://doc.dpdk.org/guides-19.11/prog_guide/env_abstraction_layer.html#support-for-externally-allocated-memory"> https://doc.dpdk.org/guides-19.11/prog_guide/env_abstraction_layer.html#support-for-externally-allocated-memory</a> I recommend reading the entire section about DPDK memory management since you're going to use an uncommon API and should understand what's going on. API (the linked function and those following it): <a href="http://doc.dpdk.org/api-19.11/rte__malloc_8h.html#a2295623c85ba41fe5bf7dce6bf0393d6"> http://doc.dpdk.org/api-19.11/rte__malloc_8h.html#a2295623c85ba41fe5bf7dce6bf0393d6</a> <a href="http://doc.dpdk.org/api-19.11/rte__memory_8h.html#a653510fb0c58bf63f54708677e3a2eba"> http://doc.dpdk.org/api-19.11/rte__memory_8h.html#a653510fb0c58bf63f54708677e3a2eba</a> > We can do 1,2,3 there is a problem once we reduce pages to 1 , kernel will free the huge pages totally > > So is there a way not to touch NR_HP, FREE_HP, and just pass arguments to boot DPDK with just 1 page ? Please let us know and later add pages we need to DPDK !! See --socket-mem EAL option: <a href="http://doc.dpdk.org/guides-19.11/linux_gsg/linux_eal_parameters.html#id3"> http://doc.dpdk.org/guides-19.11/linux_gsg/linux_eal_parameters.html#id3</a> > Why do you need legacy mode in the first place? > Looks like you're painfully trying to achieve the same result > that dynamic mode would give you automatically. > > [Uma] : Yes , we can’t avoid legacy memory design as secondary process mapped page by page to primary process , and physical addr space is same for both processes. We have to stick to legacy memory design only for now !! Sorry, I still don't understand. Virtual and physical addresses of DPDK memory are the same across processes in both legacy and dynamic memory mode.<o:p></o:p> </div> </div> </body> </html>