<!DOCTYPE html><html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<div class="moz-cite-prefix">Dear Stephen,<br>
<br>
The problem has not been solved, but I found a workaround.
According to the documentation
(<a class="moz-txt-link-freetext" href="https://doc.dpdk.org/guides/prog_guide/gpudev.html">https://doc.dpdk.org/guides/prog_guide/gpudev.html</a>, sec 11.3),
rte_extmem_register should be invoked with GPU_PAGE_SIZE as an
argument. If GPU_PAGE_SIZE is set to 2 MB instead of 64 kB,
registration of 72 GB of GPU memory (on a Grace Hopper) is done in
about ten seconds, not hours.<br>
<pre> <span class="n">rte_extmem_register</span><span class="p">(</span><span class="n">ext_mem</span><span class="p">.</span><span class="n">buf_ptr</span><span class="p">,</span> <span class="n">ext_mem</span><span class="p">.</span><span class="n">buf_len</span><span class="p">,</span> <span class="nb">NULL</span><span class="p">,</span> <span class="n">ext_mem</span><span class="p">.</span><span class="n">buf_iova</span><span class="p">,</span> <span class="n">GPU_PAGE_SIZE</span><span class="p">);</span></pre>
<br>
Thanks, John Romein<br>
<br>
On 05-10-2024 00:16, Stephen Hemminger wrote:<br>
</div>
<blockquote type="cite" cite="mid:20241004151608.478a2859@hermes.local">
<pre wrap="" class="moz-quote-pre">On Wed, 1 Nov 2023 22:21:16 +0100
John Romein <a class="moz-txt-link-rfc2396E" href="mailto:romein@astron.nl"><romein@astron.nl></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="" class="moz-quote-pre">Dear Slava,
Thank you for looking at the patch. With the original code, I saw that
the application spent literally hours in this function during program
start up, if tens of gigabytes of GPU memory are registered. This was
due to qsort being invoked for every new added item (to keep the list
sorted). So I tried to write equivalent code that sorts the list only
once, after all items were added. At least for our application, this
works well and is /much/ faster, as the complexity decreased from n^2
log(n) to n log(n). But I must admit that I have no idea /what/ is
being sorted, or why; I only understand this isolated piece of code (or
at least I think so). So if you think there are better ways to
initialize the list, then I am sure you will be absolutely right. But I
will not be able to implement this, as I do not understand the full
context of the code.
Kind Regards, John
</pre>
</blockquote>
<pre wrap="" class="moz-quote-pre">
Looks like the problem remains but patch has been sitting around for 11 months.
Was this resolved?
</pre>
</blockquote>
<br>
<br>
<br>
</body>
</html>