<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:10.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="en-IT" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt">> 04/01/2022 03:41, eagostini@nvidia.com:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > From: Elena Agostini <eagostini@nvidia.com><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> ><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > Enable the possibility to make a GPU memory area accessible from<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > the CPU.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> ><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > GPU memory has to be allocated via rte_gpu_mem_alloc().<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> ><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > This patch allows the gpudev library to pin, through the GPU driver,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > a chunk of GPU memory and to return a memory pointer usable<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > by the CPU to access the GPU memory area.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> ><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > Signed-off-by: Elena Agostini <eagostini@nvidia.com><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> [...]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > +/**<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + * @warning<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + * @b EXPERIMENTAL: this API may change without prior notice.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + *<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + * Pin a chunk of GPU memory to make it accessible from the CPU<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> You should define what means "pin" exactly.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> Which properties should we expect?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Thanks for reviewing, this is the kind of discussion I wanted to have.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Maybe "pin" is too GDRCopy specific oriented.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Here I want to make a GPU memory buffer visible from the CPU. In case<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">of NVIDIA, this means the GPU memory address has to be pinned (virtual address<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">doesn't change) and dma-mapped.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Maybe the name should be more like rte_gpu_mem_to_cpu() that's more<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">explicative and generic.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + * using the memory pointer returned by the function.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> Which function should return the pointer?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> rte_gpu_mem_pin is returning an int.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Oversight, will fix it.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + * GPU memory has to be allocated via rte_gpu_mem_alloc().<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> Why pinning is not done by rte_gpu_mem_alloc()?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> Should it be a flag?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">rte_gpu_mem_alloc() allocate virtual memory on the GPU that doesn't have<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">to be necessarily shared (pinned) to make it visible from CPU.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + *<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + * @param dev_id<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + * Device ID requiring pinned memory.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + * @param size<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + * Number of bytes to pin.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + * Requesting 0 will do nothing.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + * @param ptr<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + * Pointer to the GPU memory area to be pinned.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + * NULL is a no-op accepted value.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > +<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + * @return<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + * A pointer to the pinned GPU memory usable by the CPU, otherwise NULL and rte_errno is set:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + * - ENODEV if invalid dev_id<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + * - EINVAL if reserved flags<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> Which reserved flags?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + * - ENOTSUP if operation not supported by the driver<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + * - E2BIG if size is higher than limit<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + * - ENOMEM if out of space<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> Is out of space relevant for pinning?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Yes, let me add it<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + * - EPERM if driver error<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + */<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > +__rte_experimental<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > +int rte_gpu_mem_pin(int16_t dev_id, size_t size, void *ptr);<o:p></o:p></span></p>
</div>
</div>
</body>
</html>