<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:10.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="en-IT" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt">> From: Stephen Hemminger <stephen@networkplumber.org><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> Date: Monday, 3 January 2022 at 19:05<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> To: Elena Agostini <eagostini@nvidia.com><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> Cc: dev@dpdk.org <dev@dpdk.org><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> Subject: Re: [PATCH v1 3/3] gpu/cuda: mem alloc aligned memory<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> External email: Use caution opening links or attachments>
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> On Tue, 4 Jan 2022 01:47:21 +0000<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> <eagostini@nvidia.com> wrote:>
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > static int<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > -cuda_mem_alloc(struct rte_gpu *dev, size_t size, void **ptr)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > +cuda_mem_alloc(struct rte_gpu *dev, size_t size, void **ptr, unsigned int align)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > CUresult res;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > const char *err_string;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > @@ -610,8 +612,10 @@ cuda_mem_alloc(struct rte_gpu *dev, size_t size, void **ptr)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> ><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > /* Allocate memory */<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > mem_alloc_list_tail->size = size;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > - res = pfn_cuMemAlloc(&(mem_alloc_list_tail->ptr_d),<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > - mem_alloc_list_tail->size);<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + mem_alloc_list_tail->size_orig = size + align;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > +<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + res = pfn_cuMemAlloc(&(mem_alloc_list_tail->ptr_orig_d),<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + mem_alloc_list_tail->size_orig);<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > if (res != 0) {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > pfn_cuGetErrorString(res, &(err_string));<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > rte_cuda_log(ERR, "cuCtxSetCurrent current failed with %s",<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > @@ -620,6 +624,13 @@ cuda_mem_alloc(struct rte_gpu *dev, size_t size, void **ptr)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > return -rte_errno;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > }<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> ><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > +<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + /* Align memory address */<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + mem_alloc_list_tail->ptr_d = mem_alloc_list_tail->ptr_orig_d;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + if (align && ((uintptr_t)mem_alloc_list_tail->ptr_d) % align)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + mem_alloc_list_tail->ptr_d += (align -<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> > + (((uintptr_t)mem_alloc_list_tail->ptr_d) % align));>
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> Posix memalign takes size_t for both size and alignment.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">I've created this patch based on the rte_malloc function definition for consistency.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">void * rte_malloc(const char *type, size_t size, unsigned align)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> Better to put the input parameters first, and then the resulting output parameter last<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> for consistency; follows the Rusty API design manifesto.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Got it, will do.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> Alignment only makes sense if power of two. The code should check that and optimize<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> for that.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">The alignment value is checked in the gpudev library before<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">passing it to the driver.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Adding this kind of checks in the driver has been rejected in the past because it was<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">considered dead code (the library was already checking input parameters).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Let me know what are the preferred options.<o:p></o:p></span></p>
</div>
</div>
</body>
</html>