<div dir="ltr"><div dir="ltr"></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, May 30, 2023 at 11:04 AM Bruce Richardson <<a href="mailto:bruce.richardson@intel.com">bruce.richardson@intel.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Sun, May 28, 2023 at 11:07:40PM +0300, Baruch Even wrote:<br>
>    Hi,<br>
>    We found an issue with newer kernels (5.13+) that are found on newer<br>
>    OSes (Ubuntu22, Rocky9, Ubuntu20 with kernel 5.15) where a 2M page that<br>
>    was allocated for DPDK was migrated (moved into another physical page)<br>
>    when a 1G page was allocated.<br>
>    From our reading of the kernel commits this started with commit<br>
>    ae37c7ff79f1f030e28ec76c46ee032f8fd07607<br>
>        mm: make alloc_contig_range handle in-use hugetlb pages<br>
>    This caused what looked like memory corruptions to us and cases where<br>
>    the rings were moved from their physical location and communication was<br>
>    no longer possible.<br>
>    I wanted to ask if anyone else hit this issue and what mitigations are<br>
>    available?<br>
>    We are currently looking at using a kernel driver to pin the pages but<br>
>    I expect that this issue will affect others and that a more general<br>
>    approach is needed.<br>
>    Thanks,<br>
>    Baruch<br>
>    --<br>
<br>
Hi,<br>
<br>
what kernel driver was being used for the device I/O part? Was it a UIO<br>
based driver or "vfio-pci"? When using vfio-pci and configuring IOMMU<br>
mappings, the pages mapped should be pinned by the kernel, I would have<br>
thought, since the kernel knows they are being used by devices.<br>
<br>
/Bruce<br>
</blockquote></div><div><br></div><div>This was using igb_uio on an AWS instance with their ena driver. <br></div><div><br></div><div>Baruch<br></div><br><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature"><div><table cellspacing="0" cellpadding="0" border="0"><tbody><tr><td style="font-size:0px;height:10px" height="10"></td></tr><tr><td style="font-size:0px;height:10px" height="10"></td></tr><tr><td><table style="width:510px" width="510" cellspacing="0" cellpadding="0" border="0"><tbody><tr><td><table cellspacing="0" cellpadding="0" border="0"></table></td></tr><tr><td style="padding-top:10px">
<div> <div style="max-width:510px"> <table style="width:510px" width="510" cellspacing="0" cellpadding="0" border="0"> <tbody><tr><td style="vertical-align:top;padding-right:12px;width:113px" width="113.0" valign="top"> <table cellspacing="0" cellpadding="0" border="0"> <tbody><tr> <td><img src="https://d4d8xd20er8lg.cloudfront.net/996a286d-104f-4862-86c3-38e6ff576b6b/uWcoN7kq5T52WAzl4ESe1yVt5JWBh1at.png" style="width: 113px; height: 82px; border-radius: 5px;" width="113.0" height="82.0"> </td> </tr> </tbody></table> </td><td> <table style="width:100%" width="100%" cellspacing="0" cellpadding="0" border="0"> <tbody><tr> <td style="padding-bottom:6px;border-bottom:1px solid rgb(172,172,172);font-size:16px;line-height:24px"> <span style="color:rgb(119,61,190);padding:0px;margin:0px;font-weight:bold;font-family:Arial;font-size:18px">Baruch Even</span><br> <span style="font-family:Arial">Platform Technical Lead, </span> <span style="color:rgb(119,61,190);font-family:Arial">WEKA</span> </td> </tr> <tr> <td style="padding-top:6px;padding-bottom:7px;line-height:16px"> <table style="color:rgb(68,68,68);padding-left:2px;font-weight:normal;width:100%" width="100%" cellspacing="0" cellpadding="0" border="0"> <tbody><tr> <td style="font-size:16px;line-height:24px"> <span style="color:rgb(119,61,190);font-weight:bold;font-family:Arial">E </span><a href="mailto:baruch@weka.io" style="text-decoration:none;color:rgb(78,75,76);font-family:Arial" target="_blank">baruch@weka.io</a><i style="color:rgb(255,255,255)"> ­</i><span style="color:rgb(119,61,190);font-weight:bold;font-family:Arial">W </span><a href="https://www.weka.io?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature" style="text-decoration:none;color:rgb(78,75,76);font-family:Arial" target="_blank">www.weka.io</a><i style="color:rgb(255,255,255)"> ­</i><span style="color:rgb(119,61,190);font-weight:bold;font-family:Arial"> </span><span style="color:rgb(78,75,76);font-family:Arial"></span><i style="color:rgb(255,255,255)"> ­</i>  </td> </tr> </tbody></table> </td> </tr> <tr> <td> <table style="color:rgb(78,75,76);padding-left:2px;font-weight:bold;width:100%" width="100%" cellspacing="0" cellpadding="0" border="0"> <tbody><tr> <td><table cellspacing="0" cellpadding="0" border="0"><tbody><tr></tr></tbody></table></td> </tr> </tbody></table> </td> </tr> <tr> <td style="padding-top:6px"> </td> </tr> </tbody></table> </td> </tr> </tbody></table> </div></div></td></tr><tr><td><table cellspacing="0" cellpadding="0" border="0"><tbody><tr><td style="padding-top:8px;padding-right:8px"><div style="width:100%" width="100%"> <a href="https://www.weka.io/lp/weka-named-a-2023-customers-choice-by-gartner-peer-insights/?utm_source=signature&utm_medium=email" target="_blank"> <img src="https://d4d8xd20er8lg.cloudfront.net/7ff3fdb3-08f5-45bc-9e9f-af4382dbb940/u9B16WGa53zbO0yvcA1FtyAeRH14RmVD.png" style="width: 319px; height: 83px;" width="319" height="83"> </a></div></td></tr></tbody></table></td></tr></tbody></table></td></tr><tr><td><img style="display: none;" src="https://tracy.srv.wisestamp.com/px/teams/eyJkb21haW4iOiAid2VrYS5pbyIsICJzaWduYXR1cmVfaWQiOiA0ODY3NTgyMjg3OTM3NTM2LCAidXNlcl9lbWFpbCI6ICJiYXJ1Y2hAd2VrYS5pbyIsICJwbGFuIjogIlBsYXRpbnVtIn0=.png" alt=""></td></tr></tbody></table></div></div></div>