<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:DengXian;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Times New Roman \(Body CS\)";
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:"CMU Concrete";
panose-1:2 0 6 3 0 0 0 0 0 0;}
@font-face
{font-family:"\@DengXian";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"CMU Concrete";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<div>
<div>
<p class="MsoNormal">Hi, <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"CMU Concrete""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"CMU Concrete"">Has anyone noticed that crypto_aesni_mb virtual crypto device has issue of memory crash caused by the scanning and probe on secondary process. Can anyone cast any lights on it.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"CMU Concrete"">What I encountered is:
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"CMU Concrete"">On the primary process, the crypto_aesni_mb device is probed and created successfully and I got the mb_mgr set in the device private data. But during the packet process, the application
crashes on accessing the mb_mgr. The deugging shows this mb_mgr address has been changed to an invalid address (non-NULL). Further digging shows this memory contamination occurs after the vdev_action replies the scan request.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"CMU Concrete"">In below code, the crash is gone by either disable sending message on VDEV_SCAN_REQ or skip processing the VDEV_SCAN_ONE. It seems the insert_vdev() on secondary process triggers
another probe and break the existing device data?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"CMU Concrete"">It is also noticed there was an issue which was fixed by this patch
<a href="https://review.spdk.io/gerrit/c/spdk/dpdk/+/1056">https://review.spdk.io/gerrit/c/spdk/dpdk/+/1056</a> but this patch is cancelled. This patch was complaining the similar memory issue found during scanning and probing on the secondary process.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"CMU Concrete""><o:p> </o:p></span></p>
<table class="MsoTableGrid" border="1" cellspacing="0" cellpadding="0" style="border-collapse:collapse;border:none">
<tbody>
<tr>
<td width="917" valign="top" style="width:688.1pt;border:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt">
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New"">static int<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New"">vdev_action(const struct rte_mp_msg *mp_msg, const void *peer)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New"">{<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> struct rte_vdev_device *dev;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> struct rte_mp_msg mp_resp;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> struct vdev_param *ou = (struct vdev_param *)&mp_resp.param;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> const struct vdev_param *in = (const struct vdev_param *)mp_msg->param;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> const char *devname;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> int num;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> int ret;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> strlcpy(mp_resp.name, VDEV_MP_KEY, sizeof(mp_resp.name));<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> mp_resp.len_param = sizeof(*ou);<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> mp_resp.num_fds = 0;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> switch (in->type) {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> case VDEV_SCAN_REQ:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> VDEV_LOG(INFO, "changczh skip vdev, %s", devname);<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> ou->type = VDEV_SCAN_ONE;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> ou->num = 1;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> num = 0;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> rte_spinlock_recursive_lock(&vdev_device_list_lock);<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> TAILQ_FOREACH(dev, &vdev_device_list, next) {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> devname = rte_vdev_device_name(dev);<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> if (strlen(devname) == 0) {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> VDEV_LOG(INFO, "vdev with no name is not sent");<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> continue;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> }<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> VDEV_LOG(INFO, "send vdev, %s", devname);<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> strlcpy(ou->name, devname, RTE_DEV_NAME_MAX_LEN);<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> if (rte_mp_sendmsg(&mp_resp) < 0)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> VDEV_LOG(ERR, "send vdev, %s, failed, %s",<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> devname, strerror(rte_errno));<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> num++;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> }<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> rte_spinlock_recursive_unlock(&vdev_device_list_lock);<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> ou->type = VDEV_SCAN_REP;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> ou->num = num;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> if (rte_mp_reply(&mp_resp, peer) < 0)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> VDEV_LOG(ERR, "Failed to reply a scan request");<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> break;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> case VDEV_SCAN_ONE:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> VDEV_LOG(INFO, "receive vdev, %s", in->name);<o:p></o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:12.0pt;font-family:"Courier New""> ret = insert_vdev(in->name, NULL, NULL, false);
<o:p></o:p></span></b></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> if (ret == -EEXIST)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> VDEV_LOG(DEBUG, "device already exist, %s", in->name);<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> else if (ret < 0)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> VDEV_LOG(ERR, "failed to add vdev, %s", in->name);<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> break;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> default:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> VDEV_LOG(ERR, "vdev cannot recognize this message");<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> }<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New""> return 0;<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Courier New"">}</span><span style="font-size:12.0pt;font-family:"CMU Concrete""><o:p></o:p></span></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"CMU Concrete""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"CMU Concrete""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"CMU Concrete"">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"CMU Concrete"">Alex<o:p></o:p></span></p>
</div>
</div>
</div>
</body>
</html>