<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:宋体;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:等线;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:"\@宋体";
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"\@等线";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
p.xmsonormal, li.xmsonormal, div.xmsonormal
{mso-style-name:x_msonormal;
margin:0cm;
text-align:justify;
text-justify:inter-ideograph;
font-size:10.5pt;
font-family:等线;}
span.EmailStyle23
{mso-style-type:personal-compose;
font-family:等线;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="ZH-CN" link="#0563C1" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:等线">Hi Alex,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:等线"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:等线">I forwarded the dump to you in separate mail.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:等线">Test topology:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:等线">Spirent testcenter<=>ipsec GW <=> DUT = ipsec GW <=> Spirent testcenter<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:等线">Traffic pattern:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:等线">downlink: gtpu packet length 1236, throughput: 3.65Gbps<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:等线">uplink: gtpu packet length 302, throughput 0.47Gbps<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:等线"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:等线">Crash is seen in one of the ipsec GW<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:等线"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:等线">This commit (</span><span lang="EN-CA" style="mso-fareast-language:EN-US">547b239a21</span><span lang="EN-US" style="font-size:10.5pt;font-family:等线">) is not included in my dpdk
version.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:等线"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:等线">Thank you.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:等线"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal" style="text-align:justify;text-justify:inter-ideograph"><span lang="EN-US" style="font-size:10.5pt;font-family:等线">Br, Xiaoping<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:等线"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Alexander Kozyrev <akozyrev@nvidia.com>
<br>
<b>Sent:</b> 2023</span><span style="font-family:宋体">年</span><span lang="EN-US">7</span><span style="font-family:宋体">月</span><span lang="EN-US">5</span><span style="font-family:宋体">日</span><span lang="EN-US"> 9:45<br>
<b>To:</b> Matan Azrad <matan@nvidia.com>; Xiaoping Yan (NSB) <xiaoping.yan@nokia-sbell.com>; users@dpdk.org; Dekel Peled <dekelp@nvidia.com><br>
<b>Subject:</b> [External] RE: dpdk mlx5 driver crash in rxq_cq_decompress_v<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" align="left" width="100%" style="width:100.0%">
<tbody>
<tr>
<td style="background:#FFB900;padding:5.0pt 2.0pt 5.0pt 2.0pt">
<p class="MsoNormal" style="mso-element:frame;mso-element-frame-hspace:2.25pt;mso-element-wrap:around;mso-element-anchor-vertical:paragraph;mso-element-anchor-horizontal:column;mso-height-rule:exactly">
<span lang="EN-US" style="font-size:12.0pt;font-family:宋体"> <o:p></o:p></span></p>
</td>
<td width="100%" style="width:100.0%;background:#FFF8E5;padding:5.0pt 4.0pt 5.0pt 12.0pt">
<div>
<p class="MsoNormal" style="mso-element:frame;mso-element-frame-hspace:2.25pt;mso-element-wrap:around;mso-element-anchor-vertical:paragraph;mso-element-anchor-horizontal:column;mso-height-rule:exactly">
<b><span lang="EN-US" style="font-size:12.0pt;font-family:"Arial",sans-serif;color:#222222">CAUTION:</span></b><span lang="EN-US" style="font-size:12.0pt;font-family:"Arial",sans-serif;color:#222222"> This is an external email. Please be very careful when clicking
links or opening attachments. See <a href="http://nok.it/nsb">http://nok.it/nsb</a> for additional information.<o:p></o:p></span></p>
</div>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><span lang="EN-CA" style="font-size:12.0pt;font-family:宋体"><br>
<br>
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-CA" style="mso-fareast-language:EN-US">Hi </span>
<span lang="EN-US">Xiaoping, could you please forward the error CQE dump to me?<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Would you mind elaborating more on your traffic pattern and test case scenario?<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">The following commit supposed to ignore MTU mismatch error between VF and PF:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-CA" style="mso-fareast-language:EN-US">547b239a21 net/mlx5: ignore non-critical syndromes for Rx queue<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-CA" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-CA" style="mso-fareast-language:EN-US">Regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-CA" style="mso-fareast-language:EN-US">Alex<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-CA" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Matan Azrad <<a href="mailto:matan@nvidia.com">matan@nvidia.com</a>>
<br>
<b>Sent:</b> Sunday, July 2, 2023 11:35 PM<br>
<b>To:</b> Xiaoping Yan (NSB) <<a href="mailto:xiaoping.yan@nokia-sbell.com">xiaoping.yan@nokia-sbell.com</a>>;
<a href="mailto:users@dpdk.org">users@dpdk.org</a>; Dekel Peled <<a href="mailto:dekelp@nvidia.com">dekelp@nvidia.com</a>>; Alexander Kozyrev <<a href="mailto:akozyrev@nvidia.com">akozyrev@nvidia.com</a>><br>
<b>Subject:</b> Re: dpdk mlx5 driver crash in rxq_cq_decompress_v<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span lang="EN-CA"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-CA">+ <a id="OWAAM000001" href="mailto:akozyrev@nvidia.com">
<span style="font-family:"Calibri",sans-serif;text-decoration:none">@Alexander Kozyrev</span></a> to suggest.
<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span lang="EN-CA"><o:p> </o:p></span></p>
</div>
<div id="ms-outlook-mobile-signature">
<p class="MsoNormal" dir="RTL" style="text-align:right;direction:rtl;unicode-bidi:embed">
<span lang="HE">קבל </span><span lang="AR-SA"><a href="https://aka.ms/AAb9ysg"><span lang="EN-CA" dir="LTR">Outlook</span><span dir="RTL"></span><span dir="RTL"></span><span lang="HE"><span dir="RTL"></span><span dir="RTL"></span> עבור
</span><span lang="EN-CA" dir="LTR">Android</span><span dir="RTL"></span><span dir="RTL"></span><span lang="HE"><span dir="RTL"></span><span dir="RTL"></span></span></a></span><span lang="EN-CA" dir="LTR"><o:p></o:p></span></p>
</div>
<div class="MsoNormal" align="center" style="text-align:center"><span lang="EN-CA">
<hr size="1" width="98%" align="center">
</span></div>
<div id="divRplyFwdMsg">
<p class="MsoNormal"><b><span lang="EN-CA" style="color:black">From:</span></b><span lang="EN-CA" style="color:black"> Xiaoping Yan (NSB) <<a href="mailto:xiaoping.yan@nokia-sbell.com">xiaoping.yan@nokia-sbell.com</a>><br>
<b>Sent:</b> Monday, July 3, 2023 4:18:22 AM<br>
<b>To:</b> <a href="mailto:users@dpdk.org">users@dpdk.org</a> <<a href="mailto:users@dpdk.org">users@dpdk.org</a>>; Matan Azrad <<a href="mailto:matan@nvidia.com">matan@nvidia.com</a>>;
<a href="mailto:dekelp@nvidia.com">dekelp@nvidia.com</a> <<a href="mailto:dekelp@nvidia.com">dekelp@nvidia.com</a>><br>
<b>Subject:</b> RE: dpdk mlx5 driver crash in rxq_cq_decompress_v</span><span lang="EN-CA">
<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span lang="EN-CA"> <o:p></o:p></span></p>
</div>
</div>
<div>
<table class="MsoNormalTable" border="1" cellspacing="3" cellpadding="0" style="background:#FFEB9C">
<tbody>
<tr>
<td style="padding:.75pt .75pt .75pt .75pt">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:7.5pt;font-family:"Verdana",sans-serif;color:black">External email: Use caution opening links or attachments</span></b><span lang="EN-US" style="font-size:7.5pt;font-family:"Verdana",sans-serif;color:black">
</span><span lang="EN-US"><o:p></o:p></span></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><span lang="EN-CA"><o:p> </o:p></span></p>
<div>
<div>
<p class="xmsonormal"><span lang="EN-US">Hi,</span><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-US" style="font-family:"Calibri",sans-serif"> </span><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-US"><a id="OWAAM4F32193598C54B96899A7C78B433AACA" href="mailto:dekelp@nvidia.com"><span style="font-family:等线;text-decoration:none">@'dekelp@nvidia.com'</span></a><a id="OWAAM72C297E17CCC489B9901E36573FE6073" href="mailto:matan@nvidia.com"><span style="font-family:等线;text-decoration:none">@'Matan
Azrad'</span></a> Can you kindly suggest?</span><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-US">Thank you.</span><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-US" style="font-family:"Calibri",sans-serif"> </span><span lang="EN-CA"><o:p></o:p></span></p>
<div>
<p class="xmsonormal"><span lang="EN-US">Br, Xiaoping</span><span lang="EN-CA"><o:p></o:p></span></p>
</div>
<p class="xmsonormal"><span lang="EN-US" style="font-family:"Calibri",sans-serif"> </span><span lang="EN-CA"><o:p></o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="xmsonormal" align="left" style="text-align:left"><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Xiaoping Yan (NSB)
<br>
<b>Sent:</b> 2023</span><span style="font-size:11.0pt;font-family:宋体">年</span><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">6</span><span style="font-size:11.0pt;font-family:宋体">月</span><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">27</span><span style="font-size:11.0pt;font-family:宋体">日</span><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">
12:11<br>
<b>To:</b> <a href="mailto:users@dpdk.org">users@dpdk.org</a>; 'Matan Azrad' <<a href="mailto:matan@nvidia.com">matan@nvidia.com</a>>; 'dekelp@nvidia.com' <<a href="mailto:dekelp@nvidia.com">dekelp@nvidia.com</a>><br>
<b>Subject:</b> dpdk mlx5 driver crash in rxq_cq_decompress_v</span><span lang="EN-CA"><o:p></o:p></span></p>
</div>
</div>
<p class="xmsonormal" align="left" style="text-align:left"><span lang="EN-US" style="font-family:"Calibri",sans-serif"> </span><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-US">Hi,</span><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-US" style="font-family:"Calibri",sans-serif"> </span><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-US">dpdk version in use: 21.11.2</span><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-US" style="font-family:"Calibri",sans-serif"> </span><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-US">Mlx5 driver crashes in rxq_cq_decompress_v in traffic test after several minutes.</span><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-US">Stack trace:</span><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal" style="margin-left:10.5pt"><i><span lang="EN-US">(gdb) bt</span></i><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal" style="margin-left:10.5pt"><i><span lang="EN-US">#0</span></i><i><span lang="EN-US" style="font-family:"Calibri",sans-serif"> </span><span lang="EN-US"> 0x00007ffff58612bc in _mm_storeu_si128 (__B=..., __P=<optimized out>)</span></i><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal" style="margin-left:10.5pt"><i><span lang="EN-US" style="font-family:"Calibri",sans-serif"> </span><span lang="EN-US"> at /usr/lib/gcc/x86_64-redhat-linux/12/include/emmintrin.h:739</span></i><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal" style="margin-left:10.5pt"><i><span lang="EN-US">#1</span></i><i><span lang="EN-US" style="font-family:"Calibri",sans-serif"> </span><span lang="EN-US"> rxq_cq_decompress_v (rxq=rxq@entry=0x2abe5592f40, cq=cq@entry=0x2abe54fdb00, elts=elts@entry=0x2abe5594638)</span></i><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal" style="margin-left:10.5pt"><i><span lang="EN-US" style="font-family:"Calibri",sans-serif"> </span><span lang="EN-US"> at ../dpdk-21.11/drivers/net/mlx5/mlx5_rxtx_vec_sse.h:142</span></i><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal" style="margin-left:10.5pt"><i><span lang="EN-US">#2</span></i><i><span lang="EN-US" style="font-family:"Calibri",sans-serif"> </span><span lang="EN-US"> 0x00007ffff5862c84 in rxq_burst_v (no_cq=<synthetic pointer>, err=0x7fffffffb848,
pkts_n=4, pkts=<optimized out>, </span></i><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal" style="margin-left:10.5pt"><i><span lang="EN-US" style="font-family:"Calibri",sans-serif"> </span><span lang="EN-US">rxq=0x2abe5592f40) at ../dpdk-21.11/drivers/net/mlx5/mlx5_rxtx_vec.c:349</span></i><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal" style="margin-left:10.5pt"><i><span lang="EN-US">#3</span></i><i><span lang="EN-US" style="font-family:"Calibri",sans-serif"> </span><span lang="EN-US"> mlx5_rx_burst_vec (dpdk_rxq=0x2abe5592f40, pkts=0x7fffffffbf80, pkts_n=32) at ../dpdk-21.11/drivers/net/mlx5/mlx5_rxtx_vec.c:393</span></i><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal" style="margin-left:10.5pt"><i><span lang="EN-US">#4</span></i><i><span lang="EN-US" style="font-family:"Calibri",sans-serif"> </span><span lang="EN-US"> 0x00005555556a0f41 in rte_eth_rx_burst (nb_pkts=32, rx_pkts=0x7fffffffbf80, queue_id=0,
port_id=1)</span></i><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal" style="margin-left:10.5pt"><i><span lang="EN-US" style="font-family:"Calibri",sans-serif"> </span><span lang="EN-US"> at /usr/include/rte_ethdev.h:5721</span></i><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal" style="margin-left:10.5pt"><i>…</i><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-US">Attached is the error log </span>“<span lang="EN-US">Unexpected CQE error syndrome</span>…”<span lang="EN-US"> and dump file</span><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-US" style="font-family:"Calibri",sans-serif"> </span><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-US">I found there was a similar bug here: <a href="https://bugs.dpdk.org/show_bug.cgi?id=334">
https://bugs.dpdk.org/show_bug.cgi?id=334</a></span><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-US">But the fix (88c0733535d6 extend Rx completion with error handling) should already been included, as I</span>’<span lang="EN-US">m using 21.11.2</span><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-US">Also below commit (fix to 88c0733535d6) is already included in my dpdk version.</span><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal" style="margin-left:10.5pt"><i><span lang="EN-US">commit 60b254e3923d007bcadbb8d410f95ad89a2f13fa</span></i><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal" style="margin-left:10.5pt"><i><span lang="EN-US">Author: Matan Azrad
<a href="mailto:matan@nvidia.com">matan@nvidia.com</a></span></i><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal" style="margin-left:10.5pt"><i><span lang="EN-US">Date:</span></i><i><span lang="EN-US" style="font-family:"Calibri",sans-serif"> </span><span lang="EN-US"> Thu Aug 11 19:51:55 2022 +0300</span></i><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal" style="margin-left:10.5pt"><i><span lang="EN-US" style="font-family:"Calibri",sans-serif"> </span></i><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal" style="margin-left:10.5pt"><i><span lang="EN-US" style="font-family:"Calibri",sans-serif"> </span><span lang="EN-US"> net/mlx5: fix Rx queue recovery mechanism</span></i><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-US" style="font-family:"Calibri",sans-serif"> </span><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-US">Any suggestion?</span><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-US">Thank you.</span><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-US" style="font-family:"Calibri",sans-serif"> </span><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-US">Br, Xiaoping</span><span lang="EN-CA"><o:p></o:p></span></p>
<p class="xmsonormal"><span lang="EN-US" style="font-family:"Calibri",sans-serif"> </span><span lang="EN-CA"><o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</body>
</html>