<div dir="ltr">This was my error. I linked the code incorrectly. For those new to DPDK the correct linking methodology is, using cmake, follows:<br><br>- build DPDK from <a href="http://dpdk.org">dpdk.org</a>. For example see <a href="https://github.com/rodgarrison/reinvent">https://github.com/rodgarrison/reinvent</a> then see <a href="https://github.com/rodgarrison/reinvent/blob/main/scripts/install#L58">https://github.com/rodgarrison/reinvent/blob/main/scripts/install#L58</a>. While this write-up is for DPDK on AWS ENA NICs, building DPDK is the same no matter what NIC you are using.<br><br>- configure your linking libraries with the help of pkg-config. I do it this way <a href="https://github.com/rodgarrison/reinvent/blob/ubuntu/CMakeLists.txt#L27">https://github.com/rodgarrison/reinvent/blob/ubuntu/CMakeLists.txt#L27</a> to setup the DPDK linking command defaulting to dynamic libraries. But static link is also possible; see else subcommand. Then, in the actual, application use this setup, for example, <a href="https://github.com/rodgarrison/reinvent/blob/ubuntu/integration_tests/reinvent_dpdk_udp/CMakeLists.txt#L43">https://github.com/rodgarrison/reinvent/blob/ubuntu/integration_tests/reinvent_dpdk_udp/CMakeLists.txt#L43</a><br><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Apr 7, 2022 at 4:22 AM Erez Ferber <<a href="mailto:erezferber@gmail.com">erezferber@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">I assume your tree assumes there's a ConnectX-3 device installed, while the kernel driver doesnt support it for quite a while<div>I would suggest re-compiling while excluding mlx4 PMD support.</div><div><br></div><div>Thanks,</div><div>Erez</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, 5 Apr 2022 at 23:54, fwefew 4t4tg <<a href="mailto:7532yahoo@gmail.com" target="_blank">7532yahoo@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Mr. Ferber, much appreciated. I knew this metal box came to me with two mellanox NICs bonded. I used their util to unbond it, but alas it did not do it all the way. The /etc/network/interfaces was bad. I fixed the config and rebooted.<br><br>Voila. Success. ibv_devinfo now shows two devices. And the DPDK app runs.<div><br>Your point about bonding nudged me to double check.<br><br><b>Now, on a second machine where I believe I have done everything I did on the first machine</b>, running the DPDK application continues to look for the wrong driver:<br><br>EAL: Detected CPU lcores: 16<br>EAL: Detected NUMA nodes: 1<br>EAL: Detected shared linkage of DPDK<br>EAL: libmlx4.so.1: cannot open shared object file: No such file or directory<br>EAL: FATAL: Cannot init plugins<br>EAL: Cannot init plugins<br></div><div><br></div><div>Somehow I fixed this on the first machine, but I cannot duplicate success on the second machine.</div><div><br></div><div>Both machines report two devices; both machines show similar ifconifgs<br><br>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>device <span> <span style="white-space:pre-wrap"> </span> </span>node GUID</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>------<span> <span style="white-space:pre-wrap"> </span></span>----------------</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>mlx5_0<span> <span style="white-space:pre-wrap"> </span></span>0c42a103007ea9b8</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>mlx5_1<span> <span style="white-space:pre-wrap"> </span></span>0c42a103007ea9b9</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>device <span> <span style="white-space:pre-wrap"> </span> </span>node GUID</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>------<span> <span style="white-space:pre-wrap"> </span></span>----------------</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>mlx5_0<span> <span style="white-space:pre-wrap"> </span></span>0c42a103007ea3ec</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">
</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>mlx5_1<span> <span style="white-space:pre-wrap"> </span></span>0c42a103007ea3ed</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">root@server:~/Dev/reinvent/scripts# ibv_devinfo</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">hca_id:<span style="white-space:pre-wrap"> </span>mlx5_0</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>transport:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>InfiniBand (0)</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>fw_ver:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>14.32.1010</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>node_guid:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>0c42:a103:007e:a3ec</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>sys_image_guid:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>0c42:a103:007e:a3ec</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>vendor_id:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>0x02c9</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>vendor_part_id:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>4117</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>hw_ver:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>0x0</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>board_id:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>MT_2420110034</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>phys_port_cnt:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>1</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>port:<span style="white-space:pre-wrap"> </span>1</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>state:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>PORT_ACTIVE (4)</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>max_mtu:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>4096 (5)</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>active_mtu:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>1024 (3)</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>sm_lid:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>0</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>port_lid:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>0</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>port_lmc:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>0x00</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>link_layer:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>Ethernet</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0);min-height:13px"><span style="font-variant-ligatures:no-common-ligatures"></span><br></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">hca_id:<span style="white-space:pre-wrap"> </span>mlx5_1</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>transport:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>InfiniBand (0)</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>fw_ver:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>14.32.1010</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>node_guid:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>0c42:a103:007e:a3ed</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>sys_image_guid:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>0c42:a103:007e:a3ec</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>vendor_id:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>0x02c9</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>vendor_part_id:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>4117</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>hw_ver:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>0x0</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>board_id:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>MT_2420110034</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>phys_port_cnt:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>1</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>port:<span style="white-space:pre-wrap"> </span>1</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>state:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>PORT_ACTIVE (4)</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>max_mtu:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>4096 (5)</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>active_mtu:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>1024 (3)</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>sm_lid:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>0</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>port_lid:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>0</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>port_lmc:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>0x00</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">
</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>link_layer:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>Ethernet</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br><br><br></span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">ibv_devinfo<span> </span></span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">hca_id:<span style="white-space:pre-wrap"> </span>mlx5_0</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>transport:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>InfiniBand (0)</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>fw_ver:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>14.32.1010</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>node_guid:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>0c42:a103:007e:a9b8</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>sys_image_guid:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>0c42:a103:007e:a9b8</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>vendor_id:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>0x02c9</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>vendor_part_id:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>4117</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>hw_ver:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>0x0</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>board_id:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>MT_2420110034</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>phys_port_cnt:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>1</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>port:<span style="white-space:pre-wrap"> </span>1</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>state:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>PORT_ACTIVE (4)</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>max_mtu:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>4096 (5)</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>active_mtu:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>1024 (3)</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>sm_lid:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>0</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>port_lid:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>0</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>port_lmc:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>0x00</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>link_layer:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>Ethernet</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0);min-height:13px"><span style="font-variant-ligatures:no-common-ligatures"></span><br></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">hca_id:<span style="white-space:pre-wrap"> </span>mlx5_1</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>transport:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>InfiniBand (0)</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>fw_ver:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>14.32.1010</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>node_guid:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>0c42:a103:007e:a9b9</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>sys_image_guid:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>0c42:a103:007e:a9b8</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>vendor_id:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>0x02c9</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>vendor_part_id:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>4117</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>hw_ver:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>0x0</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>board_id:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>MT_2420110034</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span>phys_port_cnt:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>1</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>port:<span style="white-space:pre-wrap"> </span>1</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>state:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>PORT_ACTIVE (4)</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>max_mtu:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>4096 (5)</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>active_mtu:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>1024 (3)</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>sm_lid:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>0</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>port_lid:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>0</span></p><p style="margin:0px;font:11px Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>port_lmc:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>0x00</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">
</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>link_layer:<span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>Ethernet</span></p></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Apr 5, 2022 at 1:00 PM Erez Ferber <<a href="mailto:erezferber@gmail.com" target="_blank">erezferber@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi,<div><br></div><div>Based on your output, the ConnectX-4LX device is configured in LAG mode managed via the kernel bonding scripts. In this mode, both physical functions share a single port (mlx5_bond_0). You should only probe the first PCI BDF - 01:00.0, not the 2nd one.</div><div><br></div><div>By the way, the --dpdk installation flag should not be necessary, it is an old flag keps for Mellanox OFED builds lower than 5.x.</div><div><br></div><div>Regards,</div><div>Erez</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, 5 Apr 2022 at 19:17, fwefew 4t4tg <<a href="mailto:7532yahoo@gmail.com" target="_blank">7532yahoo@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">I built the current version of DPDK directly from <a href="http://dpdk.org" target="_blank">dpdk.org</a> after I installed the current OFED Mellanox driver set:<br><br>* MLNX_OFED_LINUX-5.5-1.0.3.2-ubuntu20.04-x86_64.iso<br><br>with ./install --dpdk<br><br>I am using a Mellanox Technologies MT27710 Family [ConnectX-4 Lx] which is Ethernet only; there is no IB mode for this NIC. This is a MT_2420110034 board. However, when I run dpdk-testpmd I see "No Verbs device matches PCI device 0000:01:00.1, are kernel drivers loaded?"<br><br>EAL: Detected CPU lcores: 16<br>EAL: Detected NUMA nodes: 1<br>EAL: Detected static linkage of DPDK<br>EAL: Selected IOVA mode 'PA'<br>EAL: No free 2048 kB hugepages reported on node 0<br>EAL: VFIO support initialized<br>EAL: Probe PCI driver: mlx5_pci (15b3:1015) device: 0000:01:00.1 (socket 0)<br>mlx5_common: No Verbs device matches PCI device 0000:01:00.1, are kernel drivers loaded? <br>mlx5_common: Verbs device not found: 01:00.1<br>mlx5_common: Failed to initialize device context.<br>EAL: Requested device 0000:01:00.1 cannot be used<br>EAL: Bus (pci) probe failed.<br><br>As far as I can see all the kernel modules are loaded:<br><br>lsmod | egrep "(ib|mlx)" | sort<br>ib_cm 53248 2 rdma_cm,ib_ipoib<br>ib_core 368640 8 rdma_cm,ib_ipoib,iw_cm,ib_umad,rdma_ucm,ib_uverbs,mlx5_ib,ib_cm<br>ib_ipoib 135168 0<br>ib_umad 24576 0<br>ib_uverbs 139264 2 rdma_ucm,mlx5_ib<br>libahci 36864 1 ahci<br>libcrc32c 16384 2 btrfs,raid456<br>mlx5_core 1634304 1 mlx5_ib<br>mlx5_ib 397312 0<br>mlx_compat 69632 11 rdma_cm,ib_ipoib,mlxdevm,iw_cm,ib_umad,ib_core,rdma_ucm,ib_uverbs,mlx5_ib,ib_cm,mlx5_core<br>mlxdevm 172032 1 mlx5_core<br>mlxfw 32768 1 mlx5_core<br>pci_hyperv_intf 16384 1 mlx5_core<br>psample 20480 1 mlx5_core<br>tls 94208 2 bonding,mlx5_core<br><br>root@dc-c3-small-x86-01:~/Dev/reinvent/scripts# mst status -v<br>MST modules:<br>------------<br> MST PCI module is not loaded<br> MST PCI configuration module loaded<br>PCI devices:<br>------------<br>DEVICE_TYPE MST PCI RDMA NET NUMA <br>ConnectX4LX(rev:0) /dev/mst/mt4117_pciconf0.1 01:00.1 mlx5_bond_0 net-bond0 -1 <br><br>ConnectX4LX(rev:0) /dev/mst/mt4117_pciconf0 01:00.0 mlx5_bond_0 net-bond0 -1 <br></div>
</blockquote></div>
</blockquote></div>
</blockquote></div>
</blockquote></div>