<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hi Folks,</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Reading the docs for the the 22.11 release it clearly states that the DQO_RDA queue format is not yet supported:<br>
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<a href="https://doc.dpdk.org/guides-22.11/nics/gve.html" id="LPlnk936366">https://doc.dpdk.org/guides-22.11/nics/gve.html</a></div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
I'm attempting to bring up our software router on GCP (VM instance type c4-standard-8) where we are currently using dpdk 22.11 (via debian, with a 6.1.0-26-amd64 kernel), given the lack of support for DQO_RDA I see the following expected messages when I start
vpp (our dpdk application):</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Oct 29 11:11:59 gcpt2 vnet[1919]: dpdk: gve_adminq_describe_device(): Driver is running with DQO RDA queue format.</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Oct 29 11:11:59 gcpt2 vnet[1919]: dpdk: gve_adminq_describe_device(): MAC addr: 42:01:0A:07:00:0F</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Oct 29 11:11:59 gcpt2 vnet[1919]: dpdk: gve_init_priv(): Max TX queues 2, Max RX queues 2</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Oct 29 11:11:59 gcpt2 vnet[1919]: dpdk: gve_dev_init(): DQO_RDA is not implemented and will be added in the future</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
As a result vpp sefaults shortly after:</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
(gdb) bt</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
#0 gve_adminq_create_tx_queue (queue_index=0, priv=0xbc03a59c0) at ../drivers/net/gve/base/gve_adminq.c:502</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
#1 gve_adminq_create_tx_queues (priv=0xbc03a59c0, num_queues=2) at ../drivers/net/gve/base/gve_adminq.c:516</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
#2 0x00007ffda6c9c5d6 in gve_dev_start (dev=0x7ffe1f4f7240 <rte_eth_devices>) at ../drivers/net/gve/gve_ethdev.c:181</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
#3 0x00007ffe1f489b6a in rte_eth_dev_start () from /lib/x86_64-linux-gnu/librte_ethdev.so.23</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
#4 0x00007ffff584aa14 in dpdk_device_start (xd=xd@entry=0x7ffe204fbac0) at ./src/plugins/dpdk/device/common.c:393</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
#5 0x00007ffff584c598 in dpdk_interface_admin_up_down (vnm=<optimized out>, hw_if_index=<optimized out>, flags=<optimized out>)</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
at ./src/plugins/dpdk/device/device.c:485</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
#6 0x00007ffff6ab3ccd in vnet_sw_interface_set_flags_helper (vnm=vnm@entry=0x7ffff7f6b660 <vnet_main>, sw_if_index=<optimized out>, </div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
flags=VNET_SW_INTERFACE_FLAG_ADMIN_UP, helper_flags=0, helper_flags@entry=VNET_INTERFACE_SET_FLAGS_HELPER_WANT_REDISTRIBUTE)</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
at ./src/vnet/interface.c:545</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
#7 0x00007ffff6ab489f in vnet_sw_interface_set_flags (vnm=vnm@entry=0x7ffff7f6b660 <vnet_main>, sw_if_index=<optimized out>, </div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
flags=<optimized out>) at ./src/vnet/interface.c:601</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
#8 0x00007ffff6accee9 in vl_api_sw_interface_set_flags_t_handler (mp=0x7ffe3f241ff8) at ./src/vnet/interface_api.c:100</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
#9 0x00007ffff7fa6e0d in msg_handler_internal (free_it=0, do_it=1, trace_it=<optimized out>, msg_len=<optimized out>, </div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
the_msg=0x7ffe3f241ff8, am=0x7ffff7fb8f40 <api_global_main>) at ./src/vlibapi/api_shared.c:580</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
#10 vl_msg_api_handler_no_free (the_msg=0x7ffe3f241ff8, msg_len=<optimized out>) at ./src/vlibapi/api_shared.c:652</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
#11 0x00007ffff7f8ed7f in vl_socket_process_api_msg (rp=<optimized out>, input_v=<optimized out>) at ./src/vlibmemory/socket_api.c:208</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
#12 0x00007ffff7f978d3 in vl_api_clnt_process (vm=<optimized out>, node=<optimized out>, f=<optimized out>)</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
at ./src/vlibmemory/memclnt_api.c:429</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
#13 0x00007ffff6853966 in vlib_process_bootstrap (_a=<optimized out>) at ./src/vlib/main.c:1223</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
#14 0x00007ffff67ba03c in clib_calljmp () at /usr/src/packages/BUILD/src/vppinfra/longjmp.S:123</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
#15 0x00007ffe1edfcd90 in ?? ()</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
#16 0x00007ffff6855054 in vlib_process_startup (f=0x0, p=0x7ffe2078f780, vm=0x7ffe1fa007c0) at ./src/vlib/main.c:1248</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
#17 dispatch_process (vm=0x7ffe1fa007c0, p=<optimized out>, last_time_stamp=<optimized out>, f=0x0) at ./src/vlib/main.c:1304</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
#18 0x0000000000000000 in ?? ()</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
(gdb)</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
The segfault occurs due to the complq pointer here being NULL:</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
cmd.create_tx_queue.tx_comp_ring_addr =</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
cpu_to_be64(txq->complq->tx_ring_phys_addr);</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
It may be possible to upgrade to a more recent version of dpdk which should help me progress, though before I do that I wondered if there is any other way to make progress here? Specifically, is there a way to force a different queue format in the device? This
is a c4-standard-8 VM running in GCE.<br>
<br>
Thanks!<br>
Peter.</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
</body>
</html>