<div dir="ltr"><p>Hi all/Maxime/Chenbox,</p> <p>I’m observing an issue with <code>virtio_user</code> (with <code>vhost-net</code> backend) where VLAN offload doesn't seem to insert the VLAN tag, even though both <code>vlan_tci</code> and <code>ol_flags</code> are set correctly in the mbuf. We are using DPDK Version: 24.11 (custom build) and virtio_user for exception path traffic.</p> <blockquote style="margin:0 0 0 40px;border:none;padding:0px"><h3>Context</h3></blockquote> <ul> <li> <p><strong>Scenario</strong>: When a packet with <code>vlan_tci</code> set is transmitted via <code>virtio_user</code> with <code>RTE_ETH_TX_OFFLOAD_VLAN_INSERT</code> enabled, the packet received on the parent TAP interface lacks the VLAN tag. It subsequently gets dropped, since it was meant for the TAP sub-interface corresponding to the VLAN.</p> </li> <li> <p><strong>Expected</strong>: The packet should carry the VLAN tag when seen on the TAP interface, allowing it to be delivered to the correct sub-interface.</p> </li> </ul> <blockquote style="margin:0 0 0 40px;border:none;padding:0px"><h3>Observations</h3></blockquote> <ul> <li> <p>At the point of calling <code>virtio_xmit_pkts()</code>:</p> <ul> <li> <p><code>vlan_tci</code> is correctly set (e.g., 61).</p> </li> <li> <p><code>ol_flags</code> contains <code>RTE_MBUF_F_TX_VLAN</code> (bit 0) => value = 0x1.</p> </li> </ul> </li> <li> <p>We confirmed the following in GDB just before calling <code>virtqueue_enqueue_xmit()</code>:</p></li></ul><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div>(gdb) p **tx_pkts</div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div>$6 = { ..., vlan_tci = 61, ..., ol_flags = 65, ... }</div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div>(gdb) p /t tx_pkts.ol_flags</div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div>$8 = 1000001</div></blockquote></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><br></div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div>The value <code>65</code> (<code>0b1000001</code>) confirms that <code>RTE_ETH_TX_OFFLOAD_VLAN_INSERT (bit 0)</code> is set.<br>Despite this, the packet captured on the TAP interface is a plain Ethernet frame, with no VLAN tag.<br></div><div><br></div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div>Adding few more outputs more gdb</div><div><br></div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div>(gdb) p rte_eth_devices[33]</div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div>$4 = {rx_pkt_burst = 0x558180950c36 <virtio_recv_mergeable_pkts>, tx_pkt_burst = 0x558180952440 <virtio_xmit_pkts>, tx_pkt_prepare = 0x558180951fa3 <virtio_xmit_pkts_prepare>, rx_queue_count = 0x0, rx_descriptor_status = 0x0,</div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div> tx_queue_count = 0x0, tx_descriptor_status = 0x0, recycle_tx_mbufs_reuse = 0x0, recycle_rx_descriptors_refill = 0x0, data = 0x60005fd8e080, process_private = 0x0, dev_ops = 0x55818211e2a0 <virtio_eth_dev_ops>,</div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div> flow_fp_ops = 0x55818218d140 <rte_flow_fp_default_ops>, device = 0x558185a960b0, intr_handle = 0x558185a96160, link_intr_cbs = {tqh_first = 0x0, tqh_last = 0x5581825a42f8 <rte_eth_devices+547128>}, post_rx_burst_cbs = {</div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div> 0x0 <repeats 1024 times>}, pre_tx_burst_cbs = {0x0 <repeats 1024 times>}, state = RTE_ETH_DEV_ATTACHED, security_ctx = 0x0}</div></blockquote></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><br></div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div>(gdb) p *rte_eth_devices[33].data</div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div>$6 = {name = "virtio_user0", '\000' <repeats 51 times>, rx_queues = 0x60005fc2ef80, tx_queues = 0x60005fc2cf00, nb_rx_queues = 1, nb_tx_queues = 1, sriov = {active = 0 '\000', nb_q_per_pool = 0 '\000', def_vmdq_idx = 0, def_pool_q_idx = 0},</div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div> dev_private = 0x60005fc60380, dev_link = {{val64 = 34359738367, {link_speed = 4294967295, link_duplex = 1, link_autoneg = 1, link_status = 1}}}, dev_conf = {link_speeds = 0, rxmode = {mq_mode = RTE_ETH_MQ_RX_NONE, mtu = 9000,</div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div> max_lro_pkt_size = 0, offloads = 8193, reserved_64s = {0, 0}, reserved_ptrs = {0x0, 0x0}}, txmode = {mq_mode = RTE_ETH_MQ_TX_NONE, offloads = 32813, pvid = 0, hw_vlan_reject_tagged = 0 '\000', hw_vlan_reject_untagged = 0 '\000',</div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div> hw_vlan_insert_pvid = 0 '\000', reserved_64s = {0, 0}, reserved_ptrs = {0x0, 0x0}}, lpbk_mode = 0, rx_adv_conf = {rss_conf = {rss_key = 0x0, rss_key_len = 0 '\000', rss_hf = 0, algorithm = RTE_ETH_HASH_FUNCTION_DEFAULT},</div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div> vmdq_dcb_conf = {nb_queue_pools = 0, enable_default_pool = 0 '\000', default_pool = 0 '\000', nb_pool_maps = 0 '\000', pool_map = {{vlan_id = 0, pools = 0} <repeats 64 times>}, dcb_tc = "\000\000\000\000\000\000\000"}, dcb_rx_conf = {</div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div> nb_tcs = 0, dcb_tc = "\000\000\000\000\000\000\000"}, vmdq_rx_conf = {nb_queue_pools = 0, enable_default_pool = 0 '\000', default_pool = 0 '\000', enable_loop_back = 0 '\000', nb_pool_maps = 0 '\000', rx_mode = 0, pool_map = {{</div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div> vlan_id = 0, pools = 0} <repeats 64 times>}}}, tx_adv_conf = {vmdq_dcb_tx_conf = {nb_queue_pools = 0, dcb_tc = "\000\000\000\000\000\000\000"}, dcb_tx_conf = {nb_tcs = 0, dcb_tc = "\000\000\000\000\000\000\000"}, vmdq_tx_conf = {</div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div> nb_queue_pools = 0}}, dcb_capability_en = 0, intr_conf = {lsc = 0, rxq = 0, rmv = 0}}, mtu = 9000, min_rx_buf_size = 4294967295, rx_mbuf_alloc_failed = 0, mac_addrs = 0x60005ff404c0, mac_pool_sel = {0 <repeats 128 times>},</div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div> hash_mac_addrs = 0x0, port_id = 33, promiscuous = 0 '\000', scattered_rx = 0 '\000', all_multicast = 0 '\000', dev_started = 1 '\001', lro = 0 '\000', dev_configured = 1 '\001', flow_configured = 0 '\000',</div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div> rx_queue_state = "\001", '\000' <repeats 1022 times>, tx_queue_state = "\001", '\000' <repeats 1022 times>, dev_flags = 0, numa_node = -1, vlan_filter_conf = {ids = {0 <repeats 64 times>}}, owner = {id = 0,</div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div> name = '\000' <repeats 63 times>}, representor_id = 0, backer_port_id = 64, flow_ops_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}},</div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div> __size = '\000' <repeats 39 times>, __align = 0}}</div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><br></div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div>(gdb) p *(struct virtio_user_dev*)rte_eth_devices[33].data.dev_private</div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div>$3 = {hw = {vqs = 0x60005ff40400, guest_features = 4563441697, vtnet_hdr_size = 12, started = 1 '\001', weak_barriers = 1 '\001', vlan_strip = 1 '\001', rx_ol_scatter = true, has_tx_offload = 1 '\001', has_rx_offload = 0 '\000',</div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div> use_vec_rx = 0 '\000', use_vec_tx = 0 '\000', use_inorder_rx = 0 '\000', use_inorder_tx = 0 '\000', opened = 1 '\001', port_id = 33, mac_addr = "\000PV\235#g", get_speed_via_feat = false, speed = 4294967295, duplex = 1 '\001',</div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div> intr_lsc = 1 '\001', max_mtu = 9698, max_rx_pkt_len = 9030, state_lock = {locked = 0}, inject_pkts = 0x0, max_queue_pairs = 1, rss_rx_queues = 0, rss_hash_types = 0, rss_reta = 0x0, rss_key = 0x0,</div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div> req_guest_features = 9223372445160806441, cvq = 0x0, use_va = true}, backend_type = VIRTIO_USER_BACKEND_VHOST_KERNEL, is_server = false, callfds = 0x6000396000c0, kickfds = 0x60005ff40880, mac_specified = 1, max_queue_pairs = 1,</div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div> queue_pairs = 1, queue_size = 2048, features = 4563441697, device_features = 4563442051, frontend_features = 32, unsupported_features = 17293822186582074972, status = 15 '\017', net_status = 0, mac_addr = "\000PV\235#g",</div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div> path = "/dev/vhost-net", '\000' <repeats 4081 times>, ifname = 0x558185a04b70 "avi_eth1", vrings = {ptr = 0x60005ff40780, split = 0x60005ff40780, packed = 0x60005ff40780}, packed_queues = 0x0, qp_enabled = 0x60005ff406c0,</div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div> ops = 0x55818218cc80 <virtio_ops_kernel>, mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>,</div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div> __align = 0}, started = true, hw_cvq = false, scvq = 0x0, backend_data = 0x558185a96100, notify_area = 0x0}</div></blockquote></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><ul> </ul> <h3>Workaround</h3> <p>If we explicitly insert the VLAN tag into the packet data (i.e., inline VLAN insertion), the TAP interface sees the tag as expected, and the packet is correctly forwarded to the intended sub-interface.</p> <h3>Conclusion</h3> <p>It appears that the VLAN offload flag and <code>vlan_tci</code> are not being honored or processed correctly somewhere in the virtio path with <code>vhost-net</code> backend. This leads to silent drops on the host kernel side due to missing VLAN tags.</p> <h3>Request</h3> <p>Could you please confirm if this is a known limitation or a bug? If it’s the latter, I’d be happy to provide more debug info or steps to reproduce.</p><p><br></p><p>Thanks,<br> Samar Yadav<br> Broadcom<br></p></blockquote></div>