<div dir="auto">Please put this in a bugzilla report.<div dir="auto"> </div></div> <div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">On Mon, Oct 6, 2025, 12:00 bloodyevil <<a href="mailto:bloodyevil@163.com">bloodyevil@163.com</a>> wrote: </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="line-height:1.7;color:rgb(0,0,0);font-size:14px;font-family:Arial"><div style="margin:0px"><p style="white-space:normal;color:rgb(0,0,0);font-family:Arial;font-size:14px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;word-break:break-word">Dear DPDK Development Team and Chelsio Support Team,<p style="white-space:normal;color:rgb(0,0,0);font-family:Arial;font-size:14px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;word-break:break-word">We are writing to report two severe, fundamental issues we've encountered while using Chelsio T5/T6 series NICs (with the <code style="white-space:normal;word-break:break-word">cxgbe</code> PMD) in our high-performance real-time audio streaming application. These problems prevent us from leveraging core DPDK features and require your urgent attention.<p style="white-space:normal;color:rgb(0,0,0);font-family:Arial;font-size:14px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;word-break:break-word">Issue #1: Complete Failure to Transmit Zero-Copy (Multi-Segment) Packets<p style="white-space:normal;color:rgb(0,0,0);font-family:Arial;font-size:14px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;word-break:break-word">To achieve the lowest latency, we are using the standard DPDK zero-copy mechanism: attaching an external shared memory buffer (from <code style="white-space:normal;word-break:break-word">rte_memzone</code>) containing our audio payload to an mbuf header using <code style="white-space:normal;word-break:break-word">rte_pktmbuf_attach_extbuf</code>. This correctly creates a multi-segment mbuf (<code style="white-space:normal;word-break:break-word">nb_segs > 1</code>).<p style="white-space:normal;color:rgb(0,0,0);font-family:Arial;font-size:14px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;word-break:break-word">However, we have found that the <code style="white-space:normal;word-break:break-word">cxgbe</code> driver is completely unable to transmit these multi-segment mbufs. Any attempt to send such a packet via <code style="white-space:normal;word-break:break-word">rte_eth_tx_burst</code> fails (returns 0 or results in silent packet drops), regardless of whether hardware offloads are enabled or disabled. The <code style="white-space:normal;word-break:break-word">cxgbe</code> PMD's capability report (<code style="white-space:normal;word-break:break-word">tx_offload_capa</code>) correctly does not include the <code style="white-space:normal;word-break:break-word">RTE_ETH_TX_OFFLOAD_MULTI_SEGS</code> flag.<p style="white-space:normal;color:rgb(0,0,0);font-family:Arial;font-size:14px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;word-break:break-word">This means that for the <code style="white-space:normal;word-break:break-word">cxgbe</code> driver, the standard zero-copy path in DPDK is entirely non-functional.<p style="white-space:normal;color:rgb(0,0,0);font-family:Arial;font-size:14px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;word-break:break-word">Issue #2: Incomplete Hardware Checksum Offload<p style="white-space:normal;color:rgb(0,0,0);font-family:Arial;font-size:14px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;word-break:break-word">Forced to abandon zero-copy, we implemented a "single-copy" workaround by using <code style="white-space:normal;word-break:break-word">rte_memcpy</code> to create a contiguous, single-segment mbuf. While this allows packets to be transmitted, we discovered a second critical issue: the hardware checksum offload functionality is incomplete.<p style="white-space:normal;color:rgb(0,0,0);font-family:Arial;font-size:14px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;word-break:break-word">Specifically:<ol style="white-space:normal;color:rgb(0,0,0);font-family:Arial;font-size:14px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;word-break:break-word"><li style="white-space:normal;word-break:break-word">We set the full offload flags on the mbuf: <code style="white-space:normal;word-break:break-word">m->ol_flags |= RTE_MBUF_F_TX_IPV4 | RTE_MBUF_F_TX_IP_CKSUM | RTE_MBUF_F_TX_UDP_CKSUM;</code>.</li><li style="white-space:normal;word-break:break-word">We zero out both the IP and UDP checksum fields in their respective headers before transmission.</li><li style="white-space:normal;word-break:break-word">Packet captures reveal that only the UDP checksum is correctly calculated and filled in by the hardware. The IP checksum field remains zeroed, causing the packet to be treated as invalid and dropped by the network.</li><li style="white-space:normal;word-break:break-word">For the packet to be transmitted successfully, we are forced to manually calculate the IP checksum in software (<code style="white-space:normal;word-break:break-word">ip_h->hdr_checksum = rte_ipv4_cksum(ip_h);</code>) while keeping the UDP checksum field zeroed for hardware offload.</li></ol><p style="white-space:normal;color:rgb(0,0,0);font-family:Arial;font-size:14px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;word-break:break-word">This proves that although the <code style="white-space:normal;word-break:break-word">cxgbe</code> PMD reports support for <code style="white-space:normal;word-break:break-word">RTE_ETH_TX_OFFLOAD_IPV4_CKSUM</code>, it does not actually perform IP checksum offloading in practice.<p style="white-space:normal;color:rgb(0,0,0);font-family:Arial;font-size:14px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;word-break:break-word">For contrast, we must emphasize that both of the core functionalities we've described—zero-copy (multi-segment mbuf) transmission and full (IP+UDP) hardware checksum offloading—work perfectly on the same testbed when using NICs from Intel (igc/i40e) and onboard Realtek(RTL8125) controllers. This strongly suggests that the issues are specific to the Chelsio cxgbe PMD.<p style="white-space:normal;color:rgb(0,0,0);font-family:Arial;font-size:14px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;word-break:break-word">Our Dilemma<p style="white-space:normal;color:rgb(0,0,0);font-family:Arial;font-size:14px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;word-break:break-word">These two issues leave us in an untenable position:<ul style="white-space:normal;color:rgb(0,0,0);font-family:Arial;font-size:14px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;word-break:break-word"><li style="white-space:normal;word-break:break-word">The ideal zero-copy path is completely broken, preventing us from realizing a primary performance benefit of DPDK.</li><li style="white-space:normal;word-break:break-word">The fallback single-copy path is highly inefficient, as it not only incurs the CPU cost of a <code style="white-space:normal;word-break:break-word">memcpy</code> but also requires the additional CPU overhead of software IP checksum calculation, largely defeating the purpose of hardware offloads.</li></ul><p style="white-space:normal;color:rgb(0,0,0);font-family:Arial;font-size:14px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;word-break:break-word">Our Questions<p style="white-space:normal;color:rgb(0,0,0);font-family:Arial;font-size:14px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;word-break:break-word">We urgently need your help to clarify the following:<ol style="white-space:normal;color:rgb(0,0,0);font-family:Arial;font-size:14px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;word-break:break-word"><ul style="white-space:normal;word-break:break-word"><li style="white-space:normal;word-break:break-word"></li><li>Is this behavior from the <code>cxgbe</code> PMD (offloading only UDP checksum) consistent with the design expectations for a PMD in DPDK?</li><li>Does the DPDK framework provide any debugging mechanisms to trace why an explicitly set offload flag (<code>RTE_MBUF_F_TX_IP_CKSUM</code>) would be ignored by a PMD without reporting an error?</li></ul></ol><span style="white-space:normal;color:rgb(0,0,0);font-family:Arial;font-size:14px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;word-break:break-word"> Resolving these issues is critical to the success of our project. Any information or guidance you can provide would be greatly appreciated.Our Environment<ul style="white-space:normal;word-break:break-word"><li style="white-space:normal;word-break:break-word">DPDK Version: 25.07</li><li style="white-space:normal;word-break:break-word">NIC Model: <code style="white-space:normal;word-break:break-word">Chelsio T520-CR</code></li><li style="white-space:normal;word-break:break-word">OS & Kernel: Tinycore64 16.0 kernel 6.6.63</li></ul>Thank you for your time and attention to this urgent matter. We look forward to your response.Best regards, 尊敬的 DPDK 开发团队和 Chelsio 技术支持团队：您们好！我们正在开发一个对性能要求极高的实时音频流项目，但目前在使用 Chelsio T5/T6 系列网卡（<code style="white-space:normal;word-break:break-word">cxgbe</code> PMD）时，遇到了两个严重的底层功能障碍，导致 DPDK 的核心优势无法发挥。我们恳请您们的紧急援助。【核心问题一：零拷贝（多段 mbuf）数据包完全无法发送】为了实现最低延迟，我们采用 DPDK 标准的零拷贝机制：通过 <code style="white-space:normal;word-break:break-word">rte_pktmbuf_attach_extbuf</code> 函数，将外部共享内存（<code style="white-space:normal;word-break:break-word">rte_memzone</code>）中的音频数据附加到 mbuf 头部之后。此操作会创建一个多段 mbuf (<code style="white-space:normal;word-break:break-word">nb_segs > 1</code>)。然而，我们发现 <code style="white-space:normal;word-break:break-word">cxgbe</code> 驱动完全无法发送这种多段 mbuf。一旦调用 <code style="white-space:normal;word-break:break-word">rte_eth_tx_burst</code> 发送此类数据包，无论是否开启硬件卸载，发送都会失败（返回值为 0 或导致丢包），数据包无法出现在网络上。<code style="white-space:normal;word-break:break-word">cxgbe</code> PMD 的能力报告（<code style="white-space:normal;word-break:break-word">tx_offload_capa</code>）也确实不包含 <code style="white-space:normal;word-break:break-word">RTE_ETH_TX_OFFLOAD_MULTI_SEGS</code> 标志。这表明，对于 <code style="white-space:normal;word-break:break-word">cxgbe</code> 驱动而言，DPDK 的标准零拷贝机制是完全不可用的。【核心问题二：硬件校验和卸载功能不完整】为了绕过上述问题，我们被迫采用“单拷贝”的妥协方案：手动 <code style="white-space:normal;word-break:break-word">rte_memcpy</code> 数据到一个大的、连续的单段 mbuf 中。虽然这种方式可以成功发包，但我们发现了第二个严重问题：硬件卸载功能是残缺的。具体表现为：<ol style="white-space:normal;word-break:break-word"><li style="white-space:normal;word-break:break-word">我们为 mbuf 设置了完整的卸载标志：<code style="white-space:normal;word-break:break-word">m->ol_flags |= RTE_MBUF_F_TX_IPV4 | RTE_MBUF_F_TX_IP_CKSUM | RTE_MBUF_F_TX_UDP_CKSUM;</code>。</li><li style="white-space:normal;word-break:break-word">在发包前，我们将 IP 和 UDP 头部中的校验和字段都清零。</li><li style="white-space:normal;word-break:break-word">抓包分析后发现，只有 UDP 校验和被硬件正确计算并填充了。而 IP 校验和字段依然是 0，导致该包在网络中被视为无效数据包而被丢弃。</li><li style="white-space:normal;word-break:break-word">我们必须在软件中手动计算 IP 校验和（<code style="white-space:normal;word-break:break-word">ip_h->hdr_checksum = rte_ipv4_cksum(ip_h);</code>），同时保持 UDP 校验和为 0，才能让数据包正确发送并被接收端验证。</li></ol>这证实了 <code style="white-space:normal;word-break:break-word">cxgbe</code> PMD 虽然声称支持 <code style="white-space:normal;word-break:break-word">RTE_ETH_TX_OFFLOAD_IPV4_CKSUM</code>，但在实际工作中并未执行 IP 校验和的硬件卸载。作为对比，我们需要强调的是：我们描述的这两项核心功能——即零拷贝（多段 mbuf）发送和完整的硬件校验和卸载（IP+UDP），在我们的同一测试平台上，使用 Intel (igc/i40e) 和 Realtek (RTL8125) 的板载网卡时，都完全正常工作。这使我们确信，问题是特定于 Chelsio cxgbe PMD 的。【我们的困境】这两个问题使我们陷入了绝境：<ul style="white-space:normal;word-break:break-word"><li style="white-space:normal;word-break:break-word">理想的零拷贝路径完全不通，导致 DPDK 的核心性能优势无法体现。</li><li style="white-space:normal;word-break:break-word">妥协的单拷贝路径效率低下，不仅引入了 <code style="white-space:normal;word-break:break-word">memcpy</code> 的 CPU 开销，还必须额外承担 IP 校验和的软件计算开销，使得硬件卸载的价值大打折扣。</li></ul>【我们的问题】我们急需您的帮助来澄清以下问题：<ol style="white-space:normal;word-break:break-word"><ul style="white-space:normal;word-break:break-word"><li style="white-space:normal;word-break:break-word"></li><li><code>cxgbe</code> 驱动的这种行为（只卸载 UDP 校验和）是否符合 DPDK 对 PMD 的设计预期？</li><li>DPDK 框架是否有调试机制，可以追踪为何一个明确设置的卸载标志（<code>RTE_MBUF_F_TX_IP_CKSUM</code>）会被 PMD 忽略且不报告任何错</li></ul></ol> 解决这些问题对于我们的项目能否成功至关重要。任何能够帮助我们前进的建议或信息，我们将不胜感激。【我们的环境信息】<ul style="white-space:normal;word-break:break-word"><li style="white-space:normal;word-break:break-word">DPDK 版本：25.07</li><li style="white-space:normal;word-break:break-word">网卡型号：<code style="white-space:normal;word-break:break-word">Chelsio T520-CR</code></li><li style="white-space:normal;word-break:break-word">操作系统与内核版本：Tinycore64 16.0 kernel 6.6.63</li></ul>感谢您的时间和关注，我们急切地期待您的回复！此致，敬礼！ </div></div></blockquote></div>