Trex with mlx5 driver - Multiple streams with different VLAN priority causes high CPU utilization

[Dariusz Sosnowski]

Hi Rubens,

Would you be able to provide the output of "ethtool -S <iface>" for both VFs before and after the test?
Does the same issue appear on this system if both parallel streams use the same VLAN priority?

Best regards,
Dariusz Sosnowski

> From: Rubens Figueiredo <rubens.figueiredo at bisdn.de> 
> Sent: Wednesday, April 17, 2024 19:07
> To: users at dpdk.org
> Subject: Trex with mlx5 driver - Multiple streams with different VLAN priority causes high CPU utilization
> 
> Hello community,
> I am facing a strange issue in the Trex stateless code, version v3.02 and v3.04. I am using the Mellanox Cx-5, and have created two VFs on top of the PF 0. The mlx5_core version I am using is the 5.7-1.0.2, and the ofed version is MLNX_OFED_LINUX-5.7-1.0.2.0 (OFED-5.7-1.0.2).
> I have created the following issue in the trex-core repository [here](https://github.com/cisco-system-traffic-generator/trex-core/issues/1124), and was recommended to post the issue in here. In the github issue you see screenshots of the issue I am facing. 
> I am trying to create two parallel streams with different VLAN priorities, but the load generated is not what I expect it to be, and CPU util. seems incredibly high (~99%).
> I have reproduced this issue with the --software and non software version.
> The script I used is below.
> import stl_path
> from trex.stl.api import *
> 
> import time
> import pprint
> from ipaddress import ip_address, ip_network
> 
> import argparse
> import configparser
> import os
> import json
> 
> 
> def get_packet(tos, mac_dst, ip_src, size):
>     # pkt = Ether(src="02:00:00:00:00:01",dst="00:00:00:01:00:01") / IP(src="10.0.0.2", tos=tos) / UDP(sport=4444, dport=4444)
> 
>     pkt = (
>         Ether(src="00:01:00:00:00:02", dst=mac_dst)
>         # Ether(dst="11:11:11:11:11:11")
>         # / Dot1AD(vlan=0)
>         / Dot1Q(vlan=0, prio=tos)
>         / IP(src=ip_src)
>         / UDP(sport=4444, dport=4444)
>     )
>     pad = max(0, size - len(pkt)) * "x"
> 
>     return pkt / pad
> 
> def main():
>     """ """
>     tx_port = 0
>     rx_port = 1
> 
>     c = STLClient()
> 
>     # connect to server
>     c.connect()
> 
>     # prepare our ports
>     c.reset(ports=[tx_port, rx_port])
> 
>     streams = []
>     s = STLStream(
>         packet=STLPktBuilder(
>             pkt=get_packet(4,"00:11:22:33:44:55", "10.1.0.2",512),
>             # vm = vm,
>         ),
>         isg=0 * 1000000,
>         mode=STLTXCont(pps=1.2*10**6),
>         # flow_stats = STLFlowLatencyStats(pg_id = 0)
>         flow_stats = STLFlowStats(pg_id=0),
>     )
> 
>     streams.append(s)
> 
>     s2 = STLStream(
>         packet=STLPktBuilder(
>             pkt=get_packet(2,"00:11:22:33:44:55", "10.1.0.2",512),
>             # vm = vm,
>         ),
>         isg=0 * 1000000,
>         mode=STLTXCont(pps=1.2*10**6),
>         # flow_stats = STLFlowLatencyStats(pg_id = 0)
>         flow_stats = STLFlowStats(pg_id=1),
>     )
> 
>     streams.append(s2)
> 
>     c.add_streams(streams, ports=[tx_port])
> 
>     c.clear_stats()
> 
>     c.start(ports=[tx_port], duration=60, mult="25gbpsl1")
> 
>     c.wait_on_traffic(ports=[tx_port, rx_port])
> 
>     stats = c.get_stats()
>     print(stats)
> 
> if __name__ == "__main__":
>     main()
> 
> 
> And the configuration is 
> - port_limit: 2
>   version: 2
>   port_bandwidth_gb: 100
>   interfaces: ["3b:00.2", "3b:00.3"]
>   port_info:
>     - dest_mac: 00:00:00:00:00:01
>       src_mac: 00:01:00:00:00:01
>     - dest_mac: 00:00:00:00:00:02
>       src_mac: 00:01:00:00:00:02
>   c: 14
>   platform:
>     master_thread_id: 8
>     latency_thread_id: 27
>     dual_if:
>       - socket: 0
>         threads: [9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26]
> 
> 
> BISDN GmbH
> Körnerstraße 7-10
> 10785 Berlin
> Germany
> 
> Phone: +49-30-6108-1-6100
> 
> Managing Directors: 
> Dr.-Ing. Hagen Woesner, Andreas Köpsel
> 
> Commercial register: 
> Amtsgericht Berlin-Charlottenburg HRB 141569 B
> VAT ID No: DE283257294
> ________________________________________