[dpdk-users] [VMXNET3] vmxnet3_dev_stats_get() segfault when called by pktgen

longtb5 at viettel.com.vn longtb5 at viettel.com.vn
Fri Sep 8 06:41:52 CEST 2017


Hi Keith,

All of the test runs that I listed gave me the same segfault from the same function in the driver. 
The config was not for any purposes other than trying to start pktgen. Maybe it is indeed a 
driver problem then.

I was able to run pktgen by replacing
cli_start_with_timers(NULL);
in pktgen_cli_start() in app/cli-functions.c, with
cli_start(NULL);

effectively disabled this flag setup 
this_cli->flags |= CLI_USE_TIMERS

With that I was able to run pktgen, but segfault still happens when I quit or want to access the stats page.

Do you have any ideas why this might happen?

Regards,
BL

----- Original Message -----
From: "keith wiles" <keith.wiles at intel.com>
To: longtb5 at viettel.com.vn
Sent: Friday, September 8, 2017 10:28:46 AM
Subject: Re: [dpdk-users] [VMXNET3] vmxnet3_dev_stats_get() segfault when called by pktgen


> On Sep 7, 2017, at 7:54 PM, longtb5 at viettel.com.vn wrote:
> 
> I tested these:
> $ sudo ./app/x86_64-native-linuxapp-gcc/pktgen
> $ sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-3 -n 2

The above will not work as you do not have the — arguments.

> $ sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-3 -n 2 -- -P -m "1.0, 2.1”

These above look fine except you use 4 cores and only use one for pktgen and 1 for each port for a total of three, you could have used -l 0-2

If the above still give the segfault then I do not know what is happening here and it does look like a driver problem??

> $ sudo ./tools/dpdk-run.py default

This one seem odd to me.
> 
> This is my default.cfg
> # Run command and options
> run = {
>    'exec': (
>        'sudo',
>        '-E'
>        ),
> 
>    # Application name and use app_path to help locate the app
>    'app_name': 'pktgen',
> 
>    # using (sdk) or (target) for specific variables
>    # add (app_name) of the application
>    # Each path is tested for the application
>    'app_path': (
>        './app/%(target)s/%(app_name)s',
>        '%(sdk)s/%(target)s/app/%(app_name)s',
>        ),
> 
> 	'dpdk': (
> 		'-l 0-1',
> 		'-n 2',
> 		'--proc-type auto',
> 		'--log-level 7',
> 		#'--socket-mem 2048,2048',
> 		#'--file-prefix pg'
> 		),
> 	
> 
> 	'app': (
> 		'-T',
> 		'-P',
> 		'--crc-strip',
> 		'-m 1.[2/5]’

Only doing one port trying to control port 2 and port 5?? Should this not be just 1.0 for one port.

> 		),
> 	
> 	'misc': (
> 		#'-f',
> 		'themes/black-yellow.theme'
> 		)
> 	}
> 
> $ sudo ~/dev/dpdk/usertools/cpu_layout.py
> cores =  [0, 1]
> sockets =  [0, 1]
> 
>       Socket 0   Socket 1  
>       --------   --------  
> Core 0 [0]        [2]       
> Core 1 [1]        [3]  
> 
> $ sudo ./dpdk/usertools/dpdk-devbind.py -s
> 
> Network devices using DPDK-compatible driver
> ============================================
> 0000:03:00.0 'VMXNET3 Ethernet Controller 07b0' drv=igb_uio unused=
> 0000:04:00.0 'VMXNET3 Ethernet Controller 07b0' drv=igb_uio unused=
> 0000:0b:00.0 'VMXNET3 Ethernet Controller 07b0' drv=igb_uio unused=
> 0000:0c:00.0 'VMXNET3 Ethernet Controller 07b0' drv=igb_uio unused=
> 0000:13:00.0 'VMXNET3 Ethernet Controller 07b0' drv=igb_uio unused=
> 0000:14:00.0 'VMXNET3 Ethernet Controller 07b0' drv=igb_uio unused=
> 0000:1b:00.0 'VMXNET3 Ethernet Controller 07b0' drv=igb_uio unused=
> 0000:1c:00.0 'VMXNET3 Ethernet Controller 07b0' drv=igb_uio unused=
> 
> Network devices using kernel driver
> ===================================
> 0000:05:00.0 'VMXNET3 Ethernet Controller 07b0' if=ens162 drv=vmxnet3 unused=igb_uio *Active*
> 0000:0d:00.0 'VMXNET3 Ethernet Controller 07b0' if=ens194 drv=vmxnet3 unused=igb_uio *Active*
> 
> 
>> 
>> ----- Original Message -----
>> From: "keith wiles" <keith.wiles at intel.com>
>> To: longtb5 at viettel.com.vn
>> Cc: "Stephen Hemminger" <stephen at networkplumber.org>, users at dpdk.org, skhare at vmware.com
>> Sent: Friday, September 8, 2017 9:22:25 AM
>> Subject: Re: [dpdk-users] [VMXNET3] vmxnet3_dev_stats_get() segfault when called by pktgen
>> 
>> 
>>> On Sep 7, 2017, at 7:07 PM, longtb5 at viettel.com.vn wrote:
>>> 
>>>> ----- Original Message -----
>>>> From: "keith wiles" <keith.wiles at intel.com>
>>>> To: "Stephen Hemminger" <stephen at networkplumber.org>
>>>> Cc: longtb5 at viettel.com.vn, users at dpdk.org
>>>> Sent: Thursday, September 7, 2017 11:49:39 PM
>>>> Subject: Re: [dpdk-users] [VMXNET3] vmxnet3_dev_stats_get() segfault when called by pktgen
>>>> 
>>>> 
>>>> I use the option EXTRA_CFLAGS on the command line to improve debug.
>>>> 
>>>> make install T=x86_64-native-linuxapp-gcc EXTRA_CFLAGS=“-g -O0”
>>>> 
>>>> Using -O0 for gdb is good for non-performance testing.
>>>> 
>>>> 
>>>> Regards,
>>>> Keith
>>> 
>>> Hi,
>>> 
>>> Thanks Keith and Stephen for the pointer. I have enabled the debug flags and acquired some more infos.
>>> 
>>> (gdb) r
>>> Thread 1 "pktgen" received signal SIGSEGV, Segmentation fault.
>>> 0x000000000075aa6c in vmxnet3_hw_tx_stats_get (hw=0x7fffb55b61c0, q=0, res=0x7fffffffdf10) at /home/tester/dpdk/drivers/net/vmxnet3/vmxnet3_ethdev.c:905
>>> 905		VMXNET3_UPDATE_TX_STAT(hw, q, ucastPktsTxOK, res);
>>> (gdb) backtrace
>>> #0  0x000000000075aa6c in vmxnet3_hw_tx_stats_get (hw=0x7fffb55b61c0, q=0, res=0x7fffffffdf10) at /home/tester/dpdk/drivers/net/vmxnet3/vmxnet3_ethdev.c:905
>>> #1  0x000000000075b3c8 in vmxnet3_dev_stats_get (dev=0xba34c0 <rte_eth_devices>, stats=0x7fffffffe000) at /home/tester/dpdk/drivers/net/vmxnet3/vmxnet3_ethdev.c:1049
>>> #2  0x00000000004a2f76 in rte_eth_stats_get (port_id=0 '\000', stats=0x7fffffffe000) at /home/tester/dpdk/lib/librte_ether/rte_ethdev.c:1340
>>> #3  0x0000000000467059 in pktgen_process_stats (tim=<optimized out>, arg=<optimized out>) at /home/tester/pktgen-dpdk/app/pktgen-stats.c:468
>>> #4  0x000000000048568a in rte_timer_manage () at /home/tester/dpdk/lib/librte_timer/rte_timer.c:593
>>> #5  0x00000000007b47dd in cli_start ()
>>> #6  0x000000000044891b in pktgen_cli_start () at /home/tester/pktgen-dpdk/app/cli-functions.c:1434
>>> #7  0x00000000004423f6 in main (argc=<optimized out>, argv=<optimized out>) at /home/tester/pktgen-dpdk/app/pktgen-main.c:470
>>> (gdb) l
>>> 900	{
>>> 901	#define VMXNET3_UPDATE_TX_STAT(h, i, f, r)		\
>>> 902			((r)->f = (h)->tqd_start[(i)].stats.f +	\
>>> 903				(h)->saved_tx_stats[(i)].f)
>>> 904	
>>> 905		VMXNET3_UPDATE_TX_STAT(hw, q, ucastPktsTxOK, res);
>>> 906		VMXNET3_UPDATE_TX_STAT(hw, q, mcastPktsTxOK, res);
>>> 907		VMXNET3_UPDATE_TX_STAT(hw, q, bcastPktsTxOK, res);
>>> 908		VMXNET3_UPDATE_TX_STAT(hw, q, ucastBytesTxOK, res);
>>> 909		VMXNET3_UPDATE_TX_STAT(hw, q, mcastBytesTxOK, res);
>>> (gdb) p hw->tqd_start
>>> $1 = (Vmxnet3_TxQueueDesc *) 0x0
>>> 
>>> The problem was on line 902.
>>> This tqd_start is the "start address of all tx queue desc" according to comments in source code. Looks like it was not initialized properly.
>>> I don't have enough knowledge to fix this. Any advice?
>> 
>> I am not able to fix this problem, but it seems like it could be pktgen command line configuration issue. Normally these types of problems are because the application tried to access a port/queue that is not >setup. The tqd_start[] is an internal array to the VMXNET3 driver, so I do not know how it relates to the DPDK  API requests for port/queue.>
>> 
>> Sorry, if you provided this before, but what is the command line you are using for Pktgen?
>> 
>>> 
>>> Regards,
>>> BL.
>> 
>> Regards,
>> Keith
> 

Regards,
Keith



More information about the users mailing list