[dpdk-dev] dpdk-2.0.0: crash in ixgbe_recv_scattered_pkts_vec->_recv_raw_pkts_vec->desc_to_olflags_v

[Bruce Richardson]

On Tue, Jun 30, 2015 at 08:49:32AM -0700, Gopakumar Choorakkot Edakkunni wrote:
> Hi,
> 
> I am starting to tryout dpdk-2.0.0 with a simple Rx routine very
> similar to the l2fwd example - I am running this on a c3.8xlarge aws
> sr-iov enabled vpc instance (inside the vm it uses ixgbevf driver).
> 
> Once in every 10 minutes my application crashes in the recieve path.
> And whenever I check the crash reason its because it always has three
> packets in the burst array (I have provided array size of 32) instead
> of the four that it tries to collect in one bunch. And inside
> desc_to_olflags_v(), theres the assumption that there are four
> packets, and obviously it crashes trying to access the fourth buffer.
> 
> With a brief look at the code, I really cant make out how its
> guaranteed that we will always have four descriptors fully populated ?
> After the first iteration, the loop does break out if (likely(var !=
> RTE_IXGBE_DESCS_PER_LOOP)), but how about the very first iteration
> where we might not have four ?
> 
> Any thoughts will be helpful here, trying to get my app working for
> more than 10 minutes :)
> 

The code is designed to work off the fact that it will always process four
descriptors at a time, and fill in the contents of four mbufs. The main loop
will always do the work to receive four packets, and then subsequently make
a decision as to how many of the four are actually valid packets. If the 4th
descriptor processed has not actually been written back by the NIC, then we
in effect just "throw away" the work we have done for that packet. The mbuf
that was just filled in by the receive, will be filled in again later when
the descriptor has actually been written back. This way we can get linear code
without branching for each packet, at the cost of some additional instructions
for packets that are not yet ready. [And if packets are not yet ready, then
the software is working faster than packets are arriving so we have spare cycles
to spend doing the extra bit of work].

As for the specific problem you are seeing, I'll have to try and reproduce it.
My initial test [with a 10G NIC on the host not a VM - they use the same RX path]
sending in 3 packets at a time, did not show up any issues. Can you perhaps
isolate any further the root cause of the issue. For example, does it only
occur when you get three packets at the receive ring wraps back around to zero?

Regards,
/Bruce