[dpdk-users] TimeStamping Packets Generated and Received via Pktgen Application

[Huynhtu Dang]

Hello Emmerich,

MoonGen is really helpful in measuring performance of network devices.
I wonder if we could get some information about packet loss
while running timestamps-software.lua?

Thanks,
Tu

On Oct 17, 2016, at 12:41 PM, Paul Emmerich <emmericp at net.in.tum.de<mailto:emmericp at net.in.tum.de>> wrote:

Hi,

Ajinkya D Kadam:
I was reading through your paper and I think this tool will be much more
helpful to me. Btw I am using quad X710 and dual X520 NICs.
Is this [1] the right code to look at if i want to see how you have
achieved hardware based time stamping ?

Yes, run this example script with two directly connected ports for a simple demo and test of your hardware's capabilities. It will work with both of your NICs.

In addition, I want to confirm my understanding of why MoonGen is better
than PktGen in time stamping context.
PktGen reads the value of rdtsc which it then appends to packet, this
adds more delay and hence the precision is bad.

Software timestamping by writing the TSC to the packet is also supported (but the API is less nice, see issue #153):

See examples/timestamping-tests/timestamps-software.lua for an example.

The main problem is that there is unpredictable jitter from the NIC and PCIe transfer and other random errors. Especially if you are running this at higher packet rates.
This leads to the 200-300ns random error that I've previously mentioned.


In case of MoonGen how does this work ? I am not sure. Could you please
elaborate ?

MoonGen enables the hardware timestamping feature of the NIC and uses it. The NIC will store the timestamp in a register which needs to be read before another packet can be timestamped, this limits the throughput of timestamped packets. However, I've found that you rarely need to timestamp *all* packets in a packet generator. You'll have to use software timestamping if you really need that.


Paul


Thanks,
Ajinkya


[1] https://github.com/libmoon/libmoon/blob/b5f6c2cac42c02db64073b57dd8ca82692d3858c/examples/hardware-timestamping.lua

ᐧ

On Sun, Oct 16, 2016 at 6:55 PM, Paul Emmerich <emmericp at net.in.tum.de<mailto:emmericp at net.in.tum.de>
<mailto:emmericp at net.in.tum.de>> wrote:

   Hi,


   Ajinkya D Kadam:

       If yes I would like to modify the pktgen code so that each
       transmitting and
       received packet is timestamped.  Right now I am exploring the
       example
       applications  like rxtx_callbacks which timestamps packets in
       DPDK, Is this
       the right direction to go ?


   Check out my packet generator MoonGen
   https://github.com/emmericp/MoonGen
   <https://github.com/emmericp/MoonGen>

   It uses the hardware timestamping features (PTP) to do latency
   measurements in the nanosecond-range.

   However, if you will run into hardware limitations if you want to
   timestamp *all* packets. This is sometimes supported on RX (e.g.,
   i310, X550) but I don't know a NIC that supports this on TX.

   As for the precision that is achievable: ~10ns (depending on the
   NIC) with hardware support. Software timestamping will typically
   result in a standard deviation of 200-300ns under load and there
   will be huge outliers.


    Paul