[dpdk-users] rte_eth_tx_burst: Can I insert timing gaps

Paul Emmerich emmericp at net.in.tum.de
Mon Oct 15 21:08:51 CEST 2018

> Am 15.10.2018 um 16:29 schrieb Cliff Burdick <shaklee3 at gmail.com>:
> My best guess would be that you can send the filler
> packets to your own MAC address so that the switch will drop it.

A variant of that (with broken CRC checksums; require patched drivers)
of that yielded the best results in our evaluation linked above.

Our implementation for this is here:

Implementation of a busy-wait based rate limiter that is often good enough is here:

It runs in a separate thread that is fed through an rte_ring, so there is only one tight loop
on that core that sleeps and sends out packets.


> On Mon, Oct 15, 2018, 04:21 Andrew Bainbridge <andbain at microsoft.com> wrote:
>> Is the feature you are describing is called packet "pacing"? Here's a
>> Mellanox document describing it:
>>  https://community.mellanox.com/docs/DOC-2551
>> I grep'ed the DPDK source for "rate_limit" and found
>> rte_eth_set_queue_rate_limit(). Is that the function you need?
>> From my quick grep'ing, it looks to me like this feature isn't supported
>> on Mellanox drivers but is in the ixgbe driver. However, this is all guess
>> work. I'm not an expert. I'd like to know the real answer!
>> - Andrew
>> -----Original Message-----
>> From: users <users-bounces at dpdk.org> On Behalf Of Philipp B
>> Sent: 15 October 2018 11:45
>> To: shaklee3 at gmail.com
>> Cc: users at dpdk.org
>> Subject: Re: [dpdk-users] rte_eth_tx_burst: Can I insert timing gaps
>> Maybe I should explain with some ASCII art. It's not about sleeping just
>> the remaining period of 20ms. This is exactly what I am doing already, and
>> this works fine.
>> Think of 20ms intervals like this
>> Where '|' is the start of the 20ms interval, X is a packet, and _ is an
>> idle gap of one packet. (Let's just pretend there are 1000s of Xs per
>> interval).
>> As I said, I let the CPU sleep upto the beginning of the interval ('|').
>> This beginning of interval is the only moment where CPU timing controls
>> adapter timing. Then I send out a few 1000 packets. In this phase, I have a
>> few 100 packets buffered by DPDK, so it will not help to sleep on the CPU.
>> The pattern above is what I can easily produce just with an OS sleep, a
>> single buffer pool and rte_eth_tx_burst. What I am looking for, is a way to
>> e.g. remove every second packet from that pattern, while keeping the other
>> packet's timings unchanged:
>> |X_X_X_X_X_X_X_X_X______|X_X_X_X_X_X_X_X_X______
>> Basically, I do not need to transmit anything in the gaps. I just need the
>> delay. However, as my timing of CPU isn't coupled tightly to the adapter,
>> sleeping on the cpu will not help. This is intended by
>> design: I want to blow out a massive number of packets with exact timing
>> and virtually no CPU requirement.
>> What I look for is a sleep instruction executed by the adapter, which is
>> buffered in order with the packets issued by rte_eth_tx_burst.
>> (Plus some basic math rules how to convert packet sizes to durations,
>> based on line speeds).
>> Am Sa., 13. Okt. 2018 um 23:05 Uhr schrieb Cliff Burdick <
>> shaklee3 at gmail.com>:
>>> Maybe I'm misunderstanding the problem, but do you need to transmit
>> anything? Can you just use the rte_cycles functions to sleep for the
>> remaining period in 20ms?
>>> On Thu, Oct 11, 2018 at 2:04 AM Philipp B <philippb.ontour at gmail.com>
>> wrote:
>>>> Hi all!
>>>> I am working on an RTP test traffic generator. The basic idea is
>>>> clock_nanosleep providing a 20ms clock cycle to start a (big) number
>>>> of rte_eth_tx_bursts, sending equally sized buffers. As long as the
>>>> timing within a 20ms cycle is dictated primarily by the line speed, I
>>>> can be sure that not just the first buffer of each cycle has a period
>>>> of 20ms, but also the n-th buffer. (I have sent n-1 buffers before
>>>> with the same size.)
>>>> Basically, I see one 20ms interval as a series of time slots, each
>>>> capable to store an active RTP stream. My question now is, what to to
>>>> with inactive time slots? As long as all active streams are located
>>>> at consecutive time slots from the start of the 20ms interval,
>>>> everything is fine. But I cannot guarantee this.
>>>> What I need is some kind of dummy buffer, which is not transmitted
>>>> but generates a tx timing gap as a buffer of X bytes would take to be
>>>> transferred.
>>>> Is such a functionality provided? As a workaround, I already thought
>>>> about sending invalid packets (bad IP Header checksum?). However,
>>>> this won't be optimal when multiple lines are aggregated.
>>>> Thanks!
>>>> Philipp Beyer

Chair of Network Architectures and Services
Department of Informatics
Technical University of Munich
Boltzmannstr. 3
85748 Garching bei München, Germany 

More information about the users mailing list