[dpdk-users] dpdk and bulk data (video/audio)
sofia.baran at gmx.net
Thu Sep 13 11:54:14 CEST 2018
Fiona and Keith, thanks for your help.
I've tried to set buf_addr/buf_iova of a mbuf on my own, but I guess
that I miss something because transferring such mbufs seems not to work.
Here are some more details:
I need to transfer video data with an UDP/RTP based protocol over
ethernet. First I create a rte_mempool holding the video frames (e.g.
5MB per frame). Then I create a rte_mempool for mbufs holding the
UDP/RTP header and another rte_mempool holding mbufs with no data to
store the addresses of the video frame segments:
struct rte_mempool* frame_pool = rte_mempool_create("FramePool", 32,
struct rte_mempool* hdr_pool = rte_pktmbuf_pool_create("HeaderPool",
1024 - 1, MBUF_CACHE_SIZE, 0, 256, SOCKET_ID_ANY);
struct rte_mempool* pay_pool = rte_pktmbuf_pool_create("PayloadPool",
1024 - 1, MBUF_CACHE_SIZE, 0, 0, SOCKET_ID_ANY);
Then I prepare the header mbuf (RTP header here not used):
struct rte_mbuf* mhdr = rte_pktmbuf_alloc(hdr_pool);
struct ether_hdr* eth_hdr = rte_pktmbuf_mtod(mhdr, struct ether_hdr *);
struct ipv4_hdr* ip_hdr = (struct ipv4_hdr*)(eth_hdr + 1);
struct udp_hdr* udp_hdr = (struct udp_hdr*)(ip_hdr + 1);
int payloadSize = 1024;
mhdr->nb_segs = 2;
mhdr->data_len = sizeof(struct ether_hdr) + sizeof(struct ipv4_hdr) +
mhdr->pkt_len = m->data_len + payloadSize;
// don't know if the following lines is really that important
mhdr->ol_flags = 0;
mhdr->ol_flags |= PKT_TX_IPV4;
mhdr->ol_flags |= PKT_TX_IP_CKSUM;
mhdr->vlan_tci = 0;
mhdr->vlan_tci_outer = 0;
mhdr->l2_len = sizeof(struct ether_hdr);
mhdr->l3_len = sizeof(struct ipv4_hdr);
mhdr->l4_len = 0;
Then I prepared the payload mbuf and linked it to the header mbuf:
struct rte_mbuf* mpay = rte_pktmbuf_alloc(pay_pool);
mhdr->next = mpay;
mpay->buf_addr = framebuf;
mpay->buf_iova = rte_mempool_virt2iova(framebuf);
mpay->nb_segs = 1;
mpay->data_len = payloadSize;
mpay->pkt_len = m_hdr->pkt_len;
Not really sure if I filled all required mbuf members correctly. At
least when trying to transfer the mbuf using rte_eth_tx_burst(), it
doesn't work (rte_eth_tx_burst() returns no error but no data is
transfered by the NIC).
Important remarks: when I create the payload mbufs with sizes != 0, and
don't touch the buf_addr/buf_iova members, then the transfer works! I'm
using the Mellanox mlx5 PMD.
Hope to get some hints whats wrong.
On 09/10/2018 11:46 PM, Trahe, Fiona wrote:
>> -----Original Message-----
>> From: users [mailto:users-bounces at dpdk.org] On Behalf Of Wiles, Keith
>> Sent: Monday, September 10, 2018 1:52 PM
>> To: Sofia Baran <sofia.baran at gmx.net>
>> Cc: users at dpdk.org
>> Subject: Re: [dpdk-users] dpdk and bulk data (video/audio)
>>> On Sep 10, 2018, at 6:28 AM, Sofia Baran <sofia.baran at gmx.net> wrote:
>>> Hi All,
>>> I want/try to us DPDK for transferring larger amount of data, e.g. video frames which usually are
>> stored within memory buffers with sizes of several MB (remark: by using huges pages, these buffers
>> could be physically contiguous).
>>> When looking at the DPDK documentation, library APIs and examples, I can't find a way/hint how to
>> transfer larger buffers using DPDK without copying the video buffer fragments to the payload sections
>> of the mbufs - which results in high CPU loads.
>>> Within the ip_fragmentation example indirect mbufs are used, pointing to the payload section of a
>> direct mbuf (holding the header). But in my understanding the maximum size of a mbuf payload is 65KB
>> It is true that mbufs only hold (64K - 1). The concept of mbufs is normally an ethernet packet and they
>> are limited to 64K.
>> You can create a small mbuf (128 bytes) then set offset/data in the mbuf to point to the video buffer
>> only if you can find the physical memory address for the data. The mbuf normally holds the physical
>> address of the mbuf->data not the attached buffer in this case. This of course means you have to
>> manage the mbuf internal structure members yourself and be very careful you do not rearrange the
>> mbuf members as that can cause a performance problem.
> But the 64k-1 limit still applies, unless I'm misunderstanding.
> A way to get around this is to use chained mbufs.
> So create lots of small mbufs, each 128bytes, holding no payload, just the mbuf struct.
> Chain them together with each buf_iova/buf_addr pointing to the next 64k-1 segment of the payload.
> You'll need ~17 mbufs per MB, is this an acceptable overhead?
> You can also consider using compressdev API to compress the data before transmitting.
> However will have the same problem - and can use the same solution - for passing the data to the PMD to be compressed/decompressed.
>>> I'm pretty new to DPDK so maybe I missed something. I hope that someone can provide me some hits
>> how to avoid copying the entire payload.
>>> Sofia Baran
More information about the users