[dpdk-users] How to send an mbuf packet created on secondary process without memory corruption

[Ahmet Genç]

Hello everyone,

We are trying to run a multiprocess application over DPDK, but we are getting a memory corruption error.

On secondary DPDK application, we allocate an mbuf packet as below;


....

struct rte_mbuf *created_pkt = rte_pktmbuf_alloc(mbuf_pool);
// error check

pkt_size = sizeof(struct rte_ether_hdr) + sizeof(struct rte_arp_hdr);
created_pkt->data_len = pkt_size;
created_pkt->pkt_len = pkt_size;

eth_hdr = rte_pktmbuf_mtod(created_pkt, struct rte_ether_hdr *);
rte_ether_addr_copy(&bond_mac_addr, &eth_hdr->s_addr);
memset(&eth_hdr->d_addr, 0xFF, RTE_ETHER_ADDR_LEN);
eth_hdr->ether_type = rte_cpu_to_be_16(RTE_ETHER_TYPE_ARP);

arp_hdr = (struct rte_arp_hdr *)(
   (char *)eth_hdr + sizeof(struct rte_ether_hdr));
arp_hdr->arp_hardware = rte_cpu_to_be_16(RTE_ARP_HRD_ETHER);
arp_hdr->arp_protocol = rte_cpu_to_be_16(RTE_ETHER_TYPE_IPV4);
arp_hdr->arp_hlen = RTE_ETHER_ADDR_LEN;
arp_hdr->arp_plen = sizeof(uint32_t);
arp_hdr->arp_opcode = rte_cpu_to_be_16(RTE_ARP_OP_REQUEST);

rte_ether_addr_copy(&bond_mac_addr, &arp_hdr->arp_data.arp_sha);
arp_hdr->arp_data.arp_sip = bond_ip;
memset(&arp_hdr->arp_data.arp_tha, 0, RTE_ETHER_ADDR_LEN);
arp_hdr->arp_data.arp_tip =
        ((unsigned char *)&res->ip.addr.ipv4)[0]        |
       (((unsigned char *)&res->ip.addr.ipv4)[1] << 8)  |
       (((unsigned char *)&res->ip.addr.ipv4)[2] << 16) |
       (((unsigned char *)&res->ip.addr.ipv4)[3] << 24);

rte_ring_enqueue_burst(ring, (void **)&created_pkt, 1, NULL);


The packet (created_pkt) is created and allocated successfully (packet_len is not null and we don't see error line.). After that, we send out this packet to the primary application over the ring. On the primary application, we send the packet to a physical port using rte_eth_tx_burst.
However, we think that whenever the ring size reaches full (RING_SIZE 256) secondary application tries to release the memory and we get memory corruption error.

We think we are missing a point about the virtual and physical memory addresses between processes (setting mbuf->buf_addr or mbuf->buf_iova addresses or something like that). Could anyone help us to resolve this issue?

Thanks