<div dir="ltr">Hi<br><br>I have narrowed down the problem of sending packets via the AF_PACKET/SOCK_RAW interface, which might be what DPDK does in coressonding PMD drivers. The problem does not seem to be related to DPDK codebase directly, but still can be interesting for the users who are trying to use DPDK with AF_PACKET driver inside a dockerized environment.<br><br>To reiterate the issue: Any UDP message sent via AF_PACKET/SOCK_RAW socket on one interface on virtual bridge, is not received by application listening on the other interface attached to the same bridge, however strangely enough it can be seen in tcpdump listening on the receiving interface. The application I used based on the example here [<a href="https://www.kernel.org/doc/html/latest/_sources/networking/packet_mmap.rst.txt">https://www.kernel.org/doc/html/latest/_sources/networking/packet_mmap.rst.txt</a>] but instead of receiving I added a loop for sending 5 pre-recorded L2 frames.<br><br>I am attaching the small project to easily reproduce the issue (assuming the MAC addresses are preserved between machines by the docker because the L2 packet is pre-recorded) .<br>------<br>The environment is two dockers (basic AlmaOS 8.5 having only network utils installed) attached to the same bridge. The environment is set up with docker-compose.<br><br>To reproduce the issue:<br><br>- make dockers and applications using Makefile and start the dockers.<br><b>make<br> docker-compose up<br></b><br>- in other window set up server in first docker:<br><b> docker exec -it client /bin/bash<br> nc -l -4 -u 192.168.2.1 5997<br></b><br>- in other window set up tcpdump to see the incoming packets on 5997 port:<br><b> docker exec -it client /bin/bash<br> tcpdump -vneA -i eth0 port 5997<br></b><br>- in other window send 5 messages via application then via netcat: <br><b> docker exec -it server /bin/bash<br> /share/a.out eth0 hash<br> printf "ok1\r\n" | nc -s 192.168.2.5 -p 5998 -u -4 192.168.2.1 5997<br></b><br>Observe that only packets sent by nc are received. Though there's no difference in L2 frame structure apart from identification and checksum (wrong for both cases).<br> <br>The question is how can I debug the kernel to understand what makes it drop the packet and not to pass it to the application (netcat in this case). <br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Nov 9, 2022 at 10:50 AM J.J. Mars <<a href="mailto:mars14850@gmail.com">mars14850@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>If the nic on docker 1 of echo client can receive replies, the reason should not be PMD I think.</div><div>Did you make sure every field in every layer is completely the same between eth0 in docker 1 and eth0 in docker 2, and all of them are expected?<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Boris Ouretskey <<a href="mailto:borisusun@gmail.com" target="_blank">borisusun@gmail.com</a>> 于2022年11月9日周三 03:51写道：<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><table style="border-spacing:0px;line-height:20px;font-family:"Google Sans",Roboto,RobotoDraft,Helvetica,Arial,sans-serif;font-size:14px" cellpadding="0"><tbody><tr><td colspan="2" style="vertical-align:top;width:auto;padding:2px 0px"><span style="vertical-align:top"><br><br></span></td></tr></tbody></table><font face="monospace">Hi<br><br>I am trying to run a docker based system in one VM, with some servers using the DPDK and facing a very strange issue.<br><br>The relevant part of the system is the echo server which is built with DPDK library and runs in docker, connected to the virtual bridge (with AF_PACKET driver), and echo client built with regular sockets (java application). See the drawing below for visual details.<br><br>The strange thing is that echo replies sent from DPDK echo server docker (Docker2), can be seen in tcpdump in the docker of echo client (Docker1) however the application (java client) does not receive it on socket level. When the echo server runs as a socket application everything works. Seams like OS (linux) decide not to route the packet to the application for some reason. <br><br>Some quick important notes.<br><br>- DPDK layer starts ok and no EAL errors are reported. Its echo replies as mentioned before can clearly be seen when capturing traffic on eth0 in client docker.</font><div><font face="monospace">- It is not the issue of parsing the echo replies on application level. I checked the strace logs and cannot see any traffic delivered to the application<br>- I have compared the captures from working socket based echo server and MACs seam to be ok, also the packets sent from DPDK server are formed well.<br>- No logs for martian packets when enabled.<br><br>I would like to focus the attention of dear DPDK experts on the following question: is it possible that DPDK AF_PACKET driver intercepts both packets destined to the virtual nic eth0 of Docker1 AND eth0 of Docker2, preventing it from being received by application?<br><br>DPDK Version 18<br><br>VIRTUAL BOX VM (Alma Linux)<br><br> Docker1 Docker2 <br>+---------------+ +------------+<br>| | Request | |<br>| | --------------> | DPDK |<br>| Socket Echo | | Echo Server<br>| Client | Reply | Server |<br>| | <------------- | | --vdev=eth_af_packet0,iface=eth0,qdisc_bypass=1 <br>| | | |<br>| tcpdump -i eth0 +------+-----+<br>+-------+-------+ eth0 |<br> | eth0 |<br> | No IP |<br> +------v--------------------------------------v-----+<br> | Virtual Docker Bridge |<br> +---------------------------------------------------+<br> <br></font><br>Thanks</div></div> </blockquote></div> </blockquote></div>