Query regarding netvsc PMD : VF removal

[Nandini Rangaswamy]

Hi Long Li,
Thanks for your feedback . I updated the diagram.
I am still not clear about the different kernel module interactions in the
packet path.
The VFs appear as PCI devices in the guest kernel and are managed by MLNX
drivers.
The synthetic interfaces are being managed by the uio_hv_generic driver in
the kernel.
The netvsc PMD and mlnx PMD in dpdk app are instrumental in sending packets
from app to NIC bypassing the kernel.

Questions about packet path:
1. Is the MLNX driver in the kernel used when packets are sent from NIC to
the kernel without VF ?
2. Do the MLNX driver in kernel and uio_hv_genric driver interact?
3. Do the netvsc and mlnx PMDs interact  with uio_hv_generic driver?

Regards,
Nandini

On Fri, Jul 26, 2024 at 6:14 PM Long Li <longli at microsoft.com> wrote:

> The VF data path is separate from the synthetic data path. The chart
> doesn’t show clearly that all the VF packets go directly to DPDK.
>
>
>
> From the physical NIC, the packet data can either go through VF, or
> through netvsc, but not at the same time.
>
>
>
> In practice, the synthetic path carries small number of packets (<1%). The
> majority of packets go through the VF path.
>
>
>
> *From:* Nandini Rangaswamy <nandini.rangaswamy at broadcom.com>
> *Sent:* Friday, July 26, 2024 3:46 PM
> *To:* Stephen Hemminger <stephen at networkplumber.org>
> *Cc:* users at dpdk.org; Long Li <longli at microsoft.com>
> *Subject:* Re: Query regarding netvsc PMD : VF removal
>
>
>
> Hi Stephen and Long,
>
> I am trying to understand the complete packet path and I am confused
> here.  The synthetic device is being created and managed by a driver in
> guest kernel space.
>
> Shouldn't the packets be directly DMAed from NIC to VFs in guest. If all
> packets go through synthetic interface to the VF, then wouldn't kernel get
> involved here and defeat the purpose of DPDK ?
>
> Please find attached a block diagram that i came up with highlights the
> devices and drivers and my understanding of their interactions.
>
> I have shown in the diagram that for every VNIC , a synthetic and VF is
> present . Synthetic interfaces are managed by uio_hv_generic and VFs that
> appear as PCI devices in the guest kernel are managed by MLNX.
>
> I have also shown that netvsc pmd depends on uio_hv_generic module. The
> red arrow represents synthetic data path and green arrows represent the
> accelerated data path.
>
> I am still unclear about pkt flow from NIC to VF through synthetic
> interface and vice-versa involving netvsc pmd, mlnx driver/PMD.
>
> Can you please clarify this?
>
> Regards,
>
> Nandini
>
>
>
>
>
> On Thu, Jul 25, 2024 at 2:52 PM Stephen Hemminger <
> stephen at networkplumber.org> wrote:
>
> On Thu, 25 Jul 2024 11:41:59 -0700
> Nandini Rangaswamy <nandini.rangaswamy at broadcom.com> wrote:
>
> > Hi Stephen,
> > I am using the DPDK 22.11.5 LTS. Should the DPDK code take care of
> > unbinding synthetic interface from uio_hv_generic or should it be done by
> > the app when it gets hotplug notification?
> > Regards,
> > Nandini
>
> The synthetic device (netvsc) controlled by uio_hv_generic is never hot
> plugged.
> Only the VF which is mlx device can be removed and added.
> The management of the VF for traffic is handled completely internally to
> the netvsc device (same as the Linux netdevice). What happens is the
> host notifys the guest over vmbus when VF changes. For devices handled
> by the DPDK (netvsc PMD), this notification causes it to add/remove use
> of the VF.
>
> The VF should not be used directly. The synthetic device (netvsc) routes
> traffic over the VF and reads from it when it is present and up.
>
>
> This electronic communication and the information and any files
> transmitted with it, or attached to it, are confidential and are intended
> solely for the use of the individual or entity to whom it is addressed and
> may contain information that is confidential, legally privileged, protected
> by privacy laws, or otherwise restricted from disclosure to anyone else. If
> you are not the intended recipient or the person responsible for delivering
> the e-mail to the intended recipient, you are hereby notified that any use,
> copying, distributing, dissemination, forwarding, printing, or copying of
> this e-mail is strictly prohibited. If you received this e-mail in error,
> please return the e-mail to the sender, delete it from your computer, and
> destroy any printed copy of it.
>

-- 
This electronic communication and the information and any files transmitted 
with it, or attached to it, are confidential and are intended solely for 
the use of the individual or entity to whom it is addressed and may contain 
information that is confidential, legally privileged, protected by privacy 
laws, or otherwise restricted from disclosure to anyone else. If you are 
not the intended recipient or the person responsible for delivering the 
e-mail to the intended recipient, you are hereby notified that any use, 
copying, distributing, dissemination, forwarding, printing, or copying of 
this e-mail is strictly prohibited. If you received this e-mail in error, 
please return the e-mail to the sender, delete it from your computer, and 
destroy any printed copy of it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mails.dpdk.org/archives/users/attachments/20240805/67d820e4/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screenshot 2024-08-05 at 2.49.17?PM.png
Type: image/png
Size: 176103 bytes
Desc: not available
URL: <http://mails.dpdk.org/archives/users/attachments/20240805/67d820e4/attachment-0001.png>