[dpdk-dev] [PATCH v5 0/5] vhost: support selective datapath

[Maxime Coquelin]

On 04/02/2018 01:46 PM, Zhihong Wang wrote:
> This patch set introduces support for selective datapath in DPDK vhost-user
> lib. vDPA stands for vhost Data Path Acceleration. The idea is to support
> virtio ring compatible devices to serve virtio driver directly to enable
> datapath acceleration.
> 
> Design details
> ====
> 
> A vDPA device is a virtio ring compatible backend. It's defined as follows:
> 
> struct rte_vdpa_dev_addr {
> 	enum vdpa_addr_type type;
> 	union {
> 		uint8_t __dummy[64];
> 		struct rte_pci_addr pci_addr;
> 	};
> };
> 
> struct rte_vdpa_dev_ops {
> 	/* Get capabilities of this device */
> 	int (*get_queue_num)(int did, uint32_t *queue_num);
> 	int (*get_features)(int did, uint64_t *features);
> 	int (*get_protocol_features)(int did, uint64_t *protocol_features);
> 
> 	/* Driver configure/close the device */
> 	int (*dev_conf)(int vid);
> 	int (*dev_close)(int vid);
> 
> 	/* Enable/disable this vring */
> 	int (*set_vring_state)(int vid, int vring, int state);
> 
> 	/* Set features when changed */
> 	int (*set_features)(int vid);
> 
> 	/* Destination operations when migration done */
> 	int (*migration_done)(int vid);
> 
> 	/* Get the vfio group fd */
> 	int (*get_vfio_group_fd)(int vid);
> 
> 	/* Get the vfio device fd */
> 	int (*get_vfio_device_fd)(int vid);
> 
> 	/* Get the notify area info of the queue */
> 	int (*get_notify_area)(int vid, int qid,
> 			uint64_t *offset, uint64_t *size);
> 
> 	/* Reserved for future extension */
> 	void *reserved[5];
> };
> 
> struct rte_vdpa_device {
> 	struct rte_vdpa_dev_addr addr;
> 	struct rte_vdpa_dev_ops *ops;
> } __rte_cache_aligned;
> 
> 
> Changes to the current vhost-user lib are:
> ====
> 
>   1. Make vhost device capabilities configurable to adopt various devices.
>      Such capabilities include supported features, protocol features, queue
>      number. APIs are introduced to configure these capabilities.
> 
>   2. A set of device ops is defined for device specific operations:
> 
>       a. get_queue_num: Called to get supported queue number of the device.
> 
>       b. get_features: Called to get supported features of the device.
> 
>       c. get_protocol_features: Called to get supported protocol features of
>          the device.
> 
>       d. dev_conf: Called to configure the actual device when the virtio
>          device becomes ready.
> 
>       e. dev_close: Called to close the actual device when the virtio device
>          is stopped.
> 
>       f. set_vring_state: Called to change the state of the vring in the
>          actual device when vring state changes.
> 
>       g. set_features: Called to set the negotiated features to device.
> 
>       h. migration_done: Called to allow the device to response to RARP
>          sending.
> 
>       i. get_vfio_group_fd: Called to get the VFIO group fd of the device.
> 
>       j. get_vfio_device_fd: Called to get the VFIO device fd of the device.
> 
>       k. get_notify_area: Called to get the notify area info of the queue.
> 
>   3. To make vhost aware of its datapath type, an device id (did) is added
>      into the vhost data structure to identify the actual device. APIs are
>      introduced to configure it. When the default software datapath is used,
>      did is set to -1. When alternative datapath is used, did should be set
>      to specify which device to use, and each vhost-user socket can have
>      only 1 connection in this case.
> 
> Working process:
> ====
> 
>   1. Register vDPA device with address and device ops, and get a did.
> 
>   2. For vhost device creation:
> 
>        a. Register vhost-user socket.
> 
>        b. Set did of the vhost-user socket.
> 
>        c. Register vhost-user callbacks.
> 
>        d. Start to wait for connection.
> 
>   3. When connection comes and virtio device data structure is negotiated,
>      the device will be configured with all needed info.
> 
> ---
> Changes in v5:
> 
>   1. Rename the vDPA device ops to follow convention.
> 
>   2. Rename APIs for better description.
> 
>   3. Improve sanity check.
> 
> ---
> Changes in v4:
> 
>   1. Remove the "engine" concept in the lib.
> 
> ---
> Changes in v3:
> 
>   1. Keep macro names the same as in the spec.
> 
>   2. Export new APIs where they're introduced.
> 
> ---
> Changes in v2:
> 
>   1. Ensure negotiated capabilities are supported in vhost-user lib.
> 
>   2. Add APIs for live migration.
> 
>   3. Configure the data path at the right time.
> 
>   4. Add VFIO related vDPA device ops.
> 
>   5. Rebase on dpdk-next-virtio.
> 
> Zhihong Wang (5):
>    vhost: export vhost feature definitions
>    vhost: support selective datapath
>    vhost: add apis for datapath configuration
>    vhost: adapt vhost lib for selective datapath
>    vhost: add apis for live migration
> 
>   lib/librte_vhost/Makefile              |   4 +-
>   lib/librte_vhost/rte_vdpa.h            |  87 +++++++++++++++++
>   lib/librte_vhost/rte_vhost.h           | 154 ++++++++++++++++++++++++++++++
>   lib/librte_vhost/rte_vhost_version.map |  16 ++++
>   lib/librte_vhost/socket.c              | 166 ++++++++++++++++++++++++++++++++-
>   lib/librte_vhost/vdpa.c                | 115 +++++++++++++++++++++++
>   lib/librte_vhost/vhost.c               | 105 +++++++++++++++++++++
>   lib/librte_vhost/vhost.h               |  14 ++-
>   lib/librte_vhost/vhost_user.c          |  70 +++++++++++++-
>   lib/librte_vhost/vhost_user.h          |   7 --
>   10 files changed, 718 insertions(+), 20 deletions(-)
>   create mode 100644 lib/librte_vhost/rte_vdpa.h
>   create mode 100644 lib/librte_vhost/vdpa.c
> 


Series applied to dpdk-next-virtio/master.

Thanks!
Maxime