[dpdk-dev] [RFC] DFS (DPDK Filesystem)

Wiles, Keith keith.wiles at intel.com
Tue Oct 9 00:42:50 CEST 2018


Hi Everyone,

I would like to request comments on DFS as I presented at the DPDK summit in Ireland.
 
As we know DPDK can be difficult to manage at run time and as DPDK becomes more dynamic we need to address how to configure and monitor DPDK and DPDK-based applications. A possible solution is to use FUSE file system support already in the Linux kernel through a DPDK library to expose a directory structure for anyone to be able to monitor and control the app by reading/writing files.
 
Primarily DPDK runs on Linux based systems, where FUSE is well supported. FUSE was added to FreeBSD 10 which appears to be the same design as Linux. Windows does not support FUSE directly from MS, but a open source design has a FUSE based set of wrapper APIs to help port FUSE based file systems.
 
Today the only way to get information out of DPDK is to use log files or a command line within DPDK to display the information. The information is not easy for an external application to collect or monitor. Using DFS any application can read and write into this virtual file system (FUSE) and retrieve information from DPDK/Application via a raw ASCII text file and/or JSON formatted text.
 
Data read from DFS can be static or dynamic e.g. version/copyright text would be static, but port stats would be dynamic and generated when the user requests the information. Writing new configuration data into DPDK can be a simple number or text or a JSON data structure. Writing simple ASCII text data is simple, but writing a long JSON or configuration file may some care, i.e. write a path to a file in DFS to the location of the configuration and use that file path for the configuration.
 
DPDK command line is getting to be very complex and long, but we can eliminate or greatly reduce the command line by using DFS to configure DPDK at startup along with the application configuration. Having DPDK start up with minimum resources and allow updating/creating a file in the DPDK file system to configure DPDK would be easier.
 
The DFS is a FUSE filesystem anchored at /dpdk in the host file system, then a directory is created under /dpdk for each DPDK instance or application using <name>-<PID> as a directory name format. Then it is up to DPDK/Application to create the directory structures, permissions and files within the directory mount point.
 
Currently I have a directory and files layout for DPDK, that needs more attention as I was focusing on FUSE then the layout as it can be changed fairly easily. I can email the working documentation for DFS if needed please send me a short email or I can post to the dev list. The docs also try to explain the design and APIs used by the developer for creating files/directories. The docs are still a work in progress and more attention needs to be done as we progress.
 
DFS is just a library and not required to be used plus it does not modify DPDK APIs or structures except for adding a few new dump routines for features that do not have one today. The file system or FUSE backend in DPDK is a free floating thread at this time, but could be affinitized to a core if required.
 
The FUSE protocol to the backend code (in DPDK) has a large number of operations, but we have simplified the interface for developers to 5-6. The interface on top of libfuse3 is very simple and currently provides most of the needed APIs for managing the files and directories. The FUSE file system with the added simpler API is dynamic and can be changed at runtime. The directory/files can be constructed via APIs at startup or anytime after startup.
 
I would like to put a repo someplace with DPDK/DFS say on GitHub to allow others to play with the design in the short term. I also have a doc file for DFS if anyone wants to learn more information and will be in the repo as well. If we agree I can then create a set of patches for DPDK.

Regards,
Keith



More information about the dev mailing list