[dpdk-dev] [PATCH v14 4/4] examples/ifpga: add example for ifpga APIs

Wei Huang wei.huang at intel.com
Mon Feb 22 04:15:47 CET 2021


This example application shows how to use ifpga APIs.
A set of OPAE APIs are implemented based on these ifpga APIs.
You can test each OPAE API by running corresponding command.
A guide is also added to show how to run the example.

Signed-off-by: Wei Huang <wei.huang at intel.com>
Acked-by: Tianfei Zhang <tianfei.zhang at intel.com>
Acked-by: Rosen Xu <rosen.xu at intel.com>
---
v2: fix coding style issue in commands.c
---
v3: add guide for running example
---
v4: fix compilation issue of ifpga.rst
---
v5: add ifpga.rst into sample_app_ug/index.rst
---
v6: implement OPAE APIs in example instead of in driver
---
v7: fix compilation issue of unused const variable 'kdrv'
---
 MAINTAINERS                        |    3 +
 doc/guides/sample_app_ug/ifpga.rst |  387 +++++++
 doc/guides/sample_app_ug/index.rst |    1 +
 examples/ifpga/Makefile            |   43 +
 examples/ifpga/commands.c          | 1294 ++++++++++++++++++++++
 examples/ifpga/commands.h          |   16 +
 examples/ifpga/main.c              |   38 +
 examples/ifpga/meson.build         |   20 +
 examples/ifpga/opae_api.c          | 1630 ++++++++++++++++++++++++++++
 examples/ifpga/opae_api.h          |  244 +++++
 examples/meson.build               |    2 +-
 11 files changed, 3677 insertions(+), 1 deletion(-)
 create mode 100644 doc/guides/sample_app_ug/ifpga.rst
 create mode 100644 examples/ifpga/Makefile
 create mode 100644 examples/ifpga/commands.c
 create mode 100644 examples/ifpga/commands.h
 create mode 100644 examples/ifpga/main.c
 create mode 100644 examples/ifpga/meson.build
 create mode 100644 examples/ifpga/opae_api.c
 create mode 100644 examples/ifpga/opae_api.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 1a12916f56..48c2ea44d4 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1236,9 +1236,12 @@ Rawdev Drivers
 Intel FPGA
 M: Rosen Xu <rosen.xu at intel.com>
 M: Tianfei zhang <tianfei.zhang at intel.com>
+M: Wei Huang <wei.huang at intel.com>
 T: git://dpdk.org/next/dpdk-next-net-intel
 F: drivers/raw/ifpga/
 F: doc/guides/rawdevs/ifpga.rst
+F: examples/ifpga/
+F: doc/guides/sample_app_ug/ifpga.rst
 
 IOAT Rawdev
 M: Bruce Richardson <bruce.richardson at intel.com>
diff --git a/doc/guides/sample_app_ug/ifpga.rst b/doc/guides/sample_app_ug/ifpga.rst
new file mode 100644
index 0000000000..bc05153418
--- /dev/null
+++ b/doc/guides/sample_app_ug/ifpga.rst
@@ -0,0 +1,387 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2020-2021 Intel Corporation.
+
+Intel FPGA Sample Application
+=============================
+
+The Intel FPGA sample application is an example of how to use rte_pmd_ifpga API
+to manage Intel FPGA.
+
+Overview
+--------
+
+The Intel FPGA sample application is a simple application that demonstrates
+the use of the APIs provided by ifpga driver in the DPDK.
+A set of OPAE APIs are implemented in this example with 'opae' prefix. These
+APIs use PCI address ('Domain:Bus:Dev.Func') as an unique ID of FPGA,
+application based on OPAE can manage specified FPGA easily with them.
+OPAE (Open Programmable Acceleration Engine) is a software framework for
+managing and accessing programmable accelerators (FPGAs). For more information
+about OPAE, please refer to https://opae.github.io/2.0.0/index.html .
+This application is a readline-like interface that can be used to manage
+Intel FPGA, in a Linux* application environment.
+
+Compiling the Application
+-------------------------
+
+To compile the sample application see :doc:`compiling`
+
+The application is located in the ``ifpga`` sub-directory.
+
+Running the Application
+-----------------------
+
+To run the application in linux environment, issue the following command:
+
+.. code-block:: console
+
+    $ ./<build_dir>/examples/dpdk-ifpga --proc-type=auto
+
+Refer to the *DPDK Getting Started Guide* for general information on running
+applications and the Environment Abstraction Layer (EAL) options.
+
+Commands Description
+--------------------
+
+The following sections provide some explanation of the commands.
+
+help command
+~~~~~~~~~~~~
+
+The application has on-line help for the commands that are available at runtime.
+
+.. code-block:: console
+
+   opae> help
+    get_api_version                        get OPAE API version
+    get_proc_type                          get DPDK process type
+    get_image_info <FILE>                  get information of image file
+    get_status <BDF>                       get current status & progress of FPGA
+    get_property <BDF> <0|1|2|4|8>         get property of FPGA
+    get_phy_info <BDF>                     get information of PHY
+    get_parent <BDF>                       get parent PCI device of FPGA
+    get_child <BDF>                        get child PCI device of FPGA
+    get_pf1 <BDF>                          get physical function 1 device of FPGA
+    set_log_level <0-4>                    set logging level
+    set_log_file <FILE>                    set logging file
+    set_status <BDF> <0-4> <0-100>         set current status & progress of FPGA
+    enumerate <VID> <DID>                  enumerate specified FPGA
+    bind <BDF> <DRIVER>                    bind FPGA with kernel driver
+    unbind <BDF>                           unbind FPGA from kernel driver
+    probe <BDF>                            probe FPGA with IFPGA driver
+    remove <BDF>                           remove FPGA from IFPGA driver
+    flash <BDF> <FILE>                     update flash of FPGA
+    pr <BDF> <PORT> <FILE>                 partial reconfigure FPGA
+    reboot <BDF> <fpga|bmc> <0-1>          reboot FPGA or MAX10
+    cancel <BDF>                           cancel flash update
+    pci_read <BDF> <0-1024>                read PCI configuration space
+    pci_write <BDF> <0-1024> <NUM>         write PCI configuration space
+    quit                                   exit DPDK application
+    help                                   show commands list
+
+get_api_version command
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Show OPAE API version which is same to the version of DPDK.
+
+.. code-block:: console
+
+   opae> get_api_version
+   21.2.0
+
+set_log_level command
+~~~~~~~~~~~~~~~~~~~~~
+
+Set logging level of OPAE API. Logging level is defined as below.
+0 - CRITICAL
+1 - ERROR
+2 - WARNING
+3 - INFORMATION
+4 - DEBUG
+
+.. code-block:: console
+
+   opae> set_log_level 4
+   OPAE-API: Current log level is DEBUG
+   Successful
+   opae> set_log_level 6
+   OPAE-API: Current log level is DEBUG
+   Failed
+
+set_log_file command
+~~~~~~~~~~~~~~~~~~~~
+
+Set logging file of OPAE API.
+
+.. code-block:: console
+
+   opae> set_log_file /tmp/ifpga.log
+   Successful
+
+get_proc_type command
+~~~~~~~~~~~~~~~~~~~~~
+
+Show the process type of DPDK. If you start multiple instances of the
+application, the process type of the first one is 'Primary', the others
+are 'Secondary'.
+
+.. code-block:: console
+
+   opae> get_proc_type
+   Primary
+
+get_image_info command
+~~~~~~~~~~~~~~~~~~~~~~
+
+Display information of FPGA image file.
+
+.. code-block:: console
+
+   opae> get_image_info /home/wei/a10.bin
+   Type:           FPGA_BBS
+   Action:         UPDATE
+   Total length:   58720256
+   Payload offset: 1024
+   Payload length: 58719232
+   opae> get_image_info /home/wei/data.bin
+   OPAE-ERR: Image '/home/wei/data.bin' can not be recognized
+   Invalid image file
+
+enumerate command
+~~~~~~~~~~~~~~~~~
+
+Display PCI address of FPGA with specified vendor ID and device ID. ID value can
+be set to 0xffff for arbitrary ID.
+
+.. code-block:: console
+
+   opae> enumerate 0x8086 0x0b30
+   0000:24:00.0
+
+get_property command
+~~~~~~~~~~~~~~~~~~~~
+
+Display property information of specified FPGA. Property type is defined as below.
+0 - All properties
+1 - PCI property
+2 - FME property
+4 - port property
+8 - BMC property
+PCI property is always available, other properties can only be displayed after
+ifpga driver is probed to the FPGA.
+
+.. code-block:: console
+
+   opae> get_property 24:00.0 0
+   PCI:
+    PCIe s:b:d.f     : 0000:24:00.0
+    kernel driver    : vfio-pci
+   FME:
+    platform         : Vista Creek
+    DCP version      : DCP 1.2
+    phase            : Beta
+    interface        : 2x2x25G
+    build version    : 0.0.2
+    ports num        : 1
+    boot page        : user
+    pr interface id  : a5d72a3c-c8b0-4939-912c-f715e5dc10ca
+   PORT0:
+    access type      : PF
+    accelerator id   : 8892c23e-2eed-4b44-8bb6-5c88606e07df
+   BMC:
+    MAX10 version    : D.2.0.5
+    NIOS FW version  : D.2.0.12
+
+get_phy_info command
+~~~~~~~~~~~~~~~~~~~~
+
+Display information and status of PHY connects to the specified FPGA.
+
+.. code-block:: console
+
+   opae> get_phy_info 24:00.0
+    retimers num     : 2
+    link speed       : 25G
+    link status      : 00
+
+get_parent command
+~~~~~~~~~~~~~~~~~~
+
+Display PCI address of upstream device connects to the specified FPGA.
+
+.. code-block:: console
+
+   opae> get_parent 24:00.0
+   0000:22:09.0
+
+get_child command
+~~~~~~~~~~~~~~~~~
+
+Display PCI address of downstream device connects to the specified FPGA.
+
+.. code-block:: console
+
+   opae> get_child 24:00.0
+   No child
+   opae> get_child 22:09.0
+   0000:24:00.0
+
+get_pf1 command
+~~~~~~~~~~~~~~~
+
+Display PCI address of PF1 (physical function 1) of specified FPGA.
+
+.. code-block:: console
+
+   opae> get_pf1 24:00.0
+   0000:26:00.0
+   0000:26:00.1
+
+get_status command
+~~~~~~~~~~~~~~~~~~
+
+Display current RSU status of specified FPGA.
+
+.. code-block:: console
+
+   opae> get_status 24:00.0
+   Status:   IDLE
+   Progress: 0%
+
+set_status command
+~~~~~~~~~~~~~~~~~~
+
+Set current RSU status of specified FPGA. This command is mainly used for debug
+purpose. Status value is defined as below.
+0 - IDLE
+1 - PREPARE
+2 - PROGRAM
+3 - COPY
+4 - REBOOT
+
+.. code-block:: console
+
+   opae> set_status 24:00.0 2 35
+   Successful
+   opae> get_status 24:00.0
+   Status:   PROGRAM
+   Progress: 35%
+
+unbind command
+~~~~~~~~~~~~~~
+
+Unbind kernel driver from specified FPGA.
+
+.. code-block:: console
+
+   opae> unbind 24:00.0
+   OPAE-ERR: 0000:24:00.0 is probed, remove it first
+   Failed
+   opae> remove 24:00.0
+   Successful
+   opae> unbind 24:00.0
+   Successful
+
+bind command
+~~~~~~~~~~~~
+
+Bind specified kernel driver to specified FPGA.
+
+.. code-block:: console
+
+   opae> bind 24:00.0 vfio-pci
+   Successful
+
+probe command
+~~~~~~~~~~~~~
+
+Probe specified FPGA with DPDK PMD driver.
+
+.. code-block:: console
+
+   opae> probe 24:00.0
+   Successful
+
+remove command
+~~~~~~~~~~~~~~
+
+Remove specified FPGA from DPDK PMD driver. It's a reverse operation to probe
+command.
+
+.. code-block:: console
+
+   opae> remove 24:00.0
+   Successful
+
+flash command
+~~~~~~~~~~~~~
+
+Update image in flash of specified FPGA.
+
+.. code-block:: console
+
+   opae> flash 24:00.0 /home/wei/a10.bin
+   Successful
+
+pr command
+~~~~~~~~~~
+
+Do partial reconfiguration of specified FPGA.
+
+.. code-block:: console
+
+   opae> pr 24:00.0 0 /home/wei/nlb0.gbs
+   Successful
+
+reboot command
+~~~~~~~~~~~~~~
+
+Reboot specified FPGA. Reboot type and page is defined as below.
+fpga - reboot FPGA only
+bmc - reboot whole card with FPGA
+0 - factory page
+1 - user page
+
+.. code-block:: console
+
+   opae> reboot 24:00.0 fpga 1
+   Successful
+
+cancel command
+~~~~~~~~~~~~~~
+
+Cancel flash programming of specified FPGA.
+
+.. code-block:: console
+
+   opae> cancel 24:00.0
+   Successful
+
+pci_read command
+~~~~~~~~~~~~~~~~
+
+Read PCI configuration space of specified FPGA.
+
+.. code-block:: console
+
+   opae> pci_read 24:00.0 0
+   0x0b308086
+
+pci_write command
+~~~~~~~~~~~~~~~~~
+
+Write PCI configuration space of specified FPGA.
+
+.. code-block:: console
+
+   opae> pci_write 24:00.0 4 0x100406
+   Successful
+
+quit command
+~~~~~~~~~~~~
+
+Exit this sample application.
+
+.. code-block:: console
+
+   opae> quit
diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst
index e8db83d3a7..beb94ff3e7 100644
--- a/doc/guides/sample_app_ug/index.rst
+++ b/doc/guides/sample_app_ug/index.rst
@@ -23,6 +23,7 @@ Sample Applications User Guides
     kernel_nic_interface
     keep_alive
     ioat
+    ifpga
     l2_forward_crypto
     l2_forward_job_stats
     l2_forward_real_virtual
diff --git a/examples/ifpga/Makefile b/examples/ifpga/Makefile
new file mode 100644
index 0000000000..7d76479a36
--- /dev/null
+++ b/examples/ifpga/Makefile
@@ -0,0 +1,43 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2020-2021 Intel Corporation
+
+# binary name
+APP = ifpga
+
+# all source are stored in SRCS-y
+SRCS-y := main.c commands.c opae_api.c
+
+# Build using pkg-config variables if possible
+ifneq ($(shell pkg-config --exists libdpdk && echo 0),0)
+$(error "no installation of DPDK found")
+endif
+
+all: shared
+.PHONY: shared static
+shared: build/$(APP)-shared
+	ln -sf $(APP)-shared build/$(APP)
+static: build/$(APP)-static
+	ln -sf $(APP)-static build/$(APP)
+
+PKGCONF ?= pkg-config
+
+PC_FILE := $(shell $(PKGCONF) --path libdpdk 2>/dev/null)
+CFLAGS += -O3 $(shell $(PKGCONF) --cflags libdpdk)
+LDFLAGS_SHARED = $(shell $(PKGCONF) --libs libdpdk) -lrte_bus_pci -lrte_raw_ifpga
+LDFLAGS_STATIC = $(shell $(PKGCONF) --static --libs libdpdk)
+
+CFLAGS += -DALLOW_EXPERIMENTAL_API
+
+build/$(APP)-shared: $(SRCS-y) Makefile $(PC_FILE) | build
+	$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED)
+
+build/$(APP)-static: $(SRCS-y) Makefile $(PC_FILE) | build
+	$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_STATIC)
+
+build:
+	@mkdir -p $@
+
+.PHONY: clean
+clean:
+	rm -f build/$(APP) build/$(APP)-static build/$(APP)-shared
+	test -d build && rmdir -p build || true
diff --git a/examples/ifpga/commands.c b/examples/ifpga/commands.c
new file mode 100644
index 0000000000..ee58382571
--- /dev/null
+++ b/examples/ifpga/commands.c
@@ -0,0 +1,1294 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2020-2021 Intel Corporation.
+ * All rights reserved.
+ */
+
+#include <stdio.h>
+#include <stdint.h>
+#include <string.h>
+#include <stdlib.h>
+#include <stdarg.h>
+#include <errno.h>
+
+#include <cmdline_rdline.h>
+#include <cmdline_parse.h>
+#include <cmdline_parse_num.h>
+#include <cmdline_parse_string.h>
+#include <cmdline.h>
+
+#include "commands.h"
+
+static int parse_pciaddr(const char *bdf, opae_pci_device *id)
+{
+	size_t len = 0;
+	unsigned int domain = 0;
+	unsigned int bus = 0;
+	unsigned int devid = 0;
+	unsigned int function = 0;
+
+	if (!bdf || !id)
+		return -EINVAL;
+
+	len = strlen(bdf);
+	if ((len < 5) || (len > 12))
+		return -EINVAL;
+
+	len = sscanf(bdf, "%x:%x:%x.%d", &domain, &bus, &devid, &function);
+	if (len == 4) {
+		snprintf(id->bdf, sizeof(id->bdf), "%04x:%02x:%02x.%d",
+			domain, bus, devid, function);
+	} else {
+		len = sscanf(bdf, "%x:%x.%d", &bus, &devid, &function);
+		if (len == 3) {
+			snprintf(id->bdf, sizeof(id->bdf), "%04x:%02x:%02x.%d",
+				0, bus, devid, function);
+		} else {
+			return -EINVAL;
+		}
+	}
+	return 0;
+}
+
+static void uuid_to_str(opae_uuid *id, uuid_str *str)
+{
+	uint8_t *b = NULL;
+	char *p = NULL;
+	int i, j;
+
+	if (!id || !str)
+		return;
+
+	b = &id->b[15];
+	p = str->s;
+	for (i = 0; i < 4; i++, b--, p += 2)
+		sprintf(p, "%02x", *b);
+	sprintf(p++, "-");
+	for (i = 0; i < 3; i++) {
+		for (j = 0; j < 2; j++, b--, p += 2)
+			sprintf(p, "%02x", *b);
+		sprintf(p++, "-");
+	}
+	for (i = 0; i < 6; i++, b--, p += 2)
+		sprintf(p, "%02x", *b);
+}
+
+/* *** GET API VERSION *** */
+struct cmd_version_result {
+	cmdline_fixed_string_t cmd;
+};
+
+static void cmd_version_parsed(__rte_unused void *parsed_result,
+	struct cmdline *cl, __rte_unused void *data)
+{
+	opae_api_version ver;
+	opae_get_api_version(&ver);
+	cmdline_printf(cl, "%d.%d.%d\n", ver.major, ver.minor, ver.micro);
+}
+
+cmdline_parse_token_string_t cmd_version_cmd =
+	TOKEN_STRING_INITIALIZER(struct cmd_version_result, cmd, "get_api_version");
+
+cmdline_parse_inst_t cmd_get_api_version = {
+	.f = cmd_version_parsed,
+	.data = NULL,
+	.help_str = "get OPAE API version",
+	.tokens = {
+		(void *)&cmd_version_cmd,
+		NULL,
+	},
+};
+
+/* *** GET PROC TYPE *** */
+struct cmd_proc_type_result {
+	cmdline_fixed_string_t cmd;
+};
+
+static void cmd_proc_type_parsed(__rte_unused void *parsed_result,
+	struct cmdline *cl, __rte_unused void *data)
+{
+	int type = opae_get_proc_type();
+
+	if (type == 0)
+		cmdline_printf(cl, "Primary\n");
+	else if (type == 1)
+		cmdline_printf(cl, "Secondary\n");
+	else
+		cmdline_printf(cl, "Unknown\n");
+}
+
+cmdline_parse_token_string_t cmd_proc_type_cmd =
+	TOKEN_STRING_INITIALIZER(struct cmd_proc_type_result, cmd, "get_proc_type");
+
+cmdline_parse_inst_t cmd_get_proc_type = {
+	.f = cmd_proc_type_parsed,
+	.data = NULL,
+	.help_str = "get DPDK process type",
+	.tokens = {
+		(void *)&cmd_proc_type_cmd,
+		NULL,
+	},
+};
+
+/* *** GET IMAGE INFO *** */
+struct cmd_image_info_result {
+	cmdline_fixed_string_t cmd;
+	cmdline_fixed_string_t path;
+};
+
+static void cmd_image_info_parsed(void *parsed_result,
+	struct cmdline *cl, __rte_unused void *data)
+{
+	struct cmd_image_info_result *res = parsed_result;
+	opae_img_info info;
+
+	if (opae_get_image_info(res->path, &info) == 0) {
+		cmdline_printf(cl, "%-16s", "Type:");
+		if (info.type == OPAE_IMG_TYPE_BBS)
+			cmdline_printf(cl, "FPGA_BBS\n");
+		else if (info.type == OPAE_IMG_TYPE_BMC)
+			cmdline_printf(cl, "BMC\n");
+		else if (info.type == OPAE_IMG_TYPE_GBS)
+			cmdline_printf(cl, "FGPA_GBS\n");
+		else
+			cmdline_printf(cl, "Unknown\n");
+		cmdline_printf(cl, "%-16s", "Action:");
+		if (info.subtype == OPAE_IMG_SUBTYPE_UPDATE)
+			cmdline_printf(cl, "UPDATE\n");
+		else if (info.subtype == OPAE_IMG_SUBTYPE_CANCELLATION)
+			cmdline_printf(cl, "CANCELLATION\n");
+		else if (info.subtype == OPAE_IMG_SUBTYPE_ROOT_KEY_HASH_256)
+			cmdline_printf(cl, "ROOT_HASH_256\n");
+		else if (info.subtype == OPAE_IMG_SUBTYPE_ROOT_KEY_HASH_384)
+			cmdline_printf(cl, "ROOT_HASH_384\n");
+		else
+			cmdline_printf(cl, "Unknown\n");
+		cmdline_printf(cl, "%-16s%u\n", "Total length:",
+			info.total_len);
+		cmdline_printf(cl, "%-16s%u\n", "Payload offset:",
+			info.payload_offset);
+		cmdline_printf(cl, "%-16s%u\n", "Payload length:",
+			info.payload_len);
+	} else {
+		cmdline_printf(cl, "Invalid image file\n");
+	}
+}
+
+cmdline_parse_token_string_t cmd_image_info_cmd =
+	TOKEN_STRING_INITIALIZER(struct cmd_image_info_result, cmd,
+		"get_image_info");
+cmdline_parse_token_string_t cmd_image_info_path =
+	TOKEN_STRING_INITIALIZER(struct cmd_image_info_result, path, NULL);
+
+cmdline_parse_inst_t cmd_get_image_info = {
+	.f = cmd_image_info_parsed,
+	.data = NULL,
+	.help_str = "get information of image file",
+	.tokens = {
+		(void *)&cmd_image_info_cmd,
+		(void *)&cmd_image_info_path,
+		NULL,
+	},
+};
+
+/* *** GET STATUS *** */
+struct cmd_get_status_result {
+	cmdline_fixed_string_t cmd;
+	cmdline_fixed_string_t bdf;
+};
+
+static void cmd_get_status_parsed(void *parsed_result,
+	struct cmdline *cl, __rte_unused void *data)
+{
+	struct cmd_get_status_result *res = parsed_result;
+	opae_pci_device id;
+	uint32_t stat, prog;
+
+	if (parse_pciaddr(res->bdf, &id) < 0) {
+		cmdline_printf(cl, "%s is invalid PCI address\n", res->bdf);
+		return;
+	}
+
+	if (opae_load_rsu_status(&id, &stat, &prog) == 0) {
+		cmdline_printf(cl, "%-10s", "Status:");
+		if (stat == 0)
+			cmdline_printf(cl, "IDLE\n");
+		else if (stat == 1)
+			cmdline_printf(cl, "PREPARE\n");
+		else if (stat == 2)
+			cmdline_printf(cl, "PROGRAM\n");
+		else if (stat == 3)
+			cmdline_printf(cl, "COPY\n");
+		else if (stat == 4)
+			cmdline_printf(cl, "REBOOT\n");
+		else
+			cmdline_printf(cl, "unknown\n");
+		cmdline_printf(cl, "%-10s%u%%\n", "Progress:", prog);
+	} else {
+		cmdline_printf(cl, "Failed\n");
+	}
+}
+
+cmdline_parse_token_string_t cmd_get_status_cmd =
+	TOKEN_STRING_INITIALIZER(struct cmd_get_status_result, cmd, "get_status");
+cmdline_parse_token_string_t cmd_get_status_bdf =
+	TOKEN_STRING_INITIALIZER(struct cmd_get_status_result, bdf, NULL);
+
+cmdline_parse_inst_t cmd_get_status = {
+	.f = cmd_get_status_parsed,
+	.data = NULL,
+	.help_str = "get current status & progress of FPGA",
+	.tokens = {
+		(void *)&cmd_get_status_cmd,
+		(void *)&cmd_get_status_bdf,
+		NULL,
+	},
+};
+
+/* *** GET PROPERTY *** */
+struct cmd_property_result {
+	cmdline_fixed_string_t cmd;
+	cmdline_fixed_string_t bdf;
+	int32_t type;
+};
+
+static void cmd_property_parsed(void *parsed_result,
+	struct cmdline *cl, __rte_unused void *data)
+{
+	struct cmd_property_result *res = parsed_result;
+	opae_pci_device id;
+	opae_fpga_property prop;
+	uuid_str str;
+	uint32_t port = 0;
+
+	switch (res->type) {
+	case 0:
+	case 1:
+	case 2:
+	case 4:
+	case 8:
+		break;
+	default:
+		cmdline_printf(cl, "%d is invalid type of property\n",
+			res->type);
+		return;
+	}
+	if (parse_pciaddr(res->bdf, &id) < 0) {
+		cmdline_printf(cl, "%s is invalid PCI address\n", res->bdf);
+		return;
+	}
+
+	if (opae_get_property(&id, &prop, res->type) == 0) {
+		if ((res->type == 0) || (res->type == 1)) {
+			cmdline_printf(cl, "%s:\n", "PCI");
+			cmdline_printf(cl, " %-16s : %s\n",
+				"PCIe s:b:d.f", prop.pci.pci_addr);
+			cmdline_printf(cl, " %-16s : %s\n",
+				"kernel driver", prop.pci.drv_name);
+		}
+		if ((res->type == 0) || (res->type == 2)) {
+			cmdline_printf(cl, "%s:\n", "FME");
+			cmdline_printf(cl, " %-16s : %s\n",
+				"platform", prop.fme.platform_name);
+			cmdline_printf(cl, " %-16s : %s\n",
+				"DCP version", prop.fme.dcp_version);
+			cmdline_printf(cl, " %-16s : %s\n",
+				"phase", prop.fme.release_name);
+			cmdline_printf(cl, " %-16s : %s\n",
+				"interface", prop.fme.interface_type);
+			cmdline_printf(cl, " %-16s : %s\n",
+				"build version", prop.fme.build_version);
+			cmdline_printf(cl, " %-16s : %u\n",
+				"ports num", prop.fme.num_ports);
+			cmdline_printf(cl, " %-16s : %s\n",
+				"boot page", prop.fme.boot_page ? "user" : "factory");
+			uuid_to_str(&prop.fme.pr_id, &str);
+			cmdline_printf(cl, " %-16s : %s\n", "pr interface id",
+				str.s);
+		}
+		if ((res->type == 0) || (res->type == 4)) {
+			for (port = 0; port < prop.fme.num_ports; port++) {
+				cmdline_printf(cl, "%s%d:\n", "PORT", port);
+				cmdline_printf(cl, " %-16s : %s\n",
+					"access type",
+					prop.port[port].type ? "VF" : "PF");
+				uuid_to_str(&prop.port[port].afu_id, &str);
+				cmdline_printf(cl, " %-16s : %s\n",
+					"accelerator id", str.s);
+			}
+		}
+		if ((res->type == 0) || (res->type == 8)) {
+			cmdline_printf(cl, "%s:\n", "BMC");
+			cmdline_printf(cl, " %-16s : %s\n",
+				"MAX10 version", prop.bmc.bmc_version);
+			cmdline_printf(cl, " %-16s : %s\n",
+				"NIOS FW version", prop.bmc.fw_version);
+		}
+	} else {
+		cmdline_printf(cl, "Failed\n");
+	}
+}
+
+cmdline_parse_token_string_t cmd_property_cmd =
+	TOKEN_STRING_INITIALIZER(struct cmd_property_result, cmd, "get_property");
+cmdline_parse_token_string_t cmd_property_bdf =
+	TOKEN_STRING_INITIALIZER(struct cmd_property_result, bdf, NULL);
+cmdline_parse_token_num_t cmd_property_type =
+	TOKEN_NUM_INITIALIZER(struct cmd_property_result, type, RTE_INT32);
+
+cmdline_parse_inst_t cmd_get_property = {
+	.f = cmd_property_parsed,
+	.data = NULL,
+	.help_str = "get property of FPGA",
+	.tokens = {
+		(void *)&cmd_property_cmd,
+		(void *)&cmd_property_bdf,
+		(void *)&cmd_property_type,
+		NULL,
+	},
+};
+
+/* *** GET PHY INFO *** */
+struct cmd_phy_info_result {
+	cmdline_fixed_string_t cmd;
+	cmdline_fixed_string_t bdf;
+};
+
+static void cmd_phy_info_parsed(void *parsed_result,
+	struct cmdline *cl, __rte_unused void *data)
+{
+	struct cmd_phy_info_result *res = parsed_result;
+	opae_pci_device id;
+	opae_phy_info info;
+
+	if (parse_pciaddr(res->bdf, &id) < 0) {
+		cmdline_printf(cl, "%s is invalid PCI address\n", res->bdf);
+		return;
+	}
+
+	if (opae_get_phy_info(&id, &info) == 0) {
+		cmdline_printf(cl, " %-16s : %u\n",
+			"retimers num", info.num_retimers);
+		cmdline_printf(cl, " %-16s : %uG\n",
+			"link speed", info.link_speed);
+		cmdline_printf(cl, " %-16s : %02xh\n",
+			"link status", info.link_status);
+	} else {
+		cmdline_printf(cl, "Failed\n");
+	}
+}
+
+cmdline_parse_token_string_t cmd_phy_info_cmd =
+	TOKEN_STRING_INITIALIZER(struct cmd_phy_info_result, cmd, "get_phy_info");
+cmdline_parse_token_string_t cmd_phy_info_bdf =
+	TOKEN_STRING_INITIALIZER(struct cmd_phy_info_result, bdf, NULL);
+
+cmdline_parse_inst_t cmd_phy_info = {
+	.f = cmd_phy_info_parsed,
+	.data = NULL,
+	.help_str = "get information of PHY",
+	.tokens = {
+		(void *)&cmd_phy_info_cmd,
+		(void *)&cmd_phy_info_bdf,
+		NULL,
+	},
+};
+
+/* *** GET PARENT *** */
+struct cmd_parent_result {
+	cmdline_fixed_string_t cmd;
+	cmdline_fixed_string_t bdf;
+};
+
+static void cmd_parent_parsed(void *parsed_result,
+	struct cmdline *cl, __rte_unused void *data)
+{
+	struct cmd_parent_result *res = parsed_result;
+	opae_pci_device id;
+	opae_pci_device parent;
+
+	if (parse_pciaddr(res->bdf, &id) < 0) {
+		cmdline_printf(cl, "%s is invalid PCI address\n", res->bdf);
+		return;
+	}
+
+	if (opae_get_parent(&id, &parent) > 0)
+		cmdline_printf(cl, "%s\n", parent.bdf);
+	else
+		cmdline_printf(cl, "Failed\n");
+}
+
+cmdline_parse_token_string_t cmd_parent_cmd =
+	TOKEN_STRING_INITIALIZER(struct cmd_parent_result, cmd, "get_parent");
+cmdline_parse_token_string_t cmd_parent_bdf =
+	TOKEN_STRING_INITIALIZER(struct cmd_parent_result, bdf, NULL);
+
+cmdline_parse_inst_t cmd_get_parent = {
+	.f = cmd_parent_parsed,
+	.data = NULL,
+	.help_str = "get parent PCI device of FPGA",
+	.tokens = {
+		(void *)&cmd_parent_cmd,
+		(void *)&cmd_parent_bdf,
+		NULL,
+	},
+};
+
+/* *** GET CHILD *** */
+struct cmd_child_result {
+	cmdline_fixed_string_t cmd;
+	cmdline_fixed_string_t bdf;
+};
+
+static void cmd_child_parsed(void *parsed_result,
+	struct cmdline *cl, __rte_unused void *data)
+{
+	struct cmd_child_result *res = parsed_result;
+	opae_pci_device id;
+	pcidev_id child;
+	int i, count = 0;
+
+	if (parse_pciaddr(res->bdf, &id) < 0) {
+		cmdline_printf(cl, "%s is invalid PCI address\n", res->bdf);
+		return;
+	}
+
+	count = opae_get_child(&id, NULL, 0);
+	if (count > 0) {
+		child = (pcidev_id)malloc(sizeof(opae_pci_device) * count);
+		if (child) {
+			opae_get_child(&id, child, count);
+			for (i = 0; i < count; i++)
+				cmdline_printf(cl, "%s\n", child[i].bdf);
+			free(child);
+		} else {
+			cmdline_printf(cl, "No memory\n");
+		}
+	} else if (count == 0) {
+		cmdline_printf(cl, "No child\n");
+	} else {
+		cmdline_printf(cl, "Failed\n");
+	}
+}
+
+cmdline_parse_token_string_t cmd_child_cmd =
+	TOKEN_STRING_INITIALIZER(struct cmd_child_result, cmd, "get_child");
+cmdline_parse_token_string_t cmd_child_bdf =
+	TOKEN_STRING_INITIALIZER(struct cmd_child_result, bdf, NULL);
+
+cmdline_parse_inst_t cmd_get_child = {
+	.f = cmd_child_parsed,
+	.data = NULL,
+	.help_str = "get child PCI device of FPGA",
+	.tokens = {
+		(void *)&cmd_child_cmd,
+		(void *)&cmd_child_bdf,
+		NULL,
+	},
+};
+
+/* *** GET PF1 *** */
+struct cmd_pf1_result {
+	cmdline_fixed_string_t cmd;
+	cmdline_fixed_string_t bdf;
+};
+
+static void cmd_pf1_parsed(void *parsed_result,
+	struct cmdline *cl, __rte_unused void *data)
+{
+	struct cmd_pf1_result *res = parsed_result;
+	opae_pci_device id;
+	pcidev_id peer;
+	int i, count = 0;
+
+	if (parse_pciaddr(res->bdf, &id) < 0) {
+		cmdline_printf(cl, "%s is invalid PCI address\n", res->bdf);
+		return;
+	}
+
+	count = opae_get_pf1(&id, NULL, 0);
+	if (count > 0) {
+		peer = (pcidev_id)malloc(sizeof(opae_pci_device) * count);
+		if (peer) {
+			opae_get_pf1(&id, peer, count);
+			for (i = 0; i < count; i++)
+				cmdline_printf(cl, "%s\n", peer[i].bdf);
+			free(peer);
+		} else {
+			cmdline_printf(cl, "No memory\n");
+		}
+	} else if (count == 0) {
+		cmdline_printf(cl, "No PF1\n");
+	} else {
+		cmdline_printf(cl, "Failed\n");
+	}
+}
+
+cmdline_parse_token_string_t cmd_pf1_cmd =
+	TOKEN_STRING_INITIALIZER(struct cmd_pf1_result, cmd, "get_pf1");
+cmdline_parse_token_string_t cmd_pf1_bdf =
+	TOKEN_STRING_INITIALIZER(struct cmd_pf1_result, bdf, NULL);
+
+cmdline_parse_inst_t cmd_get_pf1 = {
+	.f = cmd_pf1_parsed,
+	.data = NULL,
+	.help_str = "get physical function 1 device of FPGA",
+	.tokens = {
+		(void *)&cmd_pf1_cmd,
+		(void *)&cmd_pf1_bdf,
+		NULL,
+	},
+};
+
+/* *** SET LOG LEVEL *** */
+struct cmd_log_level_result {
+	cmdline_fixed_string_t cmd;
+	int32_t level;
+};
+
+static void cmd_log_level_parsed(void *parsed_result,
+	struct cmdline *cl, __rte_unused void *data)
+{
+	struct cmd_log_level_result *res = parsed_result;
+	if (opae_set_log_level(res->level) == res->level)
+		cmdline_printf(cl, "Successful\n");
+	else
+		cmdline_printf(cl, "Failed\n");
+}
+
+cmdline_parse_token_string_t cmd_log_level_cmd =
+	TOKEN_STRING_INITIALIZER(struct cmd_log_level_result, cmd, "set_log_level");
+cmdline_parse_token_num_t cmd_log_level_level =
+	TOKEN_NUM_INITIALIZER(struct cmd_log_level_result, level, RTE_INT32);
+
+cmdline_parse_inst_t cmd_set_log_level = {
+	.f = cmd_log_level_parsed,
+	.data = NULL,
+	.help_str = "set logging level",
+	.tokens = {
+		(void *)&cmd_log_level_cmd,
+		(void *)&cmd_log_level_level,
+		NULL,
+	},
+};
+
+/* *** SET LOG FILE *** */
+struct cmd_log_file_result {
+	cmdline_fixed_string_t cmd;
+	cmdline_fixed_string_t path;
+};
+
+static void cmd_log_file_parsed(void *parsed_result,
+	struct cmdline *cl, __rte_unused void *data)
+{
+	struct cmd_log_file_result *res = parsed_result;
+	if (opae_set_log_file(res->path, 1) == 0)
+		cmdline_printf(cl, "Successful\n");
+	else
+		cmdline_printf(cl, "Failed\n");
+}
+
+cmdline_parse_token_string_t cmd_log_file_cmd =
+	TOKEN_STRING_INITIALIZER(struct cmd_log_file_result, cmd, "set_log_file");
+cmdline_parse_token_string_t cmd_log_file_path =
+	TOKEN_STRING_INITIALIZER(struct cmd_log_file_result, path, NULL);
+
+cmdline_parse_inst_t cmd_set_log_file = {
+	.f = cmd_log_file_parsed,
+	.data = NULL,
+	.help_str = "set logging file",
+	.tokens = {
+		(void *)&cmd_log_file_cmd,
+		(void *)&cmd_log_file_path,
+		NULL,
+	},
+};
+
+/* *** SET STATUS *** */
+struct cmd_set_status_result {
+	cmdline_fixed_string_t cmd;
+	cmdline_fixed_string_t bdf;
+	uint32_t stat;
+	uint32_t prog;
+};
+
+static void cmd_set_status_parsed(void *parsed_result,
+	struct cmdline *cl, __rte_unused void *data)
+{
+	struct cmd_set_status_result *res = parsed_result;
+	opae_pci_device id;
+
+	if (parse_pciaddr(res->bdf, &id) < 0) {
+		cmdline_printf(cl, "%s is invalid PCI address\n", res->bdf);
+		return;
+	}
+	if ((res->stat > 4) || (res->prog > 100)) {
+		cmdline_printf(cl, "%u,%u is invalid status\n", res->stat,
+			res->prog);
+		return;
+	}
+
+	if (opae_store_rsu_status(&id, res->stat, res->prog) == 0)
+		cmdline_printf(cl, "Successful\n");
+	else
+		cmdline_printf(cl, "Failed\n");
+}
+
+cmdline_parse_token_string_t cmd_set_status_cmd =
+	TOKEN_STRING_INITIALIZER(struct cmd_set_status_result, cmd, "set_status");
+cmdline_parse_token_string_t cmd_set_status_bdf =
+	TOKEN_STRING_INITIALIZER(struct cmd_set_status_result, bdf, NULL);
+cmdline_parse_token_num_t cmd_set_status_stat =
+	TOKEN_NUM_INITIALIZER(struct cmd_set_status_result, stat, RTE_UINT32);
+cmdline_parse_token_num_t cmd_set_status_prog =
+	TOKEN_NUM_INITIALIZER(struct cmd_set_status_result, prog, RTE_UINT32);
+
+cmdline_parse_inst_t cmd_set_status = {
+	.f = cmd_set_status_parsed,
+	.data = NULL,
+	.help_str = "set current status & progress of FPGA",
+	.tokens = {
+		(void *)&cmd_set_status_cmd,
+		(void *)&cmd_set_status_bdf,
+		(void *)&cmd_set_status_stat,
+		(void *)&cmd_set_status_prog,
+		NULL,
+	},
+};
+
+/* *** ENUMERATE *** */
+struct cmd_enumerate_result {
+	cmdline_fixed_string_t cmd;
+	uint32_t vid;
+	uint32_t did;
+};
+
+static void cmd_enumerate_parsed(void *parsed_result,
+	struct cmdline *cl, __rte_unused void *data)
+{
+	struct cmd_enumerate_result *res = parsed_result;
+	opae_pci_id filter;
+	opae_pci_device *id;
+	int i, count = 0;
+
+	filter.vendor_id = res->vid;
+	filter.device_id = res->did;
+	filter.class_id = BIT_SET_32;
+	filter.subsystem_vendor_id = BIT_SET_16;
+	filter.subsystem_device_id = BIT_SET_16;
+
+	count = opae_enumerate(&filter, NULL, 0);
+	if (count > 0) {
+		id = (opae_pci_device *)malloc(sizeof(opae_pci_device) * count);
+		if (id) {
+			opae_enumerate(&filter, id, count);
+			for (i = 0; i < count; i++)
+				cmdline_printf(cl, "%s\n", id[i].bdf);
+			free(id);
+		} else {
+			cmdline_printf(cl, "No memory\n");
+		}
+	} else if (count == 0) {
+		cmdline_printf(cl, "Not found\n");
+	} else {
+		cmdline_printf(cl, "Failed\n");
+	}
+}
+
+cmdline_parse_token_string_t cmd_enumerate_cmd =
+	TOKEN_STRING_INITIALIZER(struct cmd_enumerate_result, cmd, "enumerate");
+cmdline_parse_token_num_t cmd_enumerate_vid =
+	TOKEN_NUM_INITIALIZER(struct cmd_enumerate_result, vid, RTE_UINT32);
+cmdline_parse_token_num_t cmd_enumerate_did =
+	TOKEN_NUM_INITIALIZER(struct cmd_enumerate_result, did, RTE_UINT32);
+
+cmdline_parse_inst_t cmd_enumerate = {
+	.f = cmd_enumerate_parsed,
+	.data = NULL,
+	.help_str = "enumerate specified FPGA",
+	.tokens = {
+		(void *)&cmd_enumerate_cmd,
+		(void *)&cmd_enumerate_vid,
+		(void *)&cmd_enumerate_did,
+		NULL,
+	},
+};
+
+/* *** BIND *** */
+struct cmd_bind_result {
+	cmdline_fixed_string_t cmd;
+	cmdline_fixed_string_t bdf;
+	cmdline_fixed_string_t drv;
+};
+
+static void cmd_bind_parsed(void *parsed_result,
+	struct cmdline *cl, __rte_unused void *data)
+{
+	struct cmd_bind_result *res = parsed_result;
+	opae_pci_device id;
+
+	if (parse_pciaddr(res->bdf, &id) < 0) {
+		cmdline_printf(cl, "%s is invalid PCI address\n", res->bdf);
+		return;
+	}
+
+	if (opae_bind_driver(&id, res->drv) == 0)
+		cmdline_printf(cl, "Successful\n");
+	else
+		cmdline_printf(cl, "Failed\n");
+}
+
+cmdline_parse_token_string_t cmd_bind_cmd =
+	TOKEN_STRING_INITIALIZER(struct cmd_bind_result, cmd, "bind");
+cmdline_parse_token_string_t cmd_bind_bdf =
+	TOKEN_STRING_INITIALIZER(struct cmd_bind_result, bdf, NULL);
+cmdline_parse_token_string_t cmd_bind_drv =
+	TOKEN_STRING_INITIALIZER(struct cmd_bind_result, drv, NULL);
+
+cmdline_parse_inst_t cmd_bind = {
+	.f = cmd_bind_parsed,
+	.data = NULL,
+	.help_str = "bind FPGA with kernel driver",
+	.tokens = {
+		(void *)&cmd_bind_cmd,
+		(void *)&cmd_bind_bdf,
+		(void *)&cmd_bind_drv,
+		NULL,
+	},
+};
+
+/* *** UNBIND *** */
+struct cmd_unbind_result {
+	cmdline_fixed_string_t cmd;
+	cmdline_fixed_string_t bdf;
+};
+
+static void cmd_unbind_parsed(void *parsed_result,
+	struct cmdline *cl, __rte_unused void *data)
+{
+	struct cmd_unbind_result *res = parsed_result;
+	opae_pci_device id;
+
+	if (parse_pciaddr(res->bdf, &id) < 0) {
+		cmdline_printf(cl, "%s is invalid PCI address\n", res->bdf);
+		return;
+	}
+
+	if (opae_unbind_driver(&id) == 0)
+		cmdline_printf(cl, "Successful\n");
+	else
+		cmdline_printf(cl, "Failed\n");
+}
+
+cmdline_parse_token_string_t cmd_unbind_cmd =
+	TOKEN_STRING_INITIALIZER(struct cmd_bind_result, cmd, "unbind");
+cmdline_parse_token_string_t cmd_unbind_bdf =
+	TOKEN_STRING_INITIALIZER(struct cmd_bind_result, bdf, NULL);
+
+cmdline_parse_inst_t cmd_unbind = {
+	.f = cmd_unbind_parsed,
+	.data = NULL,
+	.help_str = "unbind FPGA from kernel driver",
+	.tokens = {
+		(void *)&cmd_unbind_cmd,
+		(void *)&cmd_unbind_bdf,
+		NULL,
+	},
+};
+
+/* *** PROBE *** */
+struct cmd_probe_result {
+	cmdline_fixed_string_t cmd;
+	cmdline_fixed_string_t bdf;
+};
+
+static void cmd_probe_parsed(void *parsed_result,
+	struct cmdline *cl, __rte_unused void *data)
+{
+	struct cmd_probe_result *res = parsed_result;
+	opae_pci_device id;
+
+	if (parse_pciaddr(res->bdf, &id) < 0) {
+		cmdline_printf(cl, "%s is invalid PCI address\n", res->bdf);
+		return;
+	}
+
+	if (opae_probe_device(&id) == 0)
+		cmdline_printf(cl, "Successful\n");
+	else
+		cmdline_printf(cl, "Failed\n");
+}
+
+cmdline_parse_token_string_t cmd_probe_cmd =
+	TOKEN_STRING_INITIALIZER(struct cmd_probe_result, cmd, "probe");
+cmdline_parse_token_string_t cmd_probe_bdf =
+	TOKEN_STRING_INITIALIZER(struct cmd_probe_result, bdf, NULL);
+
+cmdline_parse_inst_t cmd_probe = {
+	.f = cmd_probe_parsed,
+	.data = NULL,
+	.help_str = "probe FPGA with IFPGA driver",
+	.tokens = {
+		(void *)&cmd_probe_cmd,
+		(void *)&cmd_probe_bdf,
+		NULL,
+	},
+};
+
+/* *** REMOVE *** */
+struct cmd_remove_result {
+	cmdline_fixed_string_t cmd;
+	cmdline_fixed_string_t bdf;
+};
+
+static void cmd_remove_parsed(void *parsed_result,
+	struct cmdline *cl, __rte_unused void *data)
+{
+	struct cmd_remove_result *res = parsed_result;
+	opae_pci_device id;
+
+	if (parse_pciaddr(res->bdf, &id) < 0) {
+		cmdline_printf(cl, "%s is invalid PCI address\n", res->bdf);
+		return;
+	}
+
+	if (opae_remove_device(&id) == 0)
+		cmdline_printf(cl, "Successful\n");
+	else
+		cmdline_printf(cl, "Failed\n");
+}
+
+cmdline_parse_token_string_t cmd_remove_cmd =
+	TOKEN_STRING_INITIALIZER(struct cmd_remove_result, cmd, "remove");
+cmdline_parse_token_string_t cmd_remove_bdf =
+	TOKEN_STRING_INITIALIZER(struct cmd_remove_result, bdf, NULL);
+
+cmdline_parse_inst_t cmd_remove = {
+	.f = cmd_remove_parsed,
+	.data = NULL,
+	.help_str = "remove FPGA from IFPGA driver",
+	.tokens = {
+		(void *)&cmd_remove_cmd,
+		(void *)&cmd_remove_bdf,
+		NULL,
+	},
+};
+
+/* *** FLASH *** */
+struct cmd_flash_result {
+	cmdline_fixed_string_t cmd;
+	cmdline_fixed_string_t bdf;
+	cmdline_fixed_string_t path;
+};
+
+static void cmd_flash_parsed(void *parsed_result,
+	struct cmdline *cl, __rte_unused void *data)
+{
+	struct cmd_flash_result *res = parsed_result;
+	opae_pci_device id;
+	uint64_t stat = 0;
+
+	if (parse_pciaddr(res->bdf, &id) < 0) {
+		cmdline_printf(cl, "%s is invalid PCI address\n", res->bdf);
+		return;
+	}
+
+	if (opae_update_flash(&id, res->path, &stat))
+		cmdline_printf(cl, "Error: 0x%lx\n", stat);
+}
+
+cmdline_parse_token_string_t cmd_flash_cmd =
+	TOKEN_STRING_INITIALIZER(struct cmd_flash_result, cmd, "flash");
+cmdline_parse_token_string_t cmd_flash_bdf =
+	TOKEN_STRING_INITIALIZER(struct cmd_flash_result, bdf, NULL);
+cmdline_parse_token_string_t cmd_flash_path =
+	TOKEN_STRING_INITIALIZER(struct cmd_flash_result, path, NULL);
+
+cmdline_parse_inst_t cmd_flash = {
+	.f = cmd_flash_parsed,
+	.data = NULL,
+	.help_str = "update flash of FPGA",
+	.tokens = {
+		(void *)&cmd_flash_cmd,
+		(void *)&cmd_flash_bdf,
+		(void *)&cmd_flash_path,
+		NULL,
+	},
+};
+
+/* *** PR *** */
+struct cmd_pr_result {
+	cmdline_fixed_string_t cmd;
+	cmdline_fixed_string_t bdf;
+	int32_t port;
+	cmdline_fixed_string_t path;
+};
+
+static void cmd_pr_parsed(void *parsed_result,
+	struct cmdline *cl, __rte_unused void *data)
+{
+	struct cmd_pr_result *res = parsed_result;
+	opae_pci_device id;
+
+	if (parse_pciaddr(res->bdf, &id) < 0) {
+		cmdline_printf(cl, "%s is invalid PCI address\n", res->bdf);
+		return;
+	}
+
+	if (opae_partial_reconfigure(&id, res->port, res->path) == 0)
+		cmdline_printf(cl, "Successful\n");
+	else
+		cmdline_printf(cl, "Failed\n");
+}
+
+cmdline_parse_token_string_t cmd_pr_cmd =
+	TOKEN_STRING_INITIALIZER(struct cmd_pr_result, cmd, "pr");
+cmdline_parse_token_string_t cmd_pr_bdf =
+	TOKEN_STRING_INITIALIZER(struct cmd_pr_result, bdf, NULL);
+cmdline_parse_token_num_t cmd_pr_port =
+	TOKEN_NUM_INITIALIZER(struct cmd_pr_result, port, RTE_INT32);
+cmdline_parse_token_string_t cmd_pr_path =
+	TOKEN_STRING_INITIALIZER(struct cmd_pr_result, path, NULL);
+
+cmdline_parse_inst_t cmd_pr = {
+	.f = cmd_pr_parsed,
+	.data = NULL,
+	.help_str = "partial reconfigure FPGA",
+	.tokens = {
+		(void *)&cmd_pr_cmd,
+		(void *)&cmd_pr_bdf,
+		(void *)&cmd_pr_port,
+		(void *)&cmd_pr_path,
+		NULL,
+	},
+};
+
+/* *** REBOOT *** */
+struct cmd_reboot_result {
+	cmdline_fixed_string_t cmd;
+	cmdline_fixed_string_t bdf;
+	cmdline_fixed_string_t type;
+	int32_t page;
+};
+
+static void cmd_reboot_parsed(void *parsed_result,
+	struct cmdline *cl, __rte_unused void *data)
+{
+	struct cmd_reboot_result *res = parsed_result;
+	opae_pci_device id;
+	int type = 0;
+
+	if (parse_pciaddr(res->bdf, &id) < 0) {
+		cmdline_printf(cl, "%s is invalid PCI address\n", res->bdf);
+		return;
+	}
+
+	if (!strcmp(res->type, "fpga")) {
+		type = 0;
+	} else if (!strcmp(res->type, "bmc")) {
+		type = 1;
+	} else {
+		cmdline_printf(cl, "%s is invalid reboot type\n", res->type);
+		return;
+	}
+
+	if (opae_reboot_device(&id, type, res->page) == 0)
+		cmdline_printf(cl, "Successful\n");
+	else
+		cmdline_printf(cl, "Failed\n");
+}
+
+cmdline_parse_token_string_t cmd_reboot_cmd =
+	TOKEN_STRING_INITIALIZER(struct cmd_reboot_result, cmd, "reboot");
+cmdline_parse_token_string_t cmd_reboot_bdf =
+	TOKEN_STRING_INITIALIZER(struct cmd_reboot_result, bdf, NULL);
+cmdline_parse_token_string_t cmd_reboot_type =
+	TOKEN_STRING_INITIALIZER(struct cmd_reboot_result, type, NULL);
+cmdline_parse_token_num_t cmd_reboot_page =
+	TOKEN_NUM_INITIALIZER(struct cmd_reboot_result, page, RTE_INT32);
+
+cmdline_parse_inst_t cmd_reboot = {
+	.f = cmd_reboot_parsed,
+	.data = NULL,
+	.help_str = "reboot FPGA or MAX10",
+	.tokens = {
+		(void *)&cmd_reboot_cmd,
+		(void *)&cmd_reboot_bdf,
+		(void *)&cmd_reboot_type,
+		(void *)&cmd_reboot_page,
+		NULL,
+	},
+};
+
+/* *** CANCEL *** */
+struct cmd_cancel_result {
+	cmdline_fixed_string_t cmd;
+	cmdline_fixed_string_t bdf;
+};
+
+static void cmd_cancel_parsed(void *parsed_result,
+	struct cmdline *cl, __rte_unused void *data)
+{
+	struct cmd_cancel_result *res = parsed_result;
+	opae_pci_device id;
+
+	if (parse_pciaddr(res->bdf, &id) < 0) {
+		cmdline_printf(cl, "%s is invalid PCI address\n", res->bdf);
+		return;
+	}
+
+	if (opae_cancel_flash_update(&id, 0) == 0)
+		cmdline_printf(cl, "Successful\n");
+	else
+		cmdline_printf(cl, "Failed\n");
+}
+
+cmdline_parse_token_string_t cmd_cancel_cmd =
+	TOKEN_STRING_INITIALIZER(struct cmd_cancel_result, cmd, "cancel");
+cmdline_parse_token_string_t cmd_cancel_bdf =
+	TOKEN_STRING_INITIALIZER(struct cmd_cancel_result, bdf, NULL);
+
+cmdline_parse_inst_t cmd_cancel = {
+	.f = cmd_cancel_parsed,
+	.data = NULL,
+	.help_str = "cancel flash update",
+	.tokens = {
+		(void *)&cmd_cancel_cmd,
+		(void *)&cmd_cancel_bdf,
+		NULL,
+	},
+};
+
+/* *** PCI READ *** */
+struct cmd_pci_read_result {
+	cmdline_fixed_string_t cmd;
+	cmdline_fixed_string_t bdf;
+	uint32_t offset;
+};
+
+static void cmd_pci_read_parsed(void *parsed_result,
+	struct cmdline *cl, __rte_unused void *data)
+{
+	struct cmd_pci_read_result *res = parsed_result;
+	opae_pci_device id;
+	uint32_t offset = 0;
+	uint32_t value = 0;
+
+	if (parse_pciaddr(res->bdf, &id) < 0) {
+		cmdline_printf(cl, "%s is invalid PCI address\n", res->bdf);
+		return;
+	}
+
+	if (res->offset & 0x3) {
+		offset = res->offset & ~3;
+		cmdline_printf(cl, "align offset to 0x%x\n", offset);
+	} else {
+		offset = res->offset;
+	}
+
+	if (opae_read_pci_cfg(&id, offset, &value) == 0)
+		cmdline_printf(cl, "0x%08x\n", value);
+	else
+		cmdline_printf(cl, "Failed\n");
+}
+
+cmdline_parse_token_string_t cmd_pci_read_cmd =
+	TOKEN_STRING_INITIALIZER(struct cmd_pci_read_result, cmd, "pci_read");
+cmdline_parse_token_string_t cmd_pci_read_bdf =
+	TOKEN_STRING_INITIALIZER(struct cmd_pci_read_result, bdf, NULL);
+cmdline_parse_token_num_t cmd_pci_read_offset =
+	TOKEN_NUM_INITIALIZER(struct cmd_pci_read_result, offset, RTE_UINT32);
+
+cmdline_parse_inst_t cmd_pci_read = {
+	.f = cmd_pci_read_parsed,
+	.data = NULL,
+	.help_str = "read PCI configuration space",
+	.tokens = {
+		(void *)&cmd_pci_read_cmd,
+		(void *)&cmd_pci_read_bdf,
+		(void *)&cmd_pci_read_offset,
+		NULL,
+	},
+};
+
+/* *** PCI WRITE *** */
+struct cmd_pci_write_result {
+	cmdline_fixed_string_t cmd;
+	cmdline_fixed_string_t bdf;
+	uint32_t offset;
+	uint32_t value;
+};
+
+static void cmd_pci_write_parsed(void *parsed_result,
+	struct cmdline *cl, __rte_unused void *data)
+{
+	struct cmd_pci_write_result *res = parsed_result;
+	opae_pci_device id;
+	uint32_t offset = 0;
+
+	if (parse_pciaddr(res->bdf, &id) < 0) {
+		cmdline_printf(cl, "%s is invalid PCI address\n", res->bdf);
+		return;
+	}
+
+	if (res->offset & 0x3) {
+		offset = res->offset & ~3;
+		cmdline_printf(cl, "align offset to 0x%x\n", offset);
+	} else {
+		offset = res->offset;
+	}
+
+	if (opae_write_pci_cfg(&id, offset, res->value) == 0)
+		cmdline_printf(cl, "Successful\n");
+	else
+		cmdline_printf(cl, "Failed\n");
+}
+
+cmdline_parse_token_string_t cmd_pci_write_cmd =
+	TOKEN_STRING_INITIALIZER(struct cmd_pci_write_result, cmd, "pci_write");
+cmdline_parse_token_string_t cmd_pci_write_bdf =
+	TOKEN_STRING_INITIALIZER(struct cmd_pci_write_result, bdf, NULL);
+cmdline_parse_token_num_t cmd_pci_write_offset =
+	TOKEN_NUM_INITIALIZER(struct cmd_pci_write_result, offset, RTE_UINT32);
+cmdline_parse_token_num_t cmd_pci_write_value =
+	TOKEN_NUM_INITIALIZER(struct cmd_pci_write_result, value, RTE_UINT32);
+
+cmdline_parse_inst_t cmd_pci_write = {
+	.f = cmd_pci_write_parsed,
+	.data = NULL,
+	.help_str = "write PCI configuration space",
+	.tokens = {
+		(void *)&cmd_pci_write_cmd,
+		(void *)&cmd_pci_write_bdf,
+		(void *)&cmd_pci_write_offset,
+		(void *)&cmd_pci_write_value,
+		NULL,
+	},
+};
+
+/* *** QUIT *** */
+struct cmd_quit_result {
+	cmdline_fixed_string_t quit;
+};
+
+static void cmd_quit_parsed(__rte_unused void *parsed_result,
+	struct cmdline *cl, __rte_unused void *data)
+{
+	cmdline_quit(cl);
+}
+
+cmdline_parse_token_string_t cmd_quit_quit =
+	TOKEN_STRING_INITIALIZER(struct cmd_quit_result, quit, "quit");
+
+cmdline_parse_inst_t cmd_quit = {
+	.f = cmd_quit_parsed,
+	.data = NULL,
+	.help_str = "exit DPDK application",
+	.tokens = {
+		(void *)&cmd_quit_quit,
+		NULL,
+	},
+};
+
+/* *** HELP *** */
+struct cmd_help_result {
+	cmdline_fixed_string_t help;
+};
+
+static void cmd_help_parsed(__rte_unused void *parsed_result,
+	struct cmdline *cl, __rte_unused void *data)
+{
+	cmdline_printf(cl,
+		" get_api_version               \t\t"
+			"get OPAE API version\n"
+		" get_proc_type                 \t\t"
+			"get DPDK process type\n"
+		" get_image_info <FILE>         \t\t"
+			"get information of image file\n"
+		" get_status <BDF>              \t\t"
+			"get current status & progress of FPGA\n"
+		" get_property <BDF> <0|1|2|4|8>\t\t"
+			"get property of FPGA\n"
+		" get_phy_info <BDF>            \t\t"
+			"get information of PHY\n"
+		" get_parent <BDF>              \t\t"
+			"get parent PCI device of FPGA\n"
+		" get_child <BDF>               \t\t"
+			"get child PCI device of FPGA\n"
+		" get_pf1 <BDF>                 \t\t"
+			"get physical function 1 device of FPGA\n"
+		" set_log_level <0-4>           \t\t"
+			"set logging level\n"
+		" set_log_file <FILE>           \t\t"
+			"set logging file\n"
+		" set_status <BDF> <0-4> <0-100>\t\t"
+			"set current status & progress of FPGA\n"
+		" enumerate <VID> <DID>         \t\t"
+			"enumerate specified FPGA\n"
+		" bind <BDF> <DRIVER>           \t\t"
+			"bind FPGA with kernel driver\n"
+		" unbind <BDF>                  \t\t"
+			"unbind FPGA from kernel driver\n"
+		" probe <BDF>                   \t\t"
+			"probe FPGA with IFPGA driver\n"
+		" remove <BDF>                  \t\t"
+			"remove FPGA from IFPGA driver\n"
+		" flash <BDF> <FILE>            \t\t"
+			"update flash of FPGA\n"
+		" pr <BDF> <PORT> <FILE>        \t\t"
+			"partial reconfigure FPGA\n"
+		" reboot <BDF> <fpga|bmc> <0-1> \t\t"
+			"reboot FPGA or MAX10\n"
+		" cancel <BDF>                  \t\t"
+			"cancel flash update\n"
+		" pci_read <BDF> <0-1024>       \t\t"
+			"read PCI configuration space\n"
+		" pci_write <BDF> <0-1024> <NUM>\t\t"
+			"write PCI configuration space\n"
+		" quit                          \t\t"
+			"exit DPDK application\n"
+		" help                          \t\t"
+			"show commands list\n");
+}
+
+cmdline_parse_token_string_t cmd_help_help =
+	TOKEN_STRING_INITIALIZER(struct cmd_help_result, help, "help");
+
+cmdline_parse_inst_t cmd_help = {
+	.f = cmd_help_parsed,
+	.data = NULL,
+	.help_str = "show commands list",
+	.tokens = {
+		(void *)&cmd_help_help,
+		NULL,
+	},
+};
+
+/****** CONTEXT (list of commands) */
+cmdline_parse_ctx_t main_ctx[] = {
+	(cmdline_parse_inst_t *)&cmd_get_image_info,
+	(cmdline_parse_inst_t *)&cmd_get_api_version,
+	(cmdline_parse_inst_t *)&cmd_get_proc_type,
+	(cmdline_parse_inst_t *)&cmd_get_status,
+	(cmdline_parse_inst_t *)&cmd_get_property,
+	(cmdline_parse_inst_t *)&cmd_phy_info,
+	(cmdline_parse_inst_t *)&cmd_get_parent,
+	(cmdline_parse_inst_t *)&cmd_get_child,
+	(cmdline_parse_inst_t *)&cmd_get_pf1,
+	(cmdline_parse_inst_t *)&cmd_set_log_level,
+	(cmdline_parse_inst_t *)&cmd_set_log_file,
+	(cmdline_parse_inst_t *)&cmd_set_status,
+	(cmdline_parse_inst_t *)&cmd_enumerate,
+	(cmdline_parse_inst_t *)&cmd_bind,
+	(cmdline_parse_inst_t *)&cmd_unbind,
+	(cmdline_parse_inst_t *)&cmd_probe,
+	(cmdline_parse_inst_t *)&cmd_remove,
+	(cmdline_parse_inst_t *)&cmd_flash,
+	(cmdline_parse_inst_t *)&cmd_pr,
+	(cmdline_parse_inst_t *)&cmd_reboot,
+	(cmdline_parse_inst_t *)&cmd_cancel,
+	(cmdline_parse_inst_t *)&cmd_pci_read,
+	(cmdline_parse_inst_t *)&cmd_pci_write,
+	(cmdline_parse_inst_t *)&cmd_quit,
+	(cmdline_parse_inst_t *)&cmd_help,
+	NULL,
+};
diff --git a/examples/ifpga/commands.h b/examples/ifpga/commands.h
new file mode 100644
index 0000000000..06fe9a68b2
--- /dev/null
+++ b/examples/ifpga/commands.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2020-2021 Intel Corporation
+ */
+
+#ifndef _COMMANDS_H_
+#define _COMMANDS_H_
+
+#include "opae_api.h"
+
+extern cmdline_parse_ctx_t main_ctx[];
+
+typedef struct {
+	char s[38];
+} uuid_str;
+
+#endif /* _COMMANDS_H_ */
diff --git a/examples/ifpga/main.c b/examples/ifpga/main.c
new file mode 100644
index 0000000000..e9380d581a
--- /dev/null
+++ b/examples/ifpga/main.c
@@ -0,0 +1,38 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2020-2021 Intel Corporation
+ */
+
+#include <stdio.h>
+#include <string.h>
+#include <stdint.h>
+#include <errno.h>
+#include <sys/queue.h>
+
+#include <cmdline_rdline.h>
+#include <cmdline_parse.h>
+#include <cmdline_socket.h>
+#include <cmdline.h>
+
+#include <rte_memory.h>
+#include <rte_eal.h>
+#include <rte_debug.h>
+
+#include "commands.h"
+
+
+int main(int argc, char **argv)
+{
+	struct cmdline *cl;
+	int ret;
+
+	ret = opae_init_eal(argc, argv);
+	if (ret < 0)
+		rte_panic("Cannot init EAL\n");
+	cl = cmdline_stdin_new(main_ctx, "opae> ");
+	if (cl == NULL)
+		rte_panic("Cannot create cmdline instance\n");
+	cmdline_interact(cl);
+	opae_cleanup_eal();
+	cmdline_stdin_exit(cl);
+	return 0;
+}
diff --git a/examples/ifpga/meson.build b/examples/ifpga/meson.build
new file mode 100644
index 0000000000..b88a37a7a5
--- /dev/null
+++ b/examples/ifpga/meson.build
@@ -0,0 +1,20 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2020-2021 Intel Corporation
+
+# meson file, for building this example as part of a main DPDK build.
+#
+# To build this example as a standalone application with an already-installed
+# DPDK instance, use 'make'
+
+# require the raw_ifpga library
+build = dpdk_conf.has('RTE_RAW_IFPGA')
+if not build
+	subdir_done()
+endif
+
+deps += 'raw_ifpga'
+allow_experimental_apis = true
+sources = files(
+	'main.c', 'commands.c', 'opae_api.c'
+)
+cflags += '-fPIC'
diff --git a/examples/ifpga/opae_api.c b/examples/ifpga/opae_api.c
new file mode 100644
index 0000000000..7c27a7a278
--- /dev/null
+++ b/examples/ifpga/opae_api.c
@@ -0,0 +1,1630 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2020 Intel Corporation
+ */
+
+#include <stdlib.h>
+#include <string.h>
+#include <dirent.h>
+#include <fcntl.h>
+#include <glob.h>
+#include <unistd.h>
+#include <sys/stat.h>
+#include <rte_eal.h>
+#include <rte_pci.h>
+#include <rte_bus_pci.h>
+#include <rte_rawdev_pmd.h>
+#include "rte_pmd_ifpga.h"
+#include "opae_api.h"
+
+
+int opae_log_level;
+FILE *opae_log_file;
+
+static opae_api_version api_ver = {21, 5, 0};
+static int eal_inited;
+static uint32_t dev_aer[2] = {0};
+
+static const char * const log_level_name[] = {"CRITICAL", "ERROR",
+	"WARNING", "INFORMATION", "DEBUG"};
+static const char * const proc_type_name[] = {"NON-DPDK", "PRIMARY",
+	"SECONDARY"};
+static const char * const platform_name[] = {"Vista Creek", "Rush Creek",
+	"Darby Creek", "Lightning Creek"};
+static const char * const release_name[] = {"Pre-Alpha", "Alpha", "Beta", "PV"};
+static const char * const interface_type[] = {"8x10G", "4x25G", "2x1x25G",
+	"4x25G+2x25G", "2x2x25G", "2x1x25Gx2FVL", "1x2x25G"};
+
+void opae_get_api_version(opae_api_version *version)
+{
+	if (version)
+		memcpy(version, &api_ver, sizeof(opae_api_version));
+	opae_log_info("API version is %u.%u.%u\n",
+		api_ver.major, api_ver.minor, api_ver.micro);
+}
+
+int opae_set_log_level(int level)
+{
+	if ((level >= OPAE_LOG_API) && (level <= OPAE_LOG_DEBUG))
+		opae_log_level = level;
+	opae_log_api("Current log level is %s\n",
+		log_level_name[opae_log_level]);
+	return opae_log_level;
+}
+
+int opae_set_log_file(char *path, int clean)
+{
+	FILE *f = NULL;
+	time_t start;
+	struct tm *lt = NULL;
+
+	if (path) {
+		if (clean)
+			f = fopen(path, "w+");
+		else
+			f = fopen(path, "a+");
+
+		if (f) {
+			if (opae_log_file) {
+				fclose(opae_log_file);
+				opae_log_file = NULL;
+			}
+			time(&start);
+			lt = localtime(&start);
+			if (lt)
+				fprintf(f, "================%d-%02d-%02d "
+					"%02d:%02d:%02d================\n",
+					1900 + lt->tm_year, 1 + lt->tm_mon,
+					lt->tm_mday,
+					lt->tm_hour, lt->tm_min, lt->tm_sec);
+			fflush(f);
+			opae_log_file = f;
+		} else {
+			opae_log_err("failed to open log file \'%s\'\n", path);
+			return -1;
+		}
+	} else {
+		if (opae_log_file) {
+			fclose(opae_log_file);
+			opae_log_file = NULL;
+		}
+	}
+
+	return 0;
+}
+
+int opae_get_image_info(const char *image, opae_img_info *info)
+{
+	int fd = -1;
+	off_t file_size = 0;
+	opae_img_hdr hdr;
+	ssize_t read_size = 0;
+	int ret = 0;
+
+	if (!image || !info) {
+		opae_log_err("Input parameter of %s is invalid\n", __func__);
+		return -EINVAL;
+	}
+
+	fd = open(image, O_RDONLY);
+	if (fd < 0) {
+		opae_log_err("Failed to open \'%s\' for RD [e:%s]\n",
+			image, strerror(errno));
+		return -EIO;
+	}
+
+	file_size = lseek(fd, 0, SEEK_END);
+	if (file_size < (OPAE_IMG_HDR_SIZE + OPAE_IMG_PL_MIN_SIZE)) {
+		opae_log_err("Size of \'%s\' is less than expected [e:%u]\n",
+			image, OPAE_IMG_HDR_SIZE + OPAE_IMG_PL_MIN_SIZE);
+		ret = -EINVAL;
+		goto close_fd;
+	}
+
+	/* read image header */
+	lseek(fd, 0, SEEK_SET);
+	read_size = read(fd, (void *)&hdr, sizeof(opae_img_hdr));
+	if (read_size < 0) {
+		opae_log_err("Failed to read from \'%s\' [e:%s]\n",
+			image, strerror(errno));
+		ret = -EIO;
+		goto close_fd;
+	}
+	if ((size_t)read_size != sizeof(opae_img_hdr)) {
+		opae_log_err("Read length %zd is not expected [e:%zu]\n",
+			read_size, sizeof(opae_img_hdr));
+		ret = -EIO;
+		goto close_fd;
+	}
+
+	info->total_len = file_size;
+	/* check signed image header */
+	if (hdr.magic == OPAE_IMG_BLK0_MAGIC) {
+		info->type = OPAE_IMG_TYPE(hdr.payload_type);
+		info->subtype = OPAE_IMG_SUBTYPE(hdr.payload_type);
+		info->payload_offset = OPAE_IMG_HDR_SIZE;
+		info->payload_len = hdr.payload_len;
+	} else {
+		opae_log_err("Image \'%s\' can not be recognized\n", image);
+		ret = -EINVAL;
+	}
+close_fd:
+	close(fd);
+	return ret;
+}
+
+static int write_file(char *path, char *buf, int size)
+{
+	int fd = -1;
+	ssize_t n = 0;
+
+	if (!path || !buf || (size <= 0))
+		return -EINVAL;
+
+	fd = open(path, O_WRONLY);
+	if (fd < 0) {
+		opae_log_err("Failed to open \'%s\' for WR [e:%s]\n",
+			path, strerror(errno));
+		return -EIO;
+	}
+	opae_log_dbg("Write \"%s\" to \'%s\'\n", buf, path);
+	n = write(fd, buf, size);
+	if (n < size)  {
+		opae_log_err("Failed to write to \'%s\' [e:%s]\n",
+			path, strerror(errno));
+		close(fd);
+		return -EIO;
+	}
+	close(fd);
+
+	return 0;
+}
+
+static int read_file(char *path, char *buf, int size)
+{
+	int fd = -1;
+	ssize_t n = 0;
+
+	if (!path || !buf || (size <= 0))
+		return -EINVAL;
+
+	fd = open(path, O_RDONLY);
+	if (fd < 0) {
+		opae_log_err("Failed to open \'%s\' for RD [e:%s]\n",
+			path, strerror(errno));
+		return -EIO;
+	}
+	n = read(fd, buf, size);
+	if (n < 0)  {
+		opae_log_err("Failed to read from \'%s\' [e:%s]\n",
+			path, strerror(errno));
+		close(fd);
+		return -EIO;
+	}
+	close(fd);
+
+	if (n > 0)
+		buf[n-1] = 0;
+
+	opae_log_dbg("Read \"%s\" from \'%s\'\n", buf, path);
+	return 0;
+}
+
+int opae_get_proc_type(void)
+{
+	int type = -1;
+
+	if (eal_inited) {
+		if (rte_eal_process_type() == RTE_PROC_PRIMARY)
+			type = 0;
+		else
+			type = 1;
+	}
+	opae_log_info("Current process type is %s\n", proc_type_name[type+1]);
+
+	return type;
+}
+
+static bool check_eal(int inited)
+{
+	if (!eal_inited) {
+		if (inited) {
+			opae_log_warn("EAL is not initialized\n");
+			return 0;
+		}
+	} else {
+		if (!inited) {
+			opae_log_warn("EAL is already initialized\n");
+			return 0;
+		}
+	}
+
+	return 1;
+}
+
+int opae_init_eal(int argc, char **argv)
+{
+	int ret = 0;
+
+	if (!check_eal(0))
+		return ret;
+
+	opae_log_level = OPAE_LOG_ERR;
+
+	ret = rte_eal_init(argc, argv);
+	if (ret < 0) {
+		if (rte_errno == EALREADY) {
+			eal_inited = 1;
+			return 0;
+		}
+		opae_log_err("Cannot initialize EAL [e:%d]\n", ret);
+		if (rte_eal_cleanup())
+			opae_log_warn("EAL could not release all resources\n");
+	} else {
+		eal_inited = 1;
+		opae_log_info("Initialize EAL done\n");
+	}
+
+	return ret;
+}
+
+int opae_cleanup_eal(void)
+{
+	int ret = 0;
+
+	if (!check_eal(1))
+		return -EPERM;
+
+	rte_pmd_ifpga_cleanup();
+
+	ret = rte_eal_cleanup();
+	if (ret)
+		opae_log_err("Failed to cleanup EAL [e:%d]\n", ret);
+
+	if (opae_log_file) {
+		fclose(opae_log_file);
+		opae_log_file = NULL;
+	}
+
+	return ret;
+}
+
+static int compare_pci_id(opae_pci_id *id, opae_pci_id *expected_id)
+{
+	if ((expected_id->class_id != BIT_SET_32) &&
+		(expected_id->class_id != id->class_id))
+		return -1;
+	if ((expected_id->vendor_id != BIT_SET_16) &&
+		(expected_id->vendor_id != id->vendor_id))
+		return -1;
+	if ((expected_id->device_id != BIT_SET_16) &&
+		(expected_id->device_id != id->device_id))
+		return -1;
+	if ((expected_id->subsystem_vendor_id != BIT_SET_16) &&
+		(expected_id->subsystem_vendor_id != id->subsystem_vendor_id))
+		return -1;
+	if ((expected_id->subsystem_device_id != BIT_SET_16) &&
+		(expected_id->subsystem_device_id != id->subsystem_device_id))
+		return -1;
+
+	return 0;
+}
+
+static int parse_sysfs_value(char *node, uint32_t *val)
+{
+	char buf[16];
+	char *end = NULL;
+	int ret = 0;
+
+	ret = read_file(node, buf, sizeof(buf));
+	if (ret < 0)
+		return ret;
+
+	*val = (uint32_t)strtoul(buf, &end, 0);
+	return 0;
+}
+
+static int get_pci_id(const char *dev_path, opae_pci_id *id)
+{
+	char path[PATH_MAX] = {0};
+	uint32_t tmp;
+
+	if (!dev_path || !id)
+		return -EINVAL;
+
+	snprintf(path, sizeof(path), "%s/vendor", dev_path);
+	if (parse_sysfs_value(path, &tmp) < 0)
+		return -ENODEV;
+	id->vendor_id = (uint16_t)tmp;
+
+	snprintf(path, sizeof(path), "%s/device", dev_path);
+	if (parse_sysfs_value(path, &tmp) < 0)
+		return -ENODEV;
+	id->device_id = (uint16_t)tmp;
+
+	snprintf(path, sizeof(path), "%s/subsystem_vendor", dev_path);
+	if (parse_sysfs_value(path, &tmp) < 0)
+		return -ENODEV;
+	id->subsystem_vendor_id = (uint16_t)tmp;
+
+	snprintf(path, sizeof(path), "%s/subsystem_device", dev_path);
+	if (parse_sysfs_value(path, &tmp) < 0)
+		return -ENODEV;
+	id->subsystem_device_id = (uint16_t)tmp;
+
+	snprintf(path, sizeof(path), "%s/class", dev_path);
+	if (parse_sysfs_value(path, &tmp) < 0)
+		return -ENODEV;
+	id->class_id = (uint32_t)tmp & RTE_CLASS_ANY_ID;
+
+	return 0;
+}
+
+static int extract_path(char *in, int ridx, char *out, uint32_t size)
+{
+	char src[PATH_MAX] = {0};
+	char *p = NULL;
+	int ret = 0;
+
+	if (!in || (strlen(in) > PATH_MAX) || (ridx < 0) || !out)
+		return -EINVAL;
+
+	strncpy(src, in, sizeof(src));
+	*out = 0;
+
+	while (1) {
+		p = strrchr(src, '/');
+		if (p) {
+			*p++ = 0;
+			if (*p) {
+				if (ridx-- <= 0) {
+					if (size > strlen(p)) {
+						strncpy(out, p, size);
+						ret = strlen(p);
+					}
+					break;
+				}
+			}
+		} else {
+			break;
+		}
+	}
+
+	return ret;
+}
+
+int opae_enumerate(opae_pci_id *filter, pcidev_id list, int size)
+{
+	DIR *dir = NULL;
+	struct dirent *dirent = NULL;
+	char path[PATH_MAX] = {0};
+	opae_pci_id id;
+	int n = 0;
+
+	if (!filter || (size < 0) || (!list && (size > 0))) {
+		opae_log_err("Input parameter of %s is invalid\n", __func__);
+		return -EINVAL;
+	}
+
+	dir = opendir(rte_pci_get_sysfs_path());
+	if (!dir) {
+		opae_log_err("Failed to open \'%s\'\n",
+			rte_pci_get_sysfs_path());
+		return -EINVAL;
+	}
+	while ((dirent = readdir(dir))) {
+		if (!strcmp(dirent->d_name, "."))
+			continue;
+		if (!strcmp(dirent->d_name, ".."))
+			continue;
+
+		snprintf(path, PATH_MAX, "%s/%s", rte_pci_get_sysfs_path(),
+			dirent->d_name);
+		if (get_pci_id(path, &id) < 0)
+			continue;
+		if (compare_pci_id(&id, filter) < 0)
+			continue;
+
+		if (n++ < size) {
+			snprintf(list->bdf, sizeof(list->bdf), "%s",
+				dirent->d_name);
+			list++;
+		}
+	}
+	closedir(dir);
+
+	return n;
+}
+
+static int get_driver(pcidev_id id, char *drv_name, uint32_t size)
+{
+	char path[PATH_MAX] = {0};
+	char link[PATH_MAX] = {0};
+	int ret = 0;
+
+	if (!id || !drv_name) {
+		ret = -EINVAL;
+		goto end;
+	}
+	size--;   /* reserve one byte for the end of string */
+
+	snprintf(path, PATH_MAX, "%s/%s/driver",
+		rte_pci_get_sysfs_path(), id->bdf);
+	ret = readlink(path, link, PATH_MAX);
+	if (ret >= PATH_MAX) {
+		opae_log_err("Link path too long [%d]\n", ret);
+		ret = -ENAMETOOLONG;
+		goto end;
+	}
+	if (ret > 0) {
+		ret = extract_path(link, 0, drv_name, size);
+	} else {
+		*drv_name = 0;
+		opae_log_info("No link path for \'%s\'\n", path);
+		ret = 0;
+	}
+
+end:
+	if (ret < 0)
+		opae_log_err("Failed to get driver of %s\n", id->bdf);
+
+	return ret;
+}
+
+static int get_pci_addr(const char *bdf, opae_pci_addr *addr)
+{
+	struct rte_pci_addr pci_addr;
+	int ret = 0;
+
+	if (!bdf || !addr)
+		return -EINVAL;
+
+	ret = rte_pci_addr_parse(bdf, &pci_addr);
+	if (ret == 0) {
+		addr->domain = pci_addr.domain;
+		addr->bus = pci_addr.bus;
+		addr->devid = pci_addr.devid;
+		addr->function = pci_addr.function;
+	}
+
+	return ret;
+}
+
+static int check_pcidev_id(pcidev_id id)
+{
+	if (!id) {
+		opae_log_err("ID is NULL\n");
+		return -1;
+	}
+
+	if (strnlen(id->bdf, PCI_PRI_STR_SIZE) == PCI_PRI_STR_SIZE) {
+		opae_log_err("PCI address is too long\n");
+		return -2;
+	}
+
+	return 0;
+}
+
+static int get_rawdev_id(pcidev_id id, uint16_t *dev_id, int log)
+{
+	if (check_pcidev_id(id))
+		return -1;
+
+	if (rte_pmd_ifpga_get_dev_id(id->bdf, dev_id)) {
+		if (log)
+			opae_log_warn("%s is not probed by ifpga driver\n",
+				id->bdf);
+		return -1;
+	}
+
+	return 0;
+}
+
+static struct rte_pci_device *get_rte_pcidev(pcidev_id id, int log)
+{
+	const struct rte_pci_bus *pci_bus = NULL;
+	struct rte_pci_device *pci_dev = NULL;
+	struct rte_pci_addr addr;
+
+	if (check_pcidev_id(id))
+		return NULL;
+
+	if (rte_pci_addr_parse(id->bdf, &addr)) {
+		opae_log_err("PCI address %s is invalid\n", id->bdf);
+		return NULL;
+	}
+
+	pci_bus = rte_pmd_ifpga_get_pci_bus();
+	if (pci_bus) {
+		TAILQ_FOREACH(pci_dev, &pci_bus->device_list, next) {
+			if (!rte_pci_addr_cmp(&pci_dev->addr, &addr))
+				return pci_dev;
+		}
+	}
+
+	if (log)
+		opae_log_err("No rte_pci_device for %s\n", id->bdf);
+
+	return NULL;
+}
+
+int opae_load_rsu_status(pcidev_id id, uint32_t *status, uint32_t *progress)
+{
+	uint16_t dev_id = 0;
+
+	if (!check_eal(1))
+		return -EPERM;
+
+	if (get_rawdev_id(id, &dev_id, 1))
+		return -ENODEV;
+
+	return rte_pmd_ifpga_get_rsu_status(dev_id, status, progress);
+}
+
+int opae_store_rsu_status(pcidev_id id, uint32_t status, uint32_t progress)
+{
+	uint16_t dev_id = 0;
+
+	if (!check_eal(1))
+		return -EPERM;
+
+	if (get_rawdev_id(id, &dev_id, 1))
+		return -ENODEV;
+
+	return rte_pmd_ifpga_set_rsu_status(dev_id, status, progress);
+}
+
+static int get_pci_property(pcidev_id id, opae_pci_property *prop)
+{
+	char path[PATH_MAX] = {0};
+	int ret = 0;
+
+	if (!id || !prop)
+		return -EINVAL;
+
+	snprintf(path, PATH_MAX, "%s/%s", rte_pci_get_sysfs_path(), id->bdf);
+
+	ret = get_pci_id(path, &prop->id);
+	if (ret < 0)
+		return ret;
+
+	ret = get_pci_addr(id->bdf, &prop->addr);
+	if (ret < 0)
+		return ret;
+
+	snprintf(prop->pci_addr, OPAE_NAME_SIZE, "%s", id->bdf);
+	get_driver(id, prop->drv_name, sizeof(prop->drv_name));
+
+	return 0;
+}
+
+static int get_fme_property(rte_pmd_ifpga_common_prop *common,
+	opae_fme_property *prop)
+{
+	opae_bitstream_id bbs_id;
+
+	if (!common || !prop)
+		return -EINVAL;
+
+	prop->boot_page = common->boot_page;
+	prop->num_ports = common->num_ports;
+	prop->bitstream_id = common->bitstream_id;
+	prop->bitstream_metadata = common->bitstream_metadata;
+	memcpy(prop->pr_id.b, common->pr_id.b, sizeof(opae_uuid));
+
+	bbs_id.id = prop->bitstream_id;
+	if (bbs_id.major < sizeof(platform_name) / sizeof(char *)) {
+		snprintf(prop->platform_name,
+			sizeof(prop->platform_name), "%s",
+			platform_name[bbs_id.major]);
+	} else {
+		snprintf(prop->platform_name,
+			sizeof(prop->platform_name), "unknown");
+	}
+
+	snprintf(prop->dcp_version, sizeof(prop->dcp_version),
+		"DCP 1.%u", bbs_id.minor);
+
+	if (bbs_id.patch < sizeof(release_name)/sizeof(char *)) {
+		snprintf(prop->release_name, sizeof(prop->release_name),
+			"%s", release_name[bbs_id.patch]);
+	} else {
+		snprintf(prop->release_name, sizeof(prop->release_name),
+			"unknown");
+	}
+
+	if (bbs_id.major == 0) {  /* Vista Creek */
+		if (bbs_id.interface <
+			sizeof(interface_type) / sizeof(char *)) {
+			snprintf(prop->interface_type,
+				sizeof(prop->interface_type), "%s",
+				interface_type[bbs_id.interface]);
+		} else {
+			snprintf(prop->interface_type,
+				sizeof(prop->interface_type), "unknown");
+		}
+	} else {
+		snprintf(prop->interface_type,
+			sizeof(prop->interface_type), "unknown");
+	}
+
+	snprintf(prop->build_version, sizeof(prop->build_version),
+		"%u.%u.%u", bbs_id.build_major, bbs_id.build_minor,
+		bbs_id.build_patch);
+
+	return 0;
+}
+
+static int get_port_property(rte_pmd_ifpga_port_prop *port,
+	opae_port_property *prop)
+{
+	if (!port || !prop)
+		return -EINVAL;
+
+	memcpy(prop->afu_id.b, port->afu_id.b, sizeof(opae_uuid));
+	prop->type = port->type;
+
+	return 0;
+}
+
+static int get_bmc_property(rte_pmd_ifpga_common_prop *common,
+	opae_bmc_property *prop)
+{
+	opae_bmc_version ver;
+
+	if (!common || !prop)
+		return -EINVAL;
+
+	ver.version = common->bmc_version;
+	snprintf(prop->bmc_version, sizeof(prop->bmc_version), "%c.%u.%u.%u",
+		ver.board, ver.major, ver.minor, ver.micro);
+
+	ver.version = common->bmc_nios_version;
+	snprintf(prop->fw_version, sizeof(prop->fw_version), "%c.%u.%u.%u",
+		ver.board, ver.major, ver.minor, ver.micro);
+
+	return 0;
+}
+
+int opae_get_property(pcidev_id id, opae_fpga_property *prop, int type)
+{
+	uint16_t dev_id = 0;
+	uint32_t i = 0;
+	rte_pmd_ifpga_prop fpga_prop;
+	int ret = 0;
+
+	if (!prop) {
+		opae_log_err("Input parameter of %s is invalid\n", __func__);
+		return -EINVAL;
+	}
+
+	if (type == 0)
+		type = OPAE_PROP_ALL;
+
+	memset(prop, 0, sizeof(opae_fpga_property));
+
+	/* PCI properties */
+	if (type & OPAE_PROP_PCI) {
+		if (get_pci_property(id, &prop->pci) < 0) {
+			opae_log_err("Failed to get PCI property\n");
+			return -EAGAIN;
+		}
+	}
+
+	if (type == OPAE_PROP_PCI)
+		return 0;
+
+	if (!check_eal(1))
+		return -EPERM;
+
+	if (get_rawdev_id(id, &dev_id, 1))
+		return -ENODEV;
+
+	ret = rte_pmd_ifpga_get_property(dev_id, &fpga_prop);
+	if (ret) {
+		opae_log_err("Failed to get FPGA property [e:%d]\n", ret);
+		return -EAGAIN;
+	}
+
+	/* FME properties */
+	if (type & (OPAE_PROP_FME | OPAE_PROP_PORT))
+		get_fme_property(&fpga_prop.common, &prop->fme);
+
+	/* PORT properties */
+	if (type & OPAE_PROP_PORT) {
+		for (i = 0; i < prop->fme.num_ports; i++) {
+			prop->port[i].index = i;
+			get_port_property(&fpga_prop.port[i], &prop->port[i]);
+		}
+	}
+
+	/* BMC properties */
+	if (type & OPAE_PROP_BMC)
+		get_bmc_property(&fpga_prop.common, &prop->bmc);
+
+	return 0;
+}
+
+int opae_get_phy_info(pcidev_id id, opae_phy_info *info)
+{
+	uint16_t dev_id = 0;
+	rte_pmd_ifpga_phy_info phy_info;
+	int ret = 0;
+
+	if (!info)
+		return -EINVAL;
+
+	if (get_rawdev_id(id, &dev_id, 1))
+		return -ENODEV;
+
+	ret = rte_pmd_ifpga_get_phy_info(dev_id, &phy_info);
+	if (ret) {
+		opae_log_err("Failed to get PHY information [e:%d]\n", ret);
+		return -EAGAIN;
+	}
+
+	info->num_retimers = phy_info.num_retimers;
+	info->link_speed = phy_info.link_speed;
+	info->link_status = phy_info.link_status;
+
+	return 0;
+}
+
+static int update_driver(pcidev_id id, char *drv_name)
+{
+	struct rte_pci_device *pci_dev = NULL;
+	char name[OPAE_NAME_SIZE] = {0};
+	int ret = 0;
+
+	if (check_pcidev_id(id))
+		return -EINVAL;
+
+	if (drv_name) {
+		if (strlen(drv_name) >= OPAE_NAME_SIZE) {
+			opae_log_err("Driver name \'%s\' too long\n",
+				drv_name);
+			return -EINVAL;
+		}
+		strncpy(name, drv_name, sizeof(name));
+	} else {
+		ret = get_driver(id, name, sizeof(name));
+		if (ret < 0)
+			return ret;
+	}
+
+	pci_dev = get_rte_pcidev(id, 0);
+	if (pci_dev) {
+		if (strlen(name) == 0) {
+			pci_dev->kdrv = RTE_PCI_KDRV_NONE;
+		} else {
+			if (!strcmp(name, OPAE_KDRV_VFIO_PCI))
+				pci_dev->kdrv = RTE_PCI_KDRV_VFIO;
+			else if (!strcmp(name, OPAE_KDRV_IGB_UIO))
+				pci_dev->kdrv = RTE_PCI_KDRV_IGB_UIO;
+			else if (!strcmp(name, OPAE_KDRV_UIO_PCI))
+				pci_dev->kdrv = RTE_PCI_KDRV_UIO_GENERIC;
+			else
+				pci_dev->kdrv = RTE_PCI_KDRV_UNKNOWN;
+		}
+	}
+
+	return 0;
+}
+
+int opae_unbind_driver(pcidev_id id)
+{
+	uint16_t dev_id = 0;
+	char path[PATH_MAX] = {0};
+	char drv_name[OPAE_NAME_SIZE] = {0};
+	char null[] = {0};
+	int ret = 0;
+
+	if (!get_rawdev_id(id, &dev_id, 0)) {
+		opae_log_err("%s is probed, remove it first\n", id->bdf);
+		return -EBUSY;
+	}
+
+	ret = get_driver(id, drv_name, sizeof(drv_name));
+	if (ret < 0)
+		return ret;
+
+	if (strlen(drv_name) > 0) {
+		snprintf(path, PATH_MAX, "/sys/bus/pci/drivers/%s/unbind",
+			drv_name);
+		ret = write_file(path, id->bdf, strlen(id->bdf) + 1);
+		if (ret == 0)
+			ret = update_driver(id, null);
+	}
+
+	return ret;
+}
+
+static int check_driver(const char *drv_name)
+{
+	char path[PATH_MAX] = {0};
+	struct stat buf;
+
+	if (!drv_name)
+		return -EINVAL;
+
+	if (strlen(drv_name) > 0) {
+		snprintf(path, PATH_MAX, "/sys/bus/pci/drivers/%s", drv_name);
+		if ((stat(path, &buf) < 0) || ((buf.st_mode & S_IFDIR) == 0)) {
+			opae_log_warn("Driver %s is not installed\n",
+				drv_name);
+			return -EINVAL;
+		}
+	}
+
+	return 0;
+}
+
+int opae_bind_driver(pcidev_id id, char *drv_name)
+{
+	char path[PATH_MAX] = {0};
+	char name[OPAE_NAME_SIZE] = {0};
+	char null[] = {0};
+	int ret = 0;
+
+	ret = check_driver(drv_name);
+	if (ret < 0)
+		return ret;
+
+	ret = get_driver(id, name, sizeof(name));
+	if (ret < 0)
+		return ret;
+
+	if (!strcmp(drv_name, name))   /* driver not change */
+		return 0;
+
+	ret = opae_unbind_driver(id);
+	if (ret < 0)
+		return ret;
+
+	if (strlen(drv_name) > 0) {
+		/* bind driver */
+		snprintf(path, PATH_MAX, "%s/%s/driver_override",
+			rte_pci_get_sysfs_path(), id->bdf);
+		ret = write_file(path, drv_name, strlen(drv_name) + 1);
+		if (ret < 0)
+			goto update_drv;
+
+		snprintf(path, PATH_MAX, "/sys/bus/pci/drivers/%s/bind",
+			drv_name);
+		ret = write_file(path, id->bdf, strlen(id->bdf) + 1);
+		if (ret < 0)
+			goto update_drv;
+
+		snprintf(path, PATH_MAX, "%s/%s/driver_override",
+			rte_pci_get_sysfs_path(), id->bdf);
+		ret = write_file(path, null, 1);
+		if (ret < 0)
+			goto update_drv;
+	}
+
+update_drv:
+	ret = update_driver(id, NULL);
+	if (ret < 0)
+		opae_log_err("Failed to update driver information of %s\n",
+			id->bdf);
+
+	return 0;
+}
+
+int opae_probe_device(pcidev_id id)
+{
+	const struct rte_pci_bus *pci_bus = NULL;
+	struct rte_pci_device *pci_dev = NULL;
+
+	if (check_pcidev_id(id))
+		return -EINVAL;
+
+	if (!check_eal(1))
+		return -EPERM;
+
+	/* make sure device is added in rte_pci_bus devices list */
+	pci_bus = rte_pmd_ifpga_get_pci_bus();
+	if (pci_bus && pci_bus->bus.scan)
+		pci_bus->bus.scan();
+
+	pci_dev = get_rte_pcidev(id, 1);
+	if (!pci_dev)
+		return -ENODEV;
+
+	if (pci_dev->kdrv != RTE_PCI_KDRV_VFIO) {
+		opae_log_err("vfio-pci driver is not bound to %s\n", id->bdf);
+		return -EINVAL;
+	}
+
+	if (!pci_bus || !pci_bus->bus.plug)
+		return -ENODEV;
+
+	return pci_bus->bus.plug(&pci_dev->device);
+}
+
+int opae_remove_device(pcidev_id id)
+{
+	struct rte_pci_device *pci_dev = NULL;
+	struct rte_pci_driver *pci_drv = NULL;
+	int ret = 0;
+
+	if (!check_eal(1))
+		return -EPERM;
+
+	pci_dev = get_rte_pcidev(id, 0);
+	if (pci_dev && pci_dev->driver) {
+		pci_drv = pci_dev->driver;
+		ret = pci_drv->remove(pci_dev);
+		if (ret < 0) {
+			opae_log_err("Failed to remove %s [e:%d]\n",
+				id->bdf, ret);
+			return ret;
+		}
+		pci_dev->driver = NULL;
+		pci_dev->device.driver = NULL;
+		if (pci_drv->drv_flags & RTE_PCI_DRV_NEED_MAPPING)
+			rte_pci_unmap_device(pci_dev);
+	}
+
+	return ret;
+}
+
+static int is_pac(pcidev_id id)
+{
+	char path[PATH_MAX] = {0};
+	opae_pci_id pci_id;
+
+	if (check_pcidev_id(id))
+		return 0;
+
+	snprintf(path, PATH_MAX, "%s/%s", rte_pci_get_sysfs_path(), id->bdf);
+	if (get_pci_id(path, &pci_id) < 0)
+		return 0;
+
+	if ((pci_id.vendor_id == 0x8086) && (pci_id.device_id == 0x0b30))
+		return 1;
+
+	return 0;
+}
+
+int opae_get_parent(pcidev_id id, pcidev_id parent)
+{
+	char path[PATH_MAX] = {0};
+	char link[PATH_MAX] = {0};
+	int ret = 0;
+
+	if (!id || !parent) {
+		opae_log_err("Input parameter of %s is invalid\n", __func__);
+		ret = -EINVAL;
+		goto end;
+	}
+
+	snprintf(path, PATH_MAX, "%s/%s", rte_pci_get_sysfs_path(), id->bdf);
+	ret = readlink(path, link, PATH_MAX);
+	if (ret >= PATH_MAX) {
+		opae_log_err("Length of link path exceeds %u\n", PATH_MAX);
+		ret = -ENAMETOOLONG;
+		goto end;
+	}
+
+	if (ret > 0) {
+		ret = extract_path(link, 1, parent->bdf, sizeof(parent->bdf));
+		if (!strncmp(parent->bdf, "pci", 3)) {
+			parent->bdf[0] = 0;
+			ret = -ENODEV;
+		}
+	} else {
+		parent->bdf[0] = 0;
+		if (ret == 0)
+			opae_log_err("Length of link path is 0\n");
+		else
+			opae_log_err("No link path for \'%s\'\n", path);
+	}
+end:
+	if (ret <= 0)
+		opae_log_err("%s has no parent\n", id->bdf);
+
+	return ret;
+}
+
+int opae_get_child(pcidev_id id, pcidev_id child, int size)
+{
+	glob_t pglob = {.gl_pathc = 0, .gl_pathv = NULL};
+	char path[PATH_MAX] = {0};
+	int i, count = 0;
+	int len = 0;
+	int ret = 0;
+
+	if (!id || (size < 0)) {
+		opae_log_err("Input parameter of %s is invalid\n", __func__);
+		return -EINVAL;
+	}
+
+	snprintf(path, PATH_MAX, "%s/%s/*:*:*.?", rte_pci_get_sysfs_path(),
+		id->bdf);
+	ret = glob(path, 0, NULL, &pglob);
+	if (ret == 0) {
+		if (child && (size > 0)) {
+			for (i = 0; i < (int)pglob.gl_pathc; i++) {
+				len = extract_path(pglob.gl_pathv[i], 0,
+					child->bdf, sizeof(child->bdf));
+				if (len <= 0) {
+					child->bdf[0] = 0;
+					continue;
+				}
+				if (++count >= size)
+					break;
+				child++;
+			}
+		} else {
+			count = (int)pglob.gl_pathc;
+		}
+		globfree(&pglob);
+	} else {
+		if (pglob.gl_pathv)
+			globfree(&pglob);
+	}
+
+	return count;
+}
+
+int opae_get_pf1(pcidev_id id, pcidev_id peer, int size)
+{
+	opae_pci_device parent;
+	opae_pci_device child[4];
+	int n = 0;
+	int ret = 0;
+
+	if (check_pcidev_id(id))
+		return -EINVAL;
+
+	if (!is_pac(id)) {
+		opae_log_info("%s has no peer function\n", id->bdf);
+		return -EINVAL;
+	}
+
+	ret = opae_get_parent(id, &parent);
+	if (ret < 0)
+		return -ENODEV;
+	ret = opae_get_parent(&parent, &parent);
+	if (ret < 0)
+		return -ENODEV;
+
+	n = opae_get_child(&parent, child,
+		sizeof(child) / sizeof(opae_pci_device));
+	/* there should have four downstream ports of PCI switch on board */
+	if (n == 4) {
+		n = opae_get_child(&child[3], peer, size);
+	} else {
+		peer->bdf[0] = 0;
+		opae_log_dbg("%s has %d child(s)\n", parent.bdf, n);
+		n = 0;
+	}
+
+	return n;
+}
+
+int opae_update_flash(pcidev_id id, const char *image, uint64_t *status)
+{
+	opae_img_info info;
+	uint16_t dev_id = 0;
+	int ret = 0;
+
+	ret = opae_get_image_info(image, &info);
+	if (ret < 0) {
+		opae_log_err("Failed to get image information [e:%d]\n", ret);
+		return -EINVAL;
+	}
+
+	if ((info.type != OPAE_IMG_TYPE_BBS) &&
+		(info.type != OPAE_IMG_TYPE_BMC)) {
+		opae_log_err("Image is not supported [t:%u]\n", info.type);
+		return -EOPNOTSUPP;
+	}
+
+	if (!check_eal(1))
+		return -EPERM;
+
+	if (get_rawdev_id(id, &dev_id, 1))
+		return -ENODEV;
+
+	return rte_pmd_ifpga_update_flash(dev_id, image, status);
+}
+
+int opae_cancel_flash_update(pcidev_id id, int force)
+{
+	uint16_t dev_id = 0;
+
+	if (!check_eal(1))
+		return -EPERM;
+
+	if (get_rawdev_id(id, &dev_id, 1))
+		return -ENODEV;
+
+	return rte_pmd_ifpga_stop_update(dev_id, force);
+}
+
+static int find_pci_ecap(int fd, unsigned int cap)
+{
+	uint32_t header = 0;
+	int ttl = (RTE_PCI_CFG_SPACE_EXP_SIZE - RTE_PCI_CFG_SPACE_SIZE) / 8;
+	int pos = RTE_PCI_CFG_SPACE_SIZE;
+	int ret = 0;
+
+	ret = pread(fd, &header, sizeof(header), pos);
+	if (ret < 0) {
+		opae_log_err("Failed to read from PCI configuration space [e:%s]\n",
+			strerror(errno));
+		return ret;
+	}
+	opae_log_dbg("Read 0x%08x from PCI configuration space 0x%x\n",
+		header, pos);
+
+	if (header == 0) {
+		opae_log_err("Capability is empty\n");
+		return 0;
+	}
+
+	while (ttl-- > 0) {
+		if ((RTE_PCI_EXT_CAP_ID(header) == cap) && (pos != 0))
+			return pos;
+
+		pos = RTE_PCI_EXT_CAP_NEXT(header);
+		if (pos < RTE_PCI_CFG_SPACE_SIZE) {
+			opae_log_err("Position of capability is invalid"
+						 "[e:%d]\n", pos);
+			break;
+		}
+		ret = pread(fd, &header, sizeof(header), pos);
+		if (ret < 0) {
+			opae_log_err("Failed to read from PCI config space [e:%s]\n",
+				strerror(errno));
+			return ret;
+		}
+		opae_log_dbg("Read 0x%08x from PCI configuration space 0x%x\n",
+			header, pos);
+	}
+
+	return 0;
+}
+
+static int set_aer(pcidev_id id, uint32_t v1, uint32_t v2, int record)
+{
+	char path[PATH_MAX] = {0};
+	uint32_t val = 0;
+	int fd = -1;
+	int pos = 0;
+	int ret = 0;
+
+	if (!id)
+		return -EINVAL;
+
+	snprintf(path, PATH_MAX, "%s/%s/config",
+		rte_pci_get_sysfs_path(), id->bdf);
+	fd = open(path, O_RDWR);
+	if (fd < 0) {
+		opae_log_err("Failed to open \'%s\' for RDWR [e:%s]\n",
+			path, strerror(errno));
+		return -EIO;
+	}
+
+	pos = find_pci_ecap(fd, RTE_PCI_EXT_CAP_ID_ERR);
+	if (pos <= 0) {
+		opae_log_warn("AER capability is not present\n");
+		ret = -ENXIO;
+		goto close_fd;
+	}
+
+	if (record) {
+		ret = pread(fd, &val, sizeof(val), pos + 0x08);
+		if (ret < 0) {
+			opae_log_err("Failed to read from PCI config space [e:%s]\n",
+				strerror(errno));
+			goto close_fd;
+		}
+		opae_log_dbg("Read 0x%08x from PCI configuration space 0x%x\n",
+			val, pos + 0x08);
+		dev_aer[0] = val;
+
+		ret = pread(fd, &val, sizeof(val), pos + 0x14);
+		if (ret < 0) {
+			opae_log_err("Failed to read from PCI config space [e:%s]\n",
+				strerror(errno));
+			goto close_fd;
+		}
+		opae_log_dbg("Read 0x%08x from PCI configuration space 0x%x\n",
+			val, pos + 0x14);
+		dev_aer[1] = val;
+	}
+
+	opae_log_dbg("Write 0x%08x to PCI configuration space 0x%x\n",
+		v1, pos + 0x08);
+	ret = pwrite(fd, &v1, sizeof(v1), pos + 0x08);
+	if (ret < 0) {
+		opae_log_err("Failed to write to PCI config space 0x%x [e:%s]\n",
+			pos + 0x08, strerror(errno));
+		goto close_fd;
+	}
+
+	opae_log_dbg("Write 0x%08x to PCI configuration space 0x%x\n",
+		v2, pos + 0x14);
+	ret = pwrite(fd, &v2, sizeof(v2), pos + 0x14);
+	if (ret < 0) {
+		opae_log_err("Failed to write to PCI config space 0x%x [e:%s]\n",
+			pos + 0x14, strerror(errno));
+	}
+
+close_fd:
+	close(fd);
+	return ret < 0 ? ret : 0;
+}
+
+static int enable_aer(pcidev_id id)
+{
+	if (check_pcidev_id(id))
+		return -EINVAL;
+
+	opae_log_info("Enable AER of %s\n", id->bdf);
+
+	return set_aer(id, dev_aer[0], dev_aer[1], 0);
+}
+
+static int disable_aer(pcidev_id id)
+{
+	if (check_pcidev_id(id))
+		return -EINVAL;
+
+	opae_log_info("Disable AER of %s\n", id->bdf);
+
+	return set_aer(id, 0xffffffff, 0xffffffff, 1);
+}
+
+static int remove_tree(pcidev_id id)
+{
+	int i, n = 0;
+	pcidev_id child;
+	int ret = 0;
+
+	if (check_pcidev_id(id))
+		return -EINVAL;
+
+	n = opae_get_child(id, NULL, 0);
+	if (n > 0) {
+		child = (pcidev_id)rte_zmalloc(NULL,
+			sizeof(opae_pci_device) * n, 0);
+		if (!child) {
+			opae_log_err("Failed to malloc for children of %s\n",
+				id->bdf);
+			ret = -ENOMEM;
+			goto end;
+		}
+
+		opae_get_child(id, child, n);
+		for (i = 0; i < n; i++)
+			remove_tree(&child[i]);
+		rte_free(child);
+	}
+
+end:
+	opae_remove_device(id);
+	return ret;
+}
+
+static int remove_device(pcidev_id id)
+{
+	char path[PATH_MAX] = {0};
+	char one[] = {'1', 0};
+	int ret = 0;
+
+	if (check_pcidev_id(id))
+		return -EINVAL;
+
+	opae_log_info("Remove %s from system\n", id->bdf);
+
+	snprintf(path, PATH_MAX, "%s/%s/remove",
+		rte_pci_get_sysfs_path(), id->bdf);
+	ret = write_file(path, one, strlen(one));
+	if (ret < 0) {
+		opae_log_err("Failed to remove %s from system\n", id->bdf);
+		return ret;
+	}
+
+	remove_tree(id);
+
+	return 0;
+}
+
+static int scan_device(pcidev_id parent, pcidev_id id)
+{
+	char path[PATH_MAX] = {0};
+	char bus[8] = {0};
+	char one[] = {'1', 0};
+	char pwr[16] = {0};
+	char pwr_on[] = {'o', 'n', 0};
+	int pwr_on_failed = 0;
+	int ret = 0;
+
+	if (!parent) {
+		opae_log_err("Input parameter of %s is invalid\n", __func__);
+		return -EINVAL;
+	}
+	opae_log_info("Rescan devices under %s\n", parent->bdf);
+
+	if (id) {   /* scan specified bus under parent device */
+		snprintf(path, PATH_MAX, "%s/%s/power/control",
+			rte_pci_get_sysfs_path(), parent->bdf);
+		ret = read_file(path, pwr, sizeof(pwr));
+		if (ret < 0)
+			return ret;
+
+		if (strcmp(pwr, "on")) {
+			ret = write_file(path, pwr_on, strlen(pwr_on));
+			if (ret < 0)
+				pwr_on_failed = 1;
+			else
+				sleep(1);
+		}
+
+		snprintf(bus, sizeof(bus), "%s", id->bdf);
+		snprintf(path, PATH_MAX, "%s/%s/pci_bus/%s/rescan",
+			rte_pci_get_sysfs_path(), parent->bdf, bus);
+		ret = write_file(path, one, strlen(one));
+		if (ret < 0)
+			return ret;
+
+		if (pwr_on_failed) {   /* workaround for power on failed */
+			ret = write_file(path, one, strlen(one));
+			if (ret < 0)
+				return ret;
+		}
+
+		if (strcmp(pwr, "on")) {
+			snprintf(path, PATH_MAX, "%s/%s/power/control",
+				rte_pci_get_sysfs_path(), parent->bdf);
+			ret = write_file(path, pwr, strlen(pwr));
+		}
+	} else {   /* scan all buses under parent device */
+		snprintf(path, PATH_MAX, "%s/%s/rescan",
+			rte_pci_get_sysfs_path(), parent->bdf);
+		ret = write_file(path, one, strlen(one));
+	}
+
+	return ret;
+}
+
+int opae_reboot_device(pcidev_id id, int type, int page)
+{
+	uint16_t dev_id = 0;
+	opae_pci_device fpga;    /* FPGA after reboot */
+	opae_pci_device parent;
+	opae_pci_device peer[2];   /* physical function 1 of FPGA */
+	opae_pci_device peer_parent;
+	opae_pci_device ups;   /* upstream port device */
+	opae_pci_device root;  /* port connected to PAC */
+	pcidev_id peer_primary = NULL;
+	char drv_name[OPAE_NAME_SIZE] = {0};
+	int n = 0;
+	int i = 0;
+	int ret = 0;
+
+	if (check_pcidev_id(id))
+		return -EINVAL;
+
+	if (!is_pac(id)) {
+		opae_log_err("%s can not be rebooted\n", id->bdf);
+		return -EINVAL;
+	}
+
+	if (get_rawdev_id(id, &dev_id, 1))
+		return -ENODEV;
+
+	ret = opae_get_parent(id, &parent);
+	if (ret < 0)
+		return -ENODEV;
+	ret = opae_get_parent(&parent, &ups);
+	if (ret < 0)
+		return -ENODEV;
+	ret = opae_get_parent(&ups, &root);
+	if (ret < 0)
+		return -ENODEV;
+
+	n = opae_get_pf1(id, peer, sizeof(peer) / sizeof(opae_pci_device));
+	if (n <= 0) {
+		opae_log_err("PF1 of %s is not found\n", id->bdf);
+	} else {
+		peer_primary = &peer[0];
+		ret = opae_get_parent(peer_primary, &peer_parent);
+		if (ret < 0)
+			return -ENODEV;
+	}
+
+	get_driver(id, drv_name, sizeof(drv_name));  /* save original driver */
+
+	if (!check_eal(1))
+		return -EPERM;
+
+	if (rte_pmd_ifpga_reboot_try(dev_id)) {
+		opae_log_warn("Update or reboot is in progress\n");
+		return -EAGAIN;
+	}
+
+	if (type == 0) {
+		/* disable AER */
+		ret = disable_aer(&parent);
+		if (ret < 0) {
+			opae_log_err("Failed to disable AER of %s\n",
+				parent.bdf);
+			goto reboot_end;
+		}
+		ret = disable_aer(&peer_parent);
+		if (ret < 0) {
+			opae_log_err("Failed to disable AER of %s\n",
+				peer_parent.bdf);
+			goto reboot_end;
+		}
+
+		/* trigger reconfiguration */
+		ret = rte_pmd_ifpga_reload(dev_id, type, page);
+		if (ret == 0) {
+			ret = remove_device(id);
+			for (i = 0; i < n; i++)
+				ret += remove_device(&peer[i]);
+			if (ret == 0) {
+				opae_log_info("Wait 10 seconds for FPGA reloading\n");
+				sleep(10);
+				ret = scan_device(&parent, id);
+				if (ret < 0)
+					opae_log_err("Failed to rescan %s\n",
+						id->bdf);
+				if (peer_primary) {
+					ret = scan_device(&peer_parent,
+						peer_primary);
+					if (ret < 0) {
+						opae_log_err("Failed to rescan %s\n",
+							peer_primary->bdf);
+					}
+				}
+			}
+		}
+
+		/* restore AER */
+		if (enable_aer(&parent) < 0) {
+			opae_log_err("Failed to enable AER of %s\n",
+				parent.bdf);
+		}
+		if (enable_aer(&peer_parent) < 0) {
+			opae_log_err("Failed to enable AER of %s\n",
+				peer_parent.bdf);
+		}
+	} else if (type == 1) {
+		/* disable AER */
+		ret = disable_aer(&root);
+		if (ret < 0) {
+			opae_log_err("Failed to disable AER of %s\n", root.bdf);
+			goto reboot_end;
+		}
+
+		/* trigger reconfiguration */
+		ret = rte_pmd_ifpga_reload(dev_id, type, page);
+		if (ret == 0) {
+			ret += remove_device(&ups);
+			if (ret == 0) {
+				opae_log_info("Wait 10 seconds for BMC reloading\n");
+				sleep(10);
+				ret = scan_device(&root, &ups);
+				if (ret < 0)
+					opae_log_err("Failed to rescan %s\n",
+						ups.bdf);
+			}
+		}
+
+		/* restore AER */
+		if (enable_aer(&root) < 0)
+			opae_log_err("Failed to enable AER of %s\n", root.bdf);
+	} else {
+		opae_log_err("Type of reboot is not supported [t:%d]\n", type);
+		ret = -EINVAL;
+		goto reboot_end;
+	}
+
+	/* update id if bdf changed after reboot */
+	if (opae_get_child(&parent, &fpga, 1) == 1) {
+		if (strcmp(id->bdf, fpga.bdf))
+			id = &fpga;
+	}
+
+	ret = opae_bind_driver(id, drv_name);
+	if (ret < 0)
+		opae_log_err("Failed to bind original driver of %s\n", id->bdf);
+
+	ret = opae_probe_device(id);
+	if (ret < 0)
+		opae_log_err("Failed to probe %s [e:%d]\n", id->bdf, ret);
+
+reboot_end:
+	rte_pmd_ifpga_set_rsu_status(dev_id, 0, 0);
+	return ret;
+}
+
+int opae_partial_reconfigure(pcidev_id id, int port, const char *gbs)
+{
+	uint16_t dev_id = 0;
+
+	if (!id || !gbs) {
+		opae_log_err("Input parameter of %s is invalid\n", __func__);
+		return -EINVAL;
+	}
+
+	if (!check_eal(1))
+		return -EPERM;
+
+	if (get_rawdev_id(id, &dev_id, 1))
+		return -ENODEV;
+
+	return rte_pmd_ifpga_partial_reconfigure(dev_id, port, gbs);
+}
+
+int opae_read_pci_cfg(pcidev_id id, uint32_t address, uint32_t *value)
+{
+	char path[PATH_MAX] = {0};
+	int fd = -1;
+	int ret = 0;
+
+	if (!id || !value) {
+		opae_log_err("Input parameter of %s is invalid\n", __func__);
+		return -EINVAL;
+	}
+
+	snprintf(path, PATH_MAX, "%s/%s/config", rte_pci_get_sysfs_path(),
+		id->bdf);
+	fd = open(path, O_RDONLY);
+	if (fd < 0) {
+		opae_log_dbg("Failed to open \'%s\' for RDONLY [e:%s]\n",
+			path, strerror(errno));
+		return -EIO;
+	}
+
+	ret = pread(fd, value, 4, address);
+	if (ret < 0) {
+		opae_log_err("Failed to read from PCI device %s [e:%s]\n",
+			id->bdf, strerror(errno));
+		close(fd);
+		return ret;
+	}
+
+	opae_log_dbg("CONFIG+0x%08x -> 0x%08x\n", address, *value);
+	close(fd);
+	return 0;
+}
+
+int opae_write_pci_cfg(pcidev_id id, uint32_t address, uint32_t value)
+{
+	char path[PATH_MAX] = {0};
+	int fd = -1;
+	int ret = 0;
+
+	if (!id) {
+		opae_log_err("Input parameter of %s is invalid\n", __func__);
+		return -EINVAL;
+	}
+
+	snprintf(path, PATH_MAX, "%s/%s/config", rte_pci_get_sysfs_path(),
+		id->bdf);
+	fd = open(path, O_WRONLY);
+	if (fd < 0) {
+		opae_log_dbg("Failed to open \'%s\' for WRONLY [e:%s]\n",
+			path, strerror(errno));
+		return -EIO;
+	}
+
+	ret = pwrite(fd, &value, 4, address);
+	if (ret < 0) {
+		opae_log_err("Failed to write to PCI device %s [e:%s]\n",
+			id->bdf, strerror(errno));
+		close(fd);
+		return ret;
+	}
+
+	opae_log_dbg("CONFIG+0x%08x <- 0x%08x\n", address, value);
+	close(fd);
+	return 0;
+}
diff --git a/examples/ifpga/opae_api.h b/examples/ifpga/opae_api.h
new file mode 100644
index 0000000000..3b6bc01fbc
--- /dev/null
+++ b/examples/ifpga/opae_api.h
@@ -0,0 +1,244 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2020 Intel Corporation
+ */
+
+#ifndef _OPAE_API_H
+#define _OPAE_API_H
+
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <stdint.h>
+
+extern int opae_log_level;
+extern FILE *opae_log_file;
+
+#define OPAE_LOG_API      0  /**< Critical conditions.              */
+#define OPAE_LOG_ERR      1  /**< Error conditions.                 */
+#define OPAE_LOG_WARN     2  /**< Warning conditions.               */
+#define OPAE_LOG_INFO     3  /**< Informational.                    */
+#define OPAE_LOG_DEBUG    4  /**< Debug-level messages.             */
+
+#define opae_log(type, fmt, args...)             \
+do {                                             \
+	if (opae_log_level >= OPAE_LOG_##type) {     \
+		printf(fmt, ##args);                     \
+		if (opae_log_file) {                     \
+			fprintf(opae_log_file, fmt, ##args); \
+			fflush(opae_log_file);               \
+		}                                        \
+	}                                            \
+} while (0)
+
+#define opae_log_api(fmt, args...)  opae_log(API, "OPAE-API: "fmt, ##args)
+#define opae_log_err(fmt, args...)  opae_log(ERR, "OPAE-ERR: "fmt, ##args)
+#define opae_log_dbg(fmt, args...)  opae_log(DEBUG, "OPAE-DBG: "fmt, ##args)
+#define opae_log_warn(fmt, args...) opae_log(WARN, "OPAE-WARN: "fmt, ##args)
+#define opae_log_info(fmt, args...) opae_log(INFO, "OPAE-INFO: "fmt, ##args)
+
+#define EAL_INIT_FUNCTION    "init"
+#define EAL_DEFAULT_OPTIONS  "--proc-type auto"
+
+#define OPAE_KDRV_UNKNOWN           "unknown"
+#define OPAE_KDRV_VFIO_PCI          "vfio-pci"
+#define OPAE_KDRV_IGB_UIO           "igb_uio"
+#define OPAE_KDRV_UIO_PCI           "uio_pci_generic"
+#define OPAE_KDRV_INTEL_FPGA_PCI    "intel-fpga-pci"
+
+typedef struct {
+	uint32_t major;
+	uint32_t minor;
+	uint32_t micro;
+} opae_api_version;
+
+#define OPAE_NAME_SIZE  32
+
+typedef struct {
+	char bdf[OPAE_NAME_SIZE];   /* segment:bus:device.function */
+} opae_pci_device;
+
+typedef opae_pci_device *pcidev_id;
+
+typedef struct {
+	uint32_t class_id;            /**< Class ID or RTE_CLASS_ANY_ID. */
+	uint16_t vendor_id;           /**< Vendor ID or PCI_ANY_ID. */
+	uint16_t device_id;           /**< Device ID or PCI_ANY_ID. */
+	uint16_t subsystem_vendor_id; /**< Subsystem vendor ID or PCI_ANY_ID. */
+	uint16_t subsystem_device_id; /**< Subsystem device ID or PCI_ANY_ID. */
+} opae_pci_id;
+
+typedef struct {
+	uint32_t domain;              /**< Device domain */
+	uint8_t bus;                  /**< Device bus */
+	uint8_t devid;                /**< Device ID */
+	uint8_t function;             /**< Device function. */
+} opae_pci_addr;
+
+typedef struct {
+	char pci_addr[OPAE_NAME_SIZE];  /* segment:bus:device.function */
+	char drv_name[OPAE_NAME_SIZE];  /* vfio-pci, intel-fpga-pci, etc. */
+	opae_pci_id id;
+	opae_pci_addr addr;
+} opae_pci_property;
+
+#define BIT_SET_8   0xFF
+#define BIT_SET_16  0xFFFF
+#define BIT_SET_32  0xFFFFFFFF
+
+typedef struct {
+	uint8_t b[16];
+} opae_uuid;
+
+typedef struct {
+	uint32_t boot_page;
+	uint32_t num_ports;
+	uint64_t bitstream_id;
+	uint64_t bitstream_metadata;
+	opae_uuid pr_id;
+	char platform_name[OPAE_NAME_SIZE];
+	char dcp_version[OPAE_NAME_SIZE];
+	char release_name[OPAE_NAME_SIZE];
+	char interface_type[OPAE_NAME_SIZE];
+	char build_version[OPAE_NAME_SIZE];
+} opae_fme_property;
+
+typedef struct {
+	opae_uuid afu_id;
+	uint32_t type;   /* AFU memory access control type */
+	uint32_t index;  /* PORT index */
+} opae_port_property;
+
+typedef struct {
+	char bmc_version[OPAE_NAME_SIZE];
+	char fw_version[OPAE_NAME_SIZE];
+} opae_bmc_property;
+
+typedef struct {
+	uint32_t num_retimers;
+	uint32_t link_speed;
+	uint32_t link_status;  /* each bit corresponding to one link status */
+} opae_phy_info;
+
+typedef struct {
+	union {
+		uint64_t id;
+		struct {
+			uint8_t build_patch;
+			uint8_t build_minor;
+			uint8_t build_major;
+			uint8_t fvl_bypass:1;
+			uint8_t mac_lightweight:1;
+			uint8_t disagregate:1;
+			uint8_t lightweiht:1;
+			uint8_t seu:1;
+			uint8_t ptp:1;
+			uint8_t reserve:2;
+			uint16_t interface:4;
+			uint16_t afu_revision:12;
+			uint16_t patch:4;
+			uint16_t minor:4;
+			uint16_t major:4;
+			uint16_t reserved:4;
+		};
+	};
+} opae_bitstream_id;
+
+typedef struct {
+	union {
+		uint32_t version;
+		struct {
+			uint8_t micro;
+			uint8_t minor;
+			uint8_t major;
+			uint8_t board;
+		};
+	};
+} opae_bmc_version;
+
+#define OPAE_MAX_PORT_NUM   4
+
+#define OPAE_PROP_PCI   0x01
+#define OPAE_PROP_FME   0x02
+#define OPAE_PROP_PORT  0x04
+#define OPAE_PROP_BMC   0x08
+#define OPAE_PROP_ALL   \
+	(OPAE_PROP_PCI | OPAE_PROP_FME | OPAE_PROP_PORT | OPAE_PROP_BMC)
+
+typedef struct {
+	opae_pci_property pci;
+	opae_fme_property fme;
+	opae_port_property port[OPAE_MAX_PORT_NUM];
+	opae_bmc_property bmc;
+} opae_fpga_property;
+
+typedef struct {
+	uint64_t guid_h;
+	uint64_t guid_l;
+	uint32_t metadata_len;
+} gbs_header;
+
+#define OPAE_IMG_TYPE_BBS       0
+#define OPAE_IMG_TYPE_BMC       1
+#define OPAE_IMG_TYPE_GBS       2
+#define OPAE_IMG_TYPE(t)        ((t) & 0xff)
+
+#define OPAE_IMG_SUBTYPE_UPDATE             0
+#define OPAE_IMG_SUBTYPE_CANCELLATION       1
+#define OPAE_IMG_SUBTYPE_ROOT_KEY_HASH_256  2
+#define OPAE_IMG_SUBTYPE_ROOT_KEY_HASH_384  3
+#define OPAE_IMG_SUBTYPE(t)     (((t) >> 8) & 0xff)
+
+#define OPAE_IMG_BLK0_SIZE      128
+#define OPAE_IMG_BLK0_MAGIC     0xb6eafd19
+#define OPAE_IMG_BLK1_SIZE      896
+#define OPAE_IMG_HDR_SIZE   (OPAE_IMG_BLK0_SIZE + OPAE_IMG_BLK1_SIZE)
+#define OPAE_IMG_PL_MIN_SIZE    128
+
+typedef struct {
+	uint32_t magic;
+	uint32_t payload_len;
+	uint32_t payload_type;
+} opae_img_hdr;
+
+typedef struct {
+	int type;
+	int subtype;
+	uint32_t total_len;
+	uint32_t payload_offset;
+	uint32_t payload_len;
+} opae_img_info;
+
+void opae_get_api_version(opae_api_version *version);
+int opae_set_log_level(int level);
+int opae_set_log_file(char *path, int clean);
+int opae_get_proc_type(void);
+int opae_get_parent(pcidev_id id, pcidev_id parent);
+int opae_get_child(pcidev_id id, pcidev_id child, int size);
+int opae_get_pf1(pcidev_id id, pcidev_id peer, int size);
+int opae_init_eal(int argc, char **argv);
+int opae_cleanup_eal(void);
+int opae_enumerate(opae_pci_id *filter, pcidev_id list, int size);
+int opae_probe_device(pcidev_id id);
+int opae_remove_device(pcidev_id id);
+int opae_unbind_driver(pcidev_id id);
+int opae_bind_driver(pcidev_id id, char *drv_name);
+int opae_get_property(pcidev_id id, opae_fpga_property *prop, int type);
+int opae_get_phy_info(pcidev_id id, opae_phy_info *info);
+int opae_partial_reconfigure(pcidev_id id, int port, const char *gbs);
+int opae_get_image_info(const char *image, opae_img_info *info);
+int opae_cancel_flash_update(pcidev_id id, int force);
+int opae_update_flash(pcidev_id id, const char *image, uint64_t *status);
+int opae_reboot_device(pcidev_id id, int type, int page);
+int opae_store_rsu_status(pcidev_id id, uint32_t status, uint32_t progress);
+int opae_load_rsu_status(pcidev_id id, uint32_t *status, uint32_t *progress);
+int opae_read_pci_cfg(pcidev_id id, uint32_t address, uint32_t *value);
+int opae_write_pci_cfg(pcidev_id id, uint32_t address, uint32_t value);
+
+#ifdef __cplusplus
+}
+#endif
+
+
+#endif  /* _OPAE_API_H */
diff --git a/examples/meson.build b/examples/meson.build
index b9ab24223f..b029715277 100644
--- a/examples/meson.build
+++ b/examples/meson.build
@@ -16,7 +16,7 @@ all_examples = [
 	'eventdev_pipeline',
 	'fips_validation', 'flow_classify',
 	'flow_filtering', 'helloworld',
-	'ioat',
+	'ioat', 'ifpga',
 	'ip_fragmentation', 'ip_pipeline',
 	'ip_reassembly', 'ipsec-secgw',
 	'ipv4_multicast', 'kni',
-- 
2.29.2



More information about the dev mailing list