[dpdk-dev] [PATCH v3 00/13] Implement compression API

[Pablo de Lara]

With the vast amounts of data being transported around networks
and stored in storage systems, reducing data size is becoming ever more important.

There are both software libraries and hardware devices available that
provide compression, but no common API.
Such an API is proposed in this patchset, which supports the following features:

- Deflate Algorithm (https://tools.ietf.org/html/rfc1951)
- LZS algorithm (https://tools.ietf.org/html/rfc2395)
- Static and Dynamic Huffman encoding.
- Compression levels
- Checksum generation
- Asynchronous burst API
- private_xform - a place for PMDs to hold private data derived from
  a xform and used by stateless operations.
- stream - a place for PMDs to hold private data derived from
  a xform and also maintain state and history data. For
  stateful flows.

Signed-off-by: Fiona Trahe <fiona.trahe at intel.com>
Signed-off-by: Pablo de Lara pablo.de.lara.guarch at intel.com
Signed-off-by: Shally Verma <shally.verma at caviumnetworks.com>
Signed-off-by: Ashish Gupta <ashish.gupta at caviumnetworks.com>

Opens:
 - creation of private_xform and stream mempools. In v3, config
   added to give PMD the size params needed to create mempools.
   Still open as to whether the pools should be created by
   application, in API layer or by PMDs. Expect to resolve in a v4. 
 - alternative to mbufs for passing data between application
   and accelerator. A lot of varied opinions on this, expect to
   address in a future release.
 - addition of feature to add capability to PMD to allow more than one
   inflight operation from a stateful stream - to be added by NXP 

Changes in v3:
 - Remove rte_comp_op_ctod helper functions
 - Remove param_range_check macro
 - Rename from phys_addr to iova_addr
 - Remove rte_comp_algo_strings
 - Rename rte_compressdev_pmd_is_valid_dev to rte_compressdev_is_valid_dev
 - Remove feature flags from compressdev
 - Support hash operations
 - Add shareable priv xform in feature flags, instead of returnin it
   on xform creation
 - Allow max number of queue pairs to be 0, meaning that there is no
   limit.
 - Add invalid configuration checks
 - Add capability helper functions

Changes in v2:
 - Add stream APIs
 - Remove session
 - Add SHAREABLE / NON_SHAREABLE private_xform types
 - Add algo enum 'UNSPECIFIED' to fix warning in capabilities
 - Change one remaining log to use dynamic logging.
 - Add rte_cache_aligned keyword to op
 - Rename enums with better names _ALGO, __CHECKSUM, _HUFFMAN_
 - Use const keyword when passing xform
 - Remove qp_count fn from dev_ops as never used
 - Remove max_nb_queue-pairs from compressdev_init_param as never used
 - Clarify device configure and start sequence
 - Replace OUT_OF_SPACE with OUT_OF_SPACE_RECOVERABLE and TERMINATED
   and clarified usage. 
 - Add stream and private_xform sizes to device config for use in
   mempool creation
 - Add capability helper fn
 - Use Base2 log value for window size on xforms
 - Add Meson build
 - Update MAINTAINERS
 - Update Map file
 - Change order in doxy file
 - Update Release note 

Fiona Trahe (12):
  compressdev: add basic device management
  compressdev: add queue pair management
  compressdev: add compression specific data
  compressdev: add enqueue/dequeue functions
  compressdev: add operation management
  compressdev: support stateless operations
  compressdev: support stateful operations
  compressdev: add device feature flags
  compressdev: add compression service feature flags
  compressdev: add device stats
  compressdev: add device capabilities
  compressdev: get device id from name

Shally Verma (1):
  compressdev: support hash operations

 MAINTAINERS                                        |   7 +
 config/common_base                                 |   6 +
 config/rte_config.h                                |   3 +
 doc/api/doxy-api-index.md                          |   1 +
 doc/api/doxy-api.conf                              |   1 +
 doc/guides/rel_notes/release_18_05.rst             |   6 +
 lib/Makefile                                       |   3 +
 lib/librte_compressdev/Makefile                    |  29 +
 lib/librte_compressdev/meson.build                 |   9 +
 lib/librte_compressdev/rte_comp.h                  | 539 ++++++++++++
 lib/librte_compressdev/rte_compressdev.c           | 902 +++++++++++++++++++++
 lib/librte_compressdev/rte_compressdev.h           | 701 ++++++++++++++++
 lib/librte_compressdev/rte_compressdev_pmd.c       | 157 ++++
 lib/librte_compressdev/rte_compressdev_pmd.h       | 437 ++++++++++
 lib/librte_compressdev/rte_compressdev_version.map |  44 +
 lib/meson.build                                    |   2 +-
 mk/rte.app.mk                                      |   1 +
 17 files changed, 2847 insertions(+), 1 deletion(-)
 create mode 100644 lib/librte_compressdev/Makefile
 create mode 100644 lib/librte_compressdev/meson.build
 create mode 100644 lib/librte_compressdev/rte_comp.h
 create mode 100644 lib/librte_compressdev/rte_compressdev.c
 create mode 100644 lib/librte_compressdev/rte_compressdev.h
 create mode 100644 lib/librte_compressdev/rte_compressdev_pmd.c
 create mode 100644 lib/librte_compressdev/rte_compressdev_pmd.h
 create mode 100644 lib/librte_compressdev/rte_compressdev_version.map

-- 
2.14.3