[PATCH v8 2/6] containers/inventory: Add inventory for container builder
Adam Hassick
ahassick at iol.unh.edu
Fri Aug 4 21:30:05 CEST 2023
The UNH lab no longer runs tests for this LTS version. It could be removed.
On Fri, Aug 4, 2023 at 9:58 AM Aaron Conole <aconole at redhat.com> wrote:
> Adam Hassick <ahassick at iol.unh.edu> writes:
>
> > From: Owen Hilyard <ohilyard at iol.unh.edu>
> >
> > Adds a yaml file used to define what containers should be built, what
> > packages the containers should have, what platforms to build for and how
> they
> > should be tagged.
> >
> > Also adds a JSON schema, which can be used to validate that the
> > inventory file is well-formed.
> >
> > Signed-off-by: Owen Hilyard <ohilyard at iol.unh.edu>
> > Signed-off-by: Adam Hassick <ahassick at iol.unh.edu>
> > ---
> > containers/template_engine/inventory.yaml | 470 ++++++++++++++++++
> > .../template_engine/inventory_schema.json | 221 ++++++++
> > 2 files changed, 691 insertions(+)
> > create mode 100644 containers/template_engine/inventory.yaml
> > create mode 100644 containers/template_engine/inventory_schema.json
> >
> > diff --git a/containers/template_engine/inventory.yaml
> b/containers/template_engine/inventory.yaml
> > new file mode 100644
> > index 0000000..31dbdb6
> > --- /dev/null
> > +++ b/containers/template_engine/inventory.yaml
> > @@ -0,0 +1,470 @@
> > +# SPDX-License-Identifier: BSD-3-Clause
> > +# Copyright (c) 2022 University of New Hampshire
> > +---
> > +abi:
> > + # versions should also include the abi versions for all stable ABIs.
> > + versions:
> > + # - dpdk_version: # LTS - 3
> > + # major: 20
> > + # minor: 11
> > + # revision: 0
> > + # git_tag: "20.11"
> > + # abi_version:
> > + # major: 21
> > + # minor: 0
>
> Why is this commented out?
>
> > + - dpdk_version: # LTS - 2
> > + major: 21
> > + minor: 11
> > + revision: 0
> > + git_tag: "21.11"
> > + abi_version:
> > + major: 22
> > + minor: 0
> > + - dpdk_version:
> > + major: 22 # LTS - 1
> > + minor: 11
> > + revision: 1
> > + git_tag: "tags/v22.11.1"
> > + abi_version:
> > + major: 23
> > + minor: 10 # .xml comparison
> > + - dpdk_version:
> > + major: 22 # LTS
> > + minor: 11
> > + revision: 1
> > + git_tag: "tags/v22.11.1"
> > + abi_version:
> > + major: 23
> > + minor: 11 # .so comparison
> > +
> > +dockerfiles:
> > + groups:
> > + all:
> > + package_manager_command: "NO_INSTALL_COMMAND_SET"
> > + parent: "NONE"
> > + packages:
> > + - gcc
> > + - make
> > + - git
> > + - sudo
> > + - curl
> > + - automake
> > + - autoconf
> > + - flex
> > + - bison
> > + - bc
> > + rpm:
> > + parent: "all"
> > + package_manager_command: yum install -y
> > + packages:
> > + - diffutils
> > + - pkg-config
> > + - python3
> > + - python3-pip
> > + - librdmacm
> > + - rdma-core-devel
> > + - libmnl-devel
> > + - ccache
> > + - zip
> > + - autoconf
> > + - libbpf-devel
> > + redhat_family:
> > + parent: "rpm"
> > + packages: []
> > + fedora:
> > + parent: "redhat_family"
> > + packages:
> > + - python3-pyelftools
> > + - zip
> > + - clang
> > + - ninja-build
> > + - elfutils-libelf-devel
> > + - jansson-devel
> > + - libatomic
> > + - libpcap-devel
> > + - numactl-devel
> > + - findutils
> > + - libtool
> > + - elfutils-devel
> > + - libxml2-devel
> > + - libabigail
> > + - openssl-devel
> > + fedora_coverity:
> > + parent: fedora
> > + packages: []
> > + fedora_clang:
> > + parent: fedora
> > + packages: []
> > + centos8:
> > + parent: "redhat_family"
> > + packages:
> > + - elfutils-libelf-devel
> > + - jansson-devel
> > + - libatomic
> > + - libpcap-devel
> > + - numactl-devel
> > + - libabigail
> > + - openssl-devel
> > + centos9:
> > + parent: "redhat_family"
> > + package_manager_command: "yum install --allowerasing -y"
> > + packages:
> > + - elfutils-libelf-devel
> > + - jansson-devel
> > + - libatomic
> > + - libpcap-devel
> > + - numactl-devel
> > + - libtool
> > + - g++
> > + - elfutils-devel
> > + - libxml2-devel
> > + - openssl-devel
> > + rhel:
> > + parent: "redhat_family"
> > + package_manager_command: "yum install -y"
> > + packages:
> > + - elfutils-libelf-devel
> > + - jansson-devel
> > + - libatomic
> > + - libpcap-devel
> > + - numactl-devel
> > + - openssl-devel
> > + - libabigail
> > + rhel7:
> > + parent: "rhel"
> > + packages:
> > + - ninja-build
> > + rhel8:
> > + parent: "rhel"
> > + package_manager_command: "yum --allowerasing install -y"
> > + packages:
> > + - ninja-build
> > + - libxml2-devel
> > + rhel9:
> > + parent: "rhel"
> > + package_manager_command: "yum --allowerasing install -y"
> > + packages:
> > + - libxml2-devel
> > + opensuse:
> > + parent: "rpm"
> > + package_manager_command: "zypper install -y"
> > + packages:
> > + - python3-pyelftools
> > + - ninja
> > + - gzip
> > + - libelf-devel
> > + - libjansson-devel
> > + - librdmacm1
> > + - libatomic1
> > + - libnuma-devel
> > + - libabigail-tools
> > + - tar
> > + - openssl-devel
> > + alpine:
> > + parent: "all"
> > + package_manager_command: "apk add"
> > + packages:
> > + - bsd-compat-headers
> > + - ccache
> > + - jansson-dev
> > + - libarchive-dev
> > + - libbpf-dev
> > + - libc-dev
> > + - libpcap-dev
> > + - linux-headers
> > + - ninja
> > + - numactl-dev
> > + - openssl-dev
> > + - py3-pip
> > + - py3-elftools
> > + - py3-setuptools
> > + - py3-wheel
> > + - zlib-dev
> > + - python3
> > + - bash
> > + - zip
> > + - meson
> > + arch:
> > + parent: "all"
> > + package_manager_command: "pacman -Syu --noconfirm"
> > + packages:
> > + - glibc
> > + - cmake
> > + - ninja
> > + - pandoc
> > + - python
> > + - python-docutils
> > + - ethtool
> > + - file
> > + - fakeroot
> > + - pkg-config
> > + - numactl
> > + - elfutils
> > + - gawk
> > + - diffutils
> > + - python-pip
> > + - libabigail
> > + - openssl
> > + - libbpf
> > + debian:
> > + parent: "all"
> > + package_manager_command: "apt-get update && apt-get install
> --no-install-recommends -y"
> > + packages:
> > + - libnuma-dev
> > + - zip
> > + - librdmacm1
> > + - libcrypto++-dev
> > + - python3-pyelftools
> > + - libxml2-dev
> > + - libelf-dev
> > + - libpcap-dev
> > + - libjansson-dev
> > + - python3-setuptools
> > + - autoconf
> > + - pkg-config
> > + - libtool
> > + - ccache
> > + - libatomic1
> > + - python3-wheel
> > + - libdw-dev
> > + - rdma-core
> > + - ninja-build
> > + - automake
> > + - librdmacm-dev
> > + - libmnl-dev
> > + - python3-pip
> > + - build-essential
> > + - aspell-en
> > + - parallel
> > + - python3-grpcio
> > + - lcov
> > + - libbpf-dev
> > + - libssl-dev
> > + debian10:
> > + parent: "debian"
> > + packages: []
> > + debian11:
> > + parent: "debian"
> > + packages:
> > + - nasm
> > + debian11_arm_ipsec:
> > + parent: "debian11"
> > + packages: []
> > + ubuntu:
> > + parent: 'debian'
> > + packages: []
> > + ubuntu18.04:
> > + parent: "ubuntu"
> > + packages: []
> > + ubuntu20.04:
> > + parent: "ubuntu"
> > + packages:
> > + - python3-grpcio
> > + - lcov
> > + - jq
> > + - default-jre
> > + - nodejs
> > + ubuntu22.04:
> > + parent: "ubuntu"
> > + packages:
> > + - python3-grpcio
> > + - lcov
> > + - abigail-tools
> > + - jq
> > + ubuntu_cross: # x86 -> ARM
> > + parent: "ubuntu20.04"
> > + packages:
> > + - clang
> > + - llvm
> > + - llvm-dev
> > + - llvm-runtime
> > + - lld
> > + - gcc-aarch64-linux-gnu
> > + - libgcc-9-dev-arm64-cross
> > + - libatomic1-arm64-cross
> > + - libc6-dev-arm64-cross
> > + - pkg-config-aarch64-linux-gnu
> > + - dpkg-dev
> > + - gcc-arm-linux-gnueabihf
> > + - libc6-dev-armhf-cross
> > + - pkg-config-arm-linux-gnueabihf
> > + ubuntu_sve:
> > + parent: "ubuntu20.04"
> > + packages:
> > + - gcc-10
> > + targets:
> > + # Alpine
> > + - name: alpine_compile
> > + group: "alpine"
> > + base_image: alpine
> > + platforms: [linux/amd64, linux/arm64]
> > + force_disable_abi: true
> > + minimum_dpdk_version:
> > + major: 22
> > + minor: 0
> > + revision: 0
> > + extra_information:
> > + amd64_environment_id: 45
> > + arm64_environment_id: 75
> > +
> > + # Arch
> > + - name: arch_compile
> > + group: "arch"
> > + base_image: archlinux:base
> > + platforms: [linux/amd64]
> > + extra_information:
> > + amd64_environment_id: 26
> > +
> > + - name: fedora37_compile
> > + group: "fedora"
> > + base_image: fedora:37
> > + platforms: [linux/amd64, linux/arm64]
> > + extra_information:
> > + amd64_environment_id: 69
> > + arm64_environment_id: 70
> > +
> > + - name: fedora38_compile
> > + group: "fedora"
> > + base_image: fedora:38
> > + platforms: [linux/amd64, linux/arm64]
> > + extra_information:
> > + amd64_environment_id: 71
> > + arm64_environment_id: 72
> > +
> > + - name: fedora38_clang
> > + group: "fedora_clang"
> > + base_image: fedora:38
> > + platforms: [linux/amd64, linux/arm64]
> > + extra_information:
> > + amd64_environment_id: 73
> > + arm64_environment_id: 74
> > +
> > + - name: fedora37_coverity
> > + group: "fedora_coverity"
> > + base_image: fedora:37
> > + force_disable_abi: true
> > + requires_coverity: true
> > + platforms: [linux/amd64]
> > +
> > + # CentOs
> > + - name: centosstream8_compile
> > + group: "centos8"
> > + base_image: quay.io/centos/centos:stream8
> > + platforms: [linux/amd64]
> > + extra_information:
> > + amd64_environment_id: 39
> > +
> > + - name: centosstream9_compile
> > + group: "centos9"
> > + base_image: quay.io/centos/centos:stream9
> > + platforms: [linux/amd64, linux/arm64]
> > + extra_information:
> > + amd64_environment_id: 53
> > + arm64_environment_id: 63
> > +
> > + # RHEL
> > + - name: rhel7_compile
> > + group: "rhel7"
> > + base_image: registry.access.redhat.com/ubi7/ubi
> > + platforms: [linux/amd64]
> > + extra_information:
> > + amd64_environment_id: 36
> > +
> > + - name: rhel8_compile
> > + group: "rhel8"
> > + base_image: registry.access.redhat.com/ubi8/ubi
> > + platforms: [linux/amd64]
> > + extra_information:
> > + amd64_environment_id: 37
> > +
> > + - name: rhel9_compile
> > + group: "rhel9"
> > + base_image: registry.access.redhat.com/ubi9/ubi
> > + platforms: [linux/amd64, linux/arm64]
> > + extra_information:
> > + amd64_environment_id: 76
> > + arm64_environment_id: 77
> > +
> > + # OpenSuse
> > + - name: opensuse-leap15_compile
> > + group: "opensuse"
> > + base_image: opensuse/leap:15
> > + platforms: [linux/amd64, linux/arm64]
> > + extra_information:
> > + amd64_environment_id: 25
> > + arm64_environment_id: 67
> > +
> > + # Debian
> > + - name: debian10_compile # debian 10
> > + group: "debian10"
> > + base_image: "debian:buster"
> > + libabigail_revision: libabigail-2.2
> > + platforms: [linux/amd64]
> > + extra_information:
> > + amd64_environment_id: 52
> > +
> > + - name: debian11_compile # debian 11
> > + group: "debian11"
> > + base_image: "debian:bullseye"
> > + libabigail_revision: libabigail-2.2
> > + platforms:
> > + - linux/amd64
> > + - linux/arm64
> > + extra_information:
> > + amd64_environment_id: 51
> > + arm64_environment_id: 64
> > +
> > + # Ubuntu
> > + - name: ubuntu20.04_compile
> > + group: "ubuntu20.04"
> > + base_image: "ubuntu:20.04"
> > + platforms: [linux/amd64, linux/arm64]
> > + extra_tags:
> > + - $R/ubuntu20.04_libabigail:latest
> > + - $R/ubuntu20.04_libabigail:$T
> > + - $R/ubuntu20.04_spell-check:latest
> > + - $R/ubuntu20.04_spell-check:$T
> > + - $R/ubuntu20.04_sonarscanner:latest
> > + - $R/ubuntu20.04_sonarscanner:$T
> > + extra_information:
> > + amd64_environment_id: 38
> > + arm64_environment_id: 62
> > +
> > + # x86 -> ARM cross compile
> > + - name: ubuntu20.04_cross
> > + group: "ubuntu_cross"
> > + base_image: "ubuntu:20.04"
> > + platforms: [linux/amd64]
> > + cross_file: "config/arm/arm64_armv8_linux_gcc"
> > + extra_information:
> > + amd64_environment_id: 47
> > +
> > + # Also, x86 -> ARM cross compile (for SVE test)
> > + - name: ubuntu20.04_sve_compile
> > + group: "ubuntu_sve"
> > + base_image: "arm64v8/ubuntu:20.04"
> > + platforms: [linux/arm64]
> > + extra_information:
> > + arm64_environment_id: 46
> > +
> > + # TEMPORARY: This has the forked IPsec lib developed by ARM. This
> causes issues on older LTS versions of DPDK.
> > + # Once all supported and tested LTS versions no longer break when
> this lib is installed, recombine with debian11_compile.
> > + - name: debian11_arm_ipsec
> > + group: "debian11_arm_ipsec"
> > + base_image: "debian:bullseye"
> > + platforms: [linux/arm64]
> > + force_disable_abi: true
> > + extra_tags:
> > + - $R/debian11_zuc_compile:latest
> > + - $R/debian11_snow3g_compile:latest
> > + - $R/debian11_zuc_compile:$T
> > + - $R/debian11_snow3g_compile:$T
> > + extra_information:
> > + arm64_environment_id: 58
> > +
> > + - name: ubuntu22.04_compile
> > + group: "ubuntu22.04"
> > + base_image: "ubuntu:22.04"
> > + platforms: [linux/amd64]
> > + extra_information:
> > + amd64_environment_id: 54
> > diff --git a/containers/template_engine/inventory_schema.json
> b/containers/template_engine/inventory_schema.json
> > new file mode 100644
> > index 0000000..7b79e44
> > --- /dev/null
> > +++ b/containers/template_engine/inventory_schema.json
> > @@ -0,0 +1,221 @@
> > +{
> > + "$schema": "https://json-schema.org/draft-07/schema",
> > + "title": "dpdklab-ci container inventory schema",
> > + "$comment": "SPDX-License-Identifier: BSD-3-Clause\nCopyright (c)
> 2022 University of New Hampshire",
> > + "type": "object",
> > + "properties": {
> > + "abi": {
> > + "type": "object",
> > + "properties": {
> > + "versions": {
> > + "type": "array",
> > + "items": {
> > + "type": "object",
> > + "properties": {
> > + "dpdk_version": {
> > + "type": "object",
> > + "properties": {
> > + "major": {
> > + "type": "integer",
> > + "description": "The major
> version number (usually the year of release)"
> > + },
> > + "minor": {
> > + "type": "integer",
> > + "description": "The minor
> version number (usually the month of release)"
> > + },
> > + "revision": {
> > + "type": "integer",
> > + "description": "The revision
> version number. Starts at 0."
> > + },
> > + "git_tag": {
> > + "type": "string",
> > + "description": "The git tag to
> check out to get this ABI version"
> > + }
> > + },
> > + "required": [
> > + "major",
> > + "minor",
> > + "revision",
> > + "git_tag"
> > + ],
> > + "additionalProperties": false
> > + },
> > + "abi_version": {
> > + "type": "object",
> > + "properties": {
> > + "major": {
> > + "type": "integer",
> > + "description": "The major
> version number (usually the year of release)"
> > + },
> > + "minor": {
> > + "type": "integer",
> > + "description": "The minor
> version number (usually the month of release)"
> > + }
> > + },
> > + "required": [
> > + "major",
> > + "minor"
> > + ],
> > + "additionalProperties": false
> > + }
> > + },
> > + "required": [
> > + "dpdk_version",
> > + "abi_version"
> > + ]
> > + },
> > + "minItems": 0
> > + }
> > + },
> > + "required": [
> > + "versions"
> > + ],
> > + "additionalProperties": false
> > + },
> > + "dockerfiles": {
> > + "type": "object",
> > + "properties": {
> > + "groups": {
> > + "type": "object",
> > + "patternProperties": {
> > + "^[a-z][a-z_0-9\\.]+$": {
> > + "type": "object",
> > + "properties": {
> > + "package_manager_command": {
> > + "type": "string",
> > + "description": "The command to
> install packages without user input."
> > + },
> > + "parent": {
> > + "type": "string",
> > + "description": "Which group to
> inherit from, or NONE if no parent exists"
> > + },
> > + "packages": {
> > + "type": "array",
> > + "description": "A list of package
> names to install",
> > + "items": {
> > + "type": "string"
> > + },
> > + "uniqueItems": true
> > + }
> > + },
> > + "required": [
> > + "parent",
> > + "packages"
> > + ],
> > + "additionalProperties": false
> > + }
> > + },
> > + "minProperties": 1
> > + },
> > + "targets": {
> > + "type": "array",
> > + "items": {
> > + "type": "object",
> > + "properties": {
> > + "name": {
> > + "type": "string",
> > + "description": "The name of the docker
> container"
> > + },
> > + "group": {
> > + "type": "string",
> > + "description": "Which group the target
> is a member of"
> > + },
> > + "base_image": {
> > + "type": "string",
> > + "description": "The base docker image."
> > + },
> > + "platforms": {
> > + "type": "array",
> > + "description": "What platforms to build
> for",
> > + "items": {
> > + "type": "string",
> > + "enum": [
> > + "linux/amd64",
> > + "linux/arm64",
> > + "linux/riscv64",
> > + "linux/ppc64le",
> > + "linux/arm/v7",
> > + "linux/arm/v6"
> > + ]
> > + },
> > + "minItems": 1
> > + },
> > + "cross_file": {
> > + "type": "string",
> > + "description": "A cross file to compile
> meson with. If not present, meson's automatic behavior will be used."
> > + },
> > + "libabigail_revision": {
> > + "type": "string",
> > + "description": "A revision or branch of
> libabigail to check out before building for a given target."
> > + },
> > + "force_disable_abi": {
> > + "type": "boolean",
> > + "description": "Whether to forcibly
> disable ABI for this target. This should be used as a temporary measure
> until a fix can be created.",
> > + "default": "false"
> > + },
> > + "requires_coverity": {
> > + "type": "boolean",
> > + "description": "Whether this target
> requires the Coverity Scan binaries.",
> > + "default": "false"
> > + },
> > + "minimum_dpdk_version": {
> > + "type": "object",
> > + "properties": {
> > + "major": {
> > + "type": "integer",
> > + "description": "The major
> version number (usually the year of release)"
> > + },
> > + "minor": {
> > + "type": "integer",
> > + "description": "The minor
> version number (usually the month of release)"
> > + },
> > + "revision": {
> > + "type": "integer",
> > + "description": "The revision
> version number. Starts at 0."
> > + }
> > + },
> > + "required": [
> > + "major",
> > + "minor",
> > + "revision"
> > + ],
> > + "additionalProperties": false
> > + },
> > + "extra_tags": {
> > + "type": "array",
> > + "description": "Tags to apply to this
> image",
> > + "items": {
> > + "type": "string",
> > + "description": "$R will be replaced
> with the 'registry_hostname' variable in this file. $T will be replaced
> with a timestamp in the form of '%Y-%m-%d'. $N will be replaced by the name
> of the target.. The tags '$R/$N:latest' and '$R/$N:$T' will always be
> present"
> > + }
> > + },
> > + "extra_information": {
> > + "type": "object",
> > + "description": "Extra information to be
> stored as a JSON file at /container_info.json",
> > + "additionalProperties": true
> > + }
> > + },
> > + "required": [
> > + "name",
> > + "group",
> > + "base_image",
> > + "platforms"
> > + ],
> > + "additionalProperties": false
> > + },
> > + "minItems": 1
> > + }
> > + },
> > + "required": [
> > + "groups",
> > + "targets"
> > + ],
> > + "additionalProperties": false
> > + }
> > + },
> > + "required": [
> > + "abi",
> > + "dockerfiles"
> > + ],
> > + "additionalProperties": false
> > +}
> > \ No newline at end of file
>
>
--
*Adam Hassick*
Senior Developer
UNH InterOperability Lab
ahassick at iol.unh.edu
iol.unh.edu <https://www.iol.unh.edu/>
+1 (603) 475-8248
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mails.dpdk.org/archives/ci/attachments/20230804/f82ff2a2/attachment-0001.htm>
More information about the ci
mailing list