[dpdk-dev] [PATCH v6 00/10] Virtual Machine Power Management

Pablo de Lara pablo.de.lara.guarch at intel.com
Tue Nov 25 17:18:01 CET 2014


Virtual Machine Power Management.

The following patches add two DPDK sample applications and an alternate
implementation of librte_power for use in virtualized environments.
The idea is to provide librte_power functionality from within a VM to address
the lack of MSRs to facilitate frequency changes from within a VM.
It is ideally suited for Haswell which provides per core frequency scaling.

The current librte_power affects frequency changes via the acpi-cpufreq
'userspace' power governor, accessed via sysfs.

General Overview:(more information in each patch that follows).
The VM Power Management solution provides two components:

 1)VM: Allows for the a DPDK application in a VM to reuse the librte_power
 interface. Each lcore opens a Virto-Serial endpoint channel to the host,
 where the re-implementation of librte_power simply forwards the requests for
 frequency change to a host based monitor. The host monitor itself uses
 librte_power.
 Each lcore channel corresponds to a
 serial device '/dev/virtio-ports/virtio.serial.port.poweragent.<lcore_num>'
 which is opened in non-blocking mode.
 While each Virtual CPU can be mapped to multiple physical CPUs it is
 recommended that each vCPU should be mapped to a single core only.

 2)Host: The host monitor is managed by a CLI, it allows for adding qemu/KVM
 virtual machines and associated channels to the monitor, manually changing
 CPU frequency, inspecting the state of VMs, vCPU to pCPU pinning and managing
 channels.
 Host channel endpoints are Virto-Serial endpoints configured as AF_UNIX file
 sockets which follow a specific naming convention
 i.e /tmp/powermonitor/<vm_name>.<channel_number>,
 each channel has an 1:1 mapping to a VM endpoint
 i.e. /dev/virtio-ports/virtio.serial.port.poweragent.<lcore_num>
 Host channel endpoints are opened in non-blocking mode and are monitored via epoll.
 Requests over each channel to change frequency are forwarded to the original
 librte_power.
 
Channels must be manually configured as qemu-kvm command line arguments or
libvirt domain definition(xml) e.g.
<controller type='virtio-serial' index='0'>
 <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
</controller>
<channel type='unix'>
  <source mode='bind' path='/tmp/powermonitor/<vm_name>.<channel_num>'/>
  <target type='virtio' name='virtio.serial.port.poweragent.<channel_num>/>
  <address type='virtio-serial' controller='0' bus='0' port='<N>'/>
</channel>

Where multiple channels can be configured by specifying multiple <channel>
elements, by replacing <vm_name>, <channel_num>.
<N>(port number) should be incremented by 1 for each new channel element.
More information on Virtio-Serial can be found here:
http://fedoraproject.org/wiki/Features/VirtioSerial
To enable the Hypervisor creation of channels, the host endpoint directory
must be created with qemu permissions:
mkdir /tmp/powermonitor
chown qemu:qemu /tmp/powermonitor

The host application runs on two separate lcores:
Core N) CLI: For management of Virtual Machines adding channels to Monitor thread,
 inspecting state and manually setting CPU frequency [PATCH 02/09]
Core N+1) Monitor Thread: An epoll based infinite loop that waits on channel events
 from VMs and calls the corresponding librte_power functions.

A sample application is also provided to run on Virtual Machines, this
application provides a CLI to manually set the frequency of a 
vCPU[PATCH 08/09]

The current l3fwd-power sample application can also be run on a VM.

Changes in V6:
 Fixed typos and missing some identations and blank lines

Changes in V5:
 Fixed default target in sample app Makefiles

Changes in V4:
 Fixed double free of channel during VM shutdown.

Changes in V3:
 Fixed crash in Guest CLI when host application is not running.
 Renamed #defines to be more specific to the module they belong
 Added vCPU pinning via CLI

Changes in V2:
 Runtime selection of librte_power implementations.
 Updated Unit tests to cover librte_power changes.
 PATCH[0/3] was sent twice, again as PATCH[0/4]
 Miscellaneous fixes.

Alan Carew (10):
  Channel Manager and Monitor for VM Power Management(Host).
  VM Power Management CLI(Host).
  CPU Frequency Power Management(Host).
  VM Power Management application and Makefile.
  VM Power Management CLI(Guest).
  VM communication channels for VM Power Management(Guest).
  librte_power common interface for Guest and Host
  Packet format for VM Power Management(Host and Guest).
  Build system integration for VM Power Management(Guest and Host)
  VM Power Management Unit Tests

 app/test/Makefile                                  |    3 +-
 app/test/autotest_data.py                          |   26 +
 app/test/test_power.c                              |  445 +----------
 app/test/test_power_acpi_cpufreq.c                 |  544 +++++++++++++
 app/test/test_power_kvm_vm.c                       |  308 ++++++++
 examples/vm_power_manager/Makefile                 |   57 ++
 examples/vm_power_manager/channel_manager.c        |  808 ++++++++++++++++++++
 examples/vm_power_manager/channel_manager.h        |  314 ++++++++
 examples/vm_power_manager/channel_monitor.c        |  234 ++++++
 examples/vm_power_manager/channel_monitor.h        |  102 +++
 examples/vm_power_manager/guest_cli/Makefile       |   56 ++
 examples/vm_power_manager/guest_cli/main.c         |   88 +++
 examples/vm_power_manager/guest_cli/main.h         |   52 ++
 .../guest_cli/vm_power_cli_guest.c                 |  156 ++++
 .../guest_cli/vm_power_cli_guest.h                 |   55 ++
 examples/vm_power_manager/main.c                   |  117 +++
 examples/vm_power_manager/main.h                   |   52 ++
 examples/vm_power_manager/power_manager.c          |  253 ++++++
 examples/vm_power_manager/power_manager.h          |  188 +++++
 examples/vm_power_manager/vm_power_cli.c           |  673 ++++++++++++++++
 examples/vm_power_manager/vm_power_cli.h           |   47 ++
 lib/librte_power/Makefile                          |    3 +-
 lib/librte_power/channel_commands.h                |   77 ++
 lib/librte_power/guest_channel.c                   |  162 ++++
 lib/librte_power/guest_channel.h                   |   89 +++
 lib/librte_power/rte_power.c                       |  540 ++------------
 lib/librte_power/rte_power.h                       |  120 +++-
 lib/librte_power/rte_power_acpi_cpufreq.c          |  545 +++++++++++++
 lib/librte_power/rte_power_acpi_cpufreq.h          |  192 +++++
 lib/librte_power/rte_power_common.h                |   39 +
 lib/librte_power/rte_power_kvm_vm.c                |  136 ++++
 lib/librte_power/rte_power_kvm_vm.h                |  179 +++++
 32 files changed, 5748 insertions(+), 912 deletions(-)
 create mode 100644 app/test/test_power_acpi_cpufreq.c
 create mode 100644 app/test/test_power_kvm_vm.c
 create mode 100644 examples/vm_power_manager/Makefile
 create mode 100644 examples/vm_power_manager/channel_manager.c
 create mode 100644 examples/vm_power_manager/channel_manager.h
 create mode 100644 examples/vm_power_manager/channel_monitor.c
 create mode 100644 examples/vm_power_manager/channel_monitor.h
 create mode 100644 examples/vm_power_manager/guest_cli/Makefile
 create mode 100644 examples/vm_power_manager/guest_cli/main.c
 create mode 100644 examples/vm_power_manager/guest_cli/main.h
 create mode 100644 examples/vm_power_manager/guest_cli/vm_power_cli_guest.c
 create mode 100644 examples/vm_power_manager/guest_cli/vm_power_cli_guest.h
 create mode 100644 examples/vm_power_manager/main.c
 create mode 100644 examples/vm_power_manager/main.h
 create mode 100644 examples/vm_power_manager/power_manager.c
 create mode 100644 examples/vm_power_manager/power_manager.h
 create mode 100644 examples/vm_power_manager/vm_power_cli.c
 create mode 100644 examples/vm_power_manager/vm_power_cli.h
 create mode 100644 lib/librte_power/channel_commands.h
 create mode 100644 lib/librte_power/guest_channel.c
 create mode 100644 lib/librte_power/guest_channel.h
 create mode 100644 lib/librte_power/rte_power_acpi_cpufreq.c
 create mode 100644 lib/librte_power/rte_power_acpi_cpufreq.h
 create mode 100644 lib/librte_power/rte_power_common.h
 create mode 100644 lib/librte_power/rte_power_kvm_vm.c
 create mode 100644 lib/librte_power/rte_power_kvm_vm.h

-- 
1.7.4.1



More information about the dev mailing list