[dpdk-dev] [PATCH v4 00/10] VM Power Management

Liu, Yong yong.liu at intel.com
Mon Oct 13 08:17:50 CEST 2014
Previous message: [dpdk-dev] [PATCH v4 10/10] VM Power Management Unit Tests
Next message: [dpdk-dev] [PATCH v4 00/10] VM Power Management
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Patch name:		VM Power Management
Brief description:	Verify VM power management in virtualized environments
Test Flag:		Tested-by 
Tester name:		yong.liu at intel.com
Test environment:
			OS: Fedora20 3.11.10-301.fc20.x86_64
			GCC: gcc version 4.8.3 20140911
			CPU: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz
			NIC: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb]
Test Tool Chain information:	
			Qemu: 1.6.1
			libvirt: 1.1.3
			Guest OS: Fedora20 3.11.10-301.fc20.x86_64
			Guest GCC: gcc version 4.8.3 20140624
			
Commit ID:		72d3e7ad3183f42f8b9fb3bb1c12b3e1b39eef39

Detailed Testing information	
DPDK SW Configuration:
			Default x86_64-native-linuxapp-gcc configuration
Test Result Summary: 	Total 7 cases, 7 passed, 0 failed

Test Case - name:
			VM Power Management Channel
Test Case - Description:
			Check vm power management communication channels can successfully connected
Test Case -command / instruction:
			Create folder in system temporary filesystem for power monitor socket
				mkdir -p /tmp/powermonitor
					
			Configure VM XML and pin VCPUs to specified CPUs
				<vcpu placement='static'>5</vcpu>
				<cputune>
					 <vcpupin vcpu='0' cpuset='1'/>
					 <vcpupin vcpu='1' cpuset='2'/>
					  <vcpupin vcpu='2' cpuset='3'/>
					  <vcpupin vcpu='3' cpuset='4'/>
					  <vcpupin vcpu='4' cpuset='5'/>
				</cputune>
						
			Configure VM XML to set up virtio serial ports
				<channel type='unix'>
				<source mode='bind' path='/tmp/powermonitor/<vm_name>.<channel_num>'/>
				<target type='virtio' name='virtio.serial.port.poweragent.<channel_num>'/>
				<address type='virtio-serial' controller='0' bus='0' port='4'/>
				</channel>

       			Run power-manager monitor in Host
				./build/vm_power_mgr -c 0x3 -n 4
			
			Startup VM and run guest_vm_power_mgr
				guest_vm_power_mgr -c 0x1f -n 4 -- -i
			
			Add vm in host and check vm_power_mgr can get frequency normally

				vmpower> add_vm <vm_name>
				vmpower> add_channels <vm_name> all
				vmpower> get_cpu_freq <core_num>
				
			Check vcpu/cpu mapping can be detected normally
				vmpower> show_vm <vm_name>
		
Test Case - expected test result:
			VM power management communication channels can sucessfully connected and host can get vm core information

Test Case - name:
			VM Power Management Numa
Test Case - Description:
			Check vm power management support manage cores in different sockets
Test Case -command / instruction:
			Get core and socket information by cpu_layout
   
				./tools/cpu_layout.py

			Configure VM XML to pin VCPUs on Socket1:

			Repeat Case1
		
			Check vcpu/cpu mapping can be detected normally
				vmpower> show_vm <vm_name>
			
Test Case - expected test result:
			VM power management communication channels can sucessfully connected and show correct vm core information

Test Case - name:
			VM scale cpu frequency down 
Test Case - Description:
			Check vm power management support VM configure self cores frequency down
Test Case -command / instruction:
			Setup VM power management environment
			
			Send cpu frequency down hints to Host 
			
				vmpower(guest)> set_cpu_freq 0 down
			
			Verify the frequency of physical CPU has been scaled down correctly
				vmpower> get_cpu_freq 1
				Core 1 frequency: 2700000

			Check other CPUs' frequency is not affected by actions above
			
			Check if the other VM works fine (if they use different CPUs)
			
			Repeat above actions several times
			
Test Case - expected test result:
			Frequency for VM's core can be scaling down normally
			
Test Case - name:
			VM scale cpu frequency up 
Test Case - Description:
			Check vm power management support VM configure self cores frequency up
Test Case -command / instruction:
			Setup VM power management environment
			
			Send cpu frequency up hints to Host 

				vmpower(guest)> set_cpu_freq 0 up
			
			Verify the frequency of physical CPU has been scaled up correctly
				vmpower> get_cpu_freq 1

			Check other CPUs' frequency is not affected by actions above
			
			Check if the other VM works fine (if they use different CPUs)
			
			Repeat above actions several times
			
Test Case - expected test result:
			Frequency for VM's core can be scaling up normally
			
Test Case - name:
			VM Scale CPU Frequency to Min 
Test Case - Description:
			Check vm power management support VM configure self cores frequency to minimum
Test Case -command / instruction:
			Setup VM power management environment
			Send cpu frequency scale to minimum hints
			
				vmpower(guest)> set_cpu_freq 0 min

			Verify the frequency of physical CPU has been scale to min correctly

				vmpower> get_cpu_freq 1
				Core 1 frequency: 1200000

			Check other CPUs' frequency is not affected by actions above
			check if the other VM works fine (if they use different CPUs)
			
Test Case - expected test result:
			Frequency for VM's core can be scaling to minimum normally
			
Test Case - name:
			VM Scale CPU Frequency to Max
Test Case - Description:
			Check vm power management support VM configure self cores frequency to maximum
Test Case -command / instruction:
			Setup VM power management environment
			Send cpu frequency scale to maximum hints
			
				vmpower(guest)> set_cpu_freq 0 max

			Verify the frequency of physical CPU has been scale to max correctly

				vmpower> get_cpu_freq 1
				Core 1 frequency: 2800000
	
	
			Check other CPUs' frequency is not affected by actions above
			check if the other VM works fine (if they use different CPUs)
			
Test Case - expected test result:
			Frequency for VM's core can be scaling to maximum normally
			
Test Case - name:
			VM Power Management Multi VMs
Test Case - Description:
			Check vm power management support multiple VMs
Test Case -command / instruction:
			Setup VM power management environment for VM1

			Setup VM power management environment for VM2

			Run power-manager in Host

				./build/vm_power_mgr -c 0x3 -n 4

			Startup VM1 and VM2

			Add VM1 in host and check vm_power_mgr can get frequency normally

				vmpower> add_vm <vm1_name>
				vmpower> add_channels <vm1_name> all
				vmpower> get_cpu_freq <core_num>

			Add VM2 in host and check vm_power_mgr can get frequency normally

				vmpower> add_vm <vm2_name>
				vmpower> add_channels <vm2_name> all
				vmpower> get_cpu_freq <core_num>		

			Check VM1 and VM2 cpu frequency can by modified by guest_cli

			Poweroff VM2 and remove VM2 from host vm_power_mgr
				vmpower> rm_vm <vm2_name>

Test Case - expected test result:
			VM power management	supported multiple VMs add and remove

> -----Original Message-----
> From: Carew, Alan
> Sent: Monday, October 13, 2014 3:36 AM
> To: dev at dpdk.org
> Cc: Liu, Yong
> Subject: [PATCH v4 00/10] VM Power Management
> 
> Virtual Machine Power Management.
> 
> The following patches add two DPDK sample applications and an alternate
> implementation of librte_power for use in virtualized environments.
> The idea is to provide librte_power functionality from within a VM to address
> the lack of MSRs to facilitate frequency changes from within a VM.
> It is ideally suited for Haswell which provides per core frequency scaling.
> 
> The current librte_power affects frequency changes via the acpi-cpufreq
> 'userspace' power governor, accessed via sysfs.
> 
> General Overview:(more information in each patch that follows).
> The VM Power Management solution provides two components:
> 
>  1)VM: Allows for the a DPDK application in a VM to reuse the librte_power
>  interface. Each lcore opens a Virto-Serial endpoint channel to the host,
>  where the re-implementation of librte_power simply forwards the requests
> for
>  frequency change to a host based monitor. The host monitor itself uses
>  librte_power.
>  Each lcore channel corresponds to a
>  serial device '/dev/virtio-ports/virtio.serial.port.poweragent.<lcore_num>'
>  which is opened in non-blocking mode.
>  While each Virtual CPU can be mapped to multiple physical CPUs it is
>  recommended that each vCPU should be mapped to a single core only.
> 
>  2)Host: The host monitor is managed by a CLI, it allows for adding
> qemu/KVM
>  virtual machines and associated channels to the monitor, manually changing
>  CPU frequency, inspecting the state of VMs, vCPU to pCPU pinning and
> managing
>  channels.
>  Host channel endpoints are Virto-Serial endpoints configured as AF_UNIX
> file
>  sockets which follow a specific naming convention
>  i.e /tmp/powermonitor/<vm_name>.<channel_number>,
>  each channel has an 1:1 mapping to a VM endpoint
>  i.e. /dev/virtio-ports/virtio.serial.port.poweragent.<lcore_num>
>  Host channel endpoints are opened in non-blocking mode and are
> monitored via epoll.
>  Requests over each channel to change frequency are forwarded to the
> original
>  librte_power.
> 
> Channels must be manually configured as qemu-kvm command line
> arguments or
> libvirt domain definition(xml) e.g.
> <controller type='virtio-serial' index='0'>
>  <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
> </controller>
> <channel type='unix'>
>   <source mode='bind'
> path='/tmp/powermonitor/<vm_name>.<channel_num>'/>
>   <target type='virtio' name='virtio.serial.port.poweragent.<channel_num>/>
>   <address type='virtio-serial' controller='0' bus='0' port='<N>'/>
> </channel>
> 
> Where multiple channels can be configured by specifying multiple <channel>
> elements, by replacing <vm_name>, <channel_num>.
> <N>(port number) should be incremented by 1 for each new channel
> element.
> More information on Virtio-Serial can be found here:
> http://fedoraproject.org/wiki/Features/VirtioSerial
> To enable the Hypervisor creation of channels, the host endpoint directory
> must be created with qemu permissions:
> mkdir /tmp/powermonitor
> chown qemu:qemu /tmp/powermonitor
> 
> The host application runs on two separate lcores:
> Core N) CLI: For management of Virtual Machines adding channels to
> Monitor thread,
>  inspecting state and manually setting CPU frequency [PATCH 02/09]
> Core N+1) Monitor Thread: An epoll based infinite loop that waits on channel
> events
>  from VMs and calls the corresponding librte_power functions.
> 
> A sample application is also provided to run on Virtual Machines, this
> application provides a CLI to manually set the frequency of a
> vCPU[PATCH 08/09]
> 
> The current l3fwd-power sample application can also be run on a VM.
> 
> Changes in V4:
>  Fixed double free of channel during VM shutdown.
> 
> Changes in V3:
>  Fixed crash in Guest CLI when host application is not running.
>  Renamed #defines to be more specific to the module they belong
>  Added vCPU pinning via CLI
> 
> Changes in V2:
>  Runtime selection of librte_power implementations.
>  Updated Unit tests to cover librte_power changes.
>  PATCH[0/3] was sent twice, again as PATCH[0/4]
>  Miscellaneous fixes.
> 
> Alan Carew (10):
>   Channel Manager and Monitor for VM Power Management(Host).
>   VM Power Management CLI(Host).
>   CPU Frequency Power Management(Host).
>   VM Power Management application and Makefile.
>   VM Power Management CLI(Guest).
>   VM communication channels for VM Power Management(Guest).
>   librte_power common interface for Guest and Host
>   Packet format for VM Power Management(Host and Guest).
>   Build system integration for VM Power Management(Guest and Host)
>   VM Power Management Unit Tests
> 
>  app/test/Makefile                                  |   3 +-
>  app/test/autotest_data.py                          |  26 +
>  app/test/test_power.c                              | 445 +-----------
>  app/test/test_power_acpi_cpufreq.c                 | 544 ++++++++++++++
>  app/test/test_power_kvm_vm.c                       | 308 ++++++++
>  examples/vm_power_manager/Makefile                 |  57 ++
>  examples/vm_power_manager/channel_manager.c        | 804
> +++++++++++++++++++++
>  examples/vm_power_manager/channel_manager.h        | 314 ++++++++
>  examples/vm_power_manager/channel_monitor.c        | 231 ++++++
>  examples/vm_power_manager/channel_monitor.h        | 102 +++
>  examples/vm_power_manager/guest_cli/Makefile       |  56 ++
>  examples/vm_power_manager/guest_cli/main.c         |  87 +++
>  examples/vm_power_manager/guest_cli/main.h         |  52 ++
>  .../guest_cli/vm_power_cli_guest.c                 | 155 ++++
>  .../guest_cli/vm_power_cli_guest.h                 |  55 ++
>  examples/vm_power_manager/main.c                   | 117 +++
>  examples/vm_power_manager/main.h                   |  52 ++
>  examples/vm_power_manager/power_manager.c          | 244 +++++++
>  examples/vm_power_manager/power_manager.h          | 188 +++++
>  examples/vm_power_manager/vm_power_cli.c           | 669
> +++++++++++++++++
>  examples/vm_power_manager/vm_power_cli.h           |  47 ++
>  lib/librte_power/Makefile                          |   3 +-
>  lib/librte_power/channel_commands.h                |  77 ++
>  lib/librte_power/guest_channel.c                   | 162 +++++
>  lib/librte_power/guest_channel.h                   |  89 +++
>  lib/librte_power/rte_power.c                       | 540 ++------------
>  lib/librte_power/rte_power.h                       | 120 ++-
>  lib/librte_power/rte_power_acpi_cpufreq.c          | 545 ++++++++++++++
>  lib/librte_power/rte_power_acpi_cpufreq.h          | 192 +++++
>  lib/librte_power/rte_power_common.h                |  39 +
>  lib/librte_power/rte_power_kvm_vm.c                | 135 ++++
>  lib/librte_power/rte_power_kvm_vm.h                | 179 +++++
>  32 files changed, 5725 insertions(+), 912 deletions(-)
>  create mode 100644 app/test/test_power_acpi_cpufreq.c
>  create mode 100644 app/test/test_power_kvm_vm.c
>  create mode 100644 examples/vm_power_manager/Makefile
>  create mode 100644 examples/vm_power_manager/channel_manager.c
>  create mode 100644 examples/vm_power_manager/channel_manager.h
>  create mode 100644 examples/vm_power_manager/channel_monitor.c
>  create mode 100644 examples/vm_power_manager/channel_monitor.h
>  create mode 100644 examples/vm_power_manager/guest_cli/Makefile
>  create mode 100644 examples/vm_power_manager/guest_cli/main.c
>  create mode 100644 examples/vm_power_manager/guest_cli/main.h
>  create mode 100644
> examples/vm_power_manager/guest_cli/vm_power_cli_guest.c
>  create mode 100644
> examples/vm_power_manager/guest_cli/vm_power_cli_guest.h
>  create mode 100644 examples/vm_power_manager/main.c
>  create mode 100644 examples/vm_power_manager/main.h
>  create mode 100644 examples/vm_power_manager/power_manager.c
>  create mode 100644 examples/vm_power_manager/power_manager.h
>  create mode 100644 examples/vm_power_manager/vm_power_cli.c
>  create mode 100644 examples/vm_power_manager/vm_power_cli.h
>  create mode 100644 lib/librte_power/channel_commands.h
>  create mode 100644 lib/librte_power/guest_channel.c
>  create mode 100644 lib/librte_power/guest_channel.h
>  create mode 100644 lib/librte_power/rte_power_acpi_cpufreq.c
>  create mode 100644 lib/librte_power/rte_power_acpi_cpufreq.h
>  create mode 100644 lib/librte_power/rte_power_common.h
>  create mode 100644 lib/librte_power/rte_power_kvm_vm.c
>  create mode 100644 lib/librte_power/rte_power_kvm_vm.h
> 
> --
> 1.9.3
Previous message: [dpdk-dev] [PATCH v4 10/10] VM Power Management Unit Tests
Next message: [dpdk-dev] [PATCH v4 00/10] VM Power Management
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
More information about the dev mailing list