[dpdk-dev] [PATCH v6 3/4] test: add test case to validate VFIO DMA map/unmap

David Christensen drc at linux.vnet.ibm.com
Wed Jan 6 22:20:44 CET 2021



On 1/6/21 12:40 AM, Nithin Dabilpuram wrote:
> On Tue, Jan 05, 2021 at 11:33:20AM -0800, David Christensen wrote:
>> Hey Nithin,
>>
>>>> +static int
>>>> +test_memory_vfio_dma_map(void)
>>>> +{
>>>> +	uint64_t sz1, sz2, sz = 2 * rte_mem_page_size();
>>>> +	uint64_t unmap1, unmap2;
>>>> +	uint8_t *alloc_mem;
>>>> +	uint8_t *mem;
>>>> +	int ret;
>>>> +
>>>> +	/* Allocate twice size of requirement from heap to align later */
>>>> +	alloc_mem = malloc(sz * 2);
>>>> +	if (!alloc_mem) {
>>>> +		printf("Skipping test as unable to alloc %"PRIx64"B from heap\n",
>>>> +		       sz * 2);
>>>> +		return 1;
>>>> +	}
>>>> +
>>>> +	/* Force page allocation */
>>>> +	memset(alloc_mem, 0, sz * 2);
>>>> +
>>>> +	mem = RTE_PTR_ALIGN(alloc_mem, rte_mem_page_size());
>>>> +
>>>> +	/* map the whole region */
>>>> +	ret = rte_vfio_container_dma_map(RTE_VFIO_DEFAULT_CONTAINER_FD,
>>>> +					 (uintptr_t)mem, (rte_iova_t)mem, sz);
>>
>> I'm not sure how to resolve this patch for POWER systems.  The patch
>> currently fails with the error:
>>
>> EAL:   cannot map vaddr for IOMMU, error 22 (Invalid argument)
>>
>> The problem is that the size argument (page size of 64KB * 2) is smaller
>> than the page size set when the DMA window is created (2MB or 1GB depending
>> on system configuration for hugepages), resulting in the EINVAL error.  When
>> I tried bumping the sz value up to 2 * 1GB the test also failed because the
>> VA address was well outside the DMA window set when scanning memseg lists.
>>
>> Allocating heap memory dynamically through the EAL works since it's
>> allocated in hugepage size segments and the EAL attempts to keep VA memory
>> addresses contiguous, therefore within the defined DMA window. But the
>> downside is that the memory is DMA mapped behind the scenes in
>> vfio_mem_event_callback().
>>
>> Not sure how to get around this without duplicating a lot of the heap
>> management code in your test.  Maybe others have a suggestion.
> 
> David, Anatoly Burakov,
> 
> Given that both malloc'ed memory and mmap'd memory is not working for POWER9
> setup, I can either drop this test patch(3/4) alone or restrict it to non-POWER9
> system's. Since main fix is already acked, I think it shouldn't be a problem to
> drop test case which was only added to demostrate the problem.
> 
> Any thoughts ?

I dislike having to special case the architecture in general but I don't 
see an easy solution in this case. I'd be fine with either option, 
dropping the test case of disabling test case support for all POWER 
architecture.

Dave


More information about the dev mailing list