[dpdk-dev] [PATCH v6 3/4] test: add test case to validate VFIO DMA map/unmap

[Nithin Dabilpuram]

On Tue, Jan 05, 2021 at 11:33:20AM -0800, David Christensen wrote:
> Hey Nithin,
> 
> > > +static int
> > > +test_memory_vfio_dma_map(void)
> > > +{
> > > +	uint64_t sz1, sz2, sz = 2 * rte_mem_page_size();
> > > +	uint64_t unmap1, unmap2;
> > > +	uint8_t *alloc_mem;
> > > +	uint8_t *mem;
> > > +	int ret;
> > > +
> > > +	/* Allocate twice size of requirement from heap to align later */
> > > +	alloc_mem = malloc(sz * 2);
> > > +	if (!alloc_mem) {
> > > +		printf("Skipping test as unable to alloc %"PRIx64"B from heap\n",
> > > +		       sz * 2);
> > > +		return 1;
> > > +	}
> > > +
> > > +	/* Force page allocation */
> > > +	memset(alloc_mem, 0, sz * 2);
> > > +
> > > +	mem = RTE_PTR_ALIGN(alloc_mem, rte_mem_page_size());
> > > +
> > > +	/* map the whole region */
> > > +	ret = rte_vfio_container_dma_map(RTE_VFIO_DEFAULT_CONTAINER_FD,
> > > +					 (uintptr_t)mem, (rte_iova_t)mem, sz);
> 
> I'm not sure how to resolve this patch for POWER systems.  The patch
> currently fails with the error:
> 
> EAL:   cannot map vaddr for IOMMU, error 22 (Invalid argument)
> 
> The problem is that the size argument (page size of 64KB * 2) is smaller
> than the page size set when the DMA window is created (2MB or 1GB depending
> on system configuration for hugepages), resulting in the EINVAL error.  When
> I tried bumping the sz value up to 2 * 1GB the test also failed because the
> VA address was well outside the DMA window set when scanning memseg lists.
> 
> Allocating heap memory dynamically through the EAL works since it's
> allocated in hugepage size segments and the EAL attempts to keep VA memory
> addresses contiguous, therefore within the defined DMA window. But the
> downside is that the memory is DMA mapped behind the scenes in
> vfio_mem_event_callback().
> 
> Not sure how to get around this without duplicating a lot of the heap
> management code in your test.  Maybe others have a suggestion.

David, Anatoly Burakov,

Given that both malloc'ed memory and mmap'd memory is not working for POWER9
setup, I can either drop this test patch(3/4) alone or restrict it to non-POWER9
system's. Since main fix is already acked, I think it shouldn't be a problem to
drop test case which was only added to demostrate the problem.

Any thoughts ?
> 
> Dave