[dpdk-ci] Meson failing on master

[Jeremy Plsek]

On Wed, Dec 4, 2019 at 6:09 AM David Marchand <david.marchand at redhat.com> wrote:
>
> Hello Jeremy,
>
> I noticed that meson is failing for a patch on master.
> Example: https://lab.dpdk.org/results/dashboard/patchsets/8804/
>
> The artifact gives the non-obvious (to me) error log:
>
> Build targets in project: 810
> Option default_library is: shared [default: static]
> Found ninja-1.8.2 at /usr/bin/ninja
> ninja: error: manifest 'build.ninja' still dirty after 100 tries
>
> [1] Error running command.
>
> Talked to Bruce, and he suspects something on the filesystem level.
> Is this issue happening on a system with the overlayfs stuff you mentioned?
>

Hi David,

This started happening last week as well
(https://lab.dpdk.org/results/dashboard/patchsets/8711/), so I
disabled it temporarily.
I enabled it yesterday, and it was fine for a few patch sets
(https://lab.dpdk.org/results/dashboard/patchsets/8803/), but, as you
mentioned, it broke again. I've disabled it again for future tests
until it can be resolved.

This is not related to overlayfs work I've been working on (the
overlayfs work was for the bare metal machines). These tests run in a
container, using podman, with a mount to the jenkins workspace. The
containers run on a no-so-special Ubuntu 18.04 VM.

I've noticed similar issues for other CI
(https://github.com/ninja-build/ninja/issues/1599,
https://github.com/neovim/neovim/issues/10213), which, for the most
part, wasn't really resolved or gave a solution, and may not be
related to our issue.

Hmmm... I was able to reproduce it with the tarball that "failed" most
recently (ps 8804), but it passed with the older "failed" tarball (ps
8711). The difference between them that I noticed, was that when
untarring, it (ps 8804) would complain that the date was in the
future.

$ date
Wed Dec  4 14:28:18 UTC 2019
$ ls -l --full-time
...
-rw-rw-r--  1 jenkins-local jenkins-local   12 2019-12-04
15:03:12.000000000 +0000 VERSION

I tried to just update the files to the current time...

$ find . -exec touch {} +

And now it compiles... Hmmm... Yes, it seems that causes it? Updating
the date fixed it? Interesting bug.

So there are a few issues to look into:
1. Why the dates in the tarball are wrong in the first place (my
issue. or just add a dirty fix to the compile script to just update
the date after extracting)
2. Why the date of the file matters so much in meson/ninja (I
understand that build systems typically use the date to check whether
to recompile a file, but it should be more graceful about handling
this type of in-the-future problem)

I'll see if I can make a simple example and maybe make a bug report.
(At the minimum meson/ninja should fail gracefully rather than retry
100 times.)

>
> --
> David Marchand
>


-- 
Jeremy Plsek
UNH InterOperability Laboratory