[dpdk-ci] [PATCH v2 10/10] tools: skip the IDs we already fetched
Ali Alnubani
alialnu at oss.nvidia.com
Mon Oct 11 21:30:07 CEST 2021
> -----Original Message-----
> From: Thomas Monjalon <thomas at monjalon.net>
> Sent: Thursday, September 30, 2021 1:33 PM
> To: Ali Alnubani <alialnu at nvidia.com>
> Cc: ci at dpdk.org; jerinj at marvell.com; ferruh.yigit at intel.com;
> david.marchand at redhat.com; juraj.linkes at pantheon.tech
> Subject: Re: [PATCH v2 10/10] tools: skip the IDs we already fetched
>
> 21/09/2021 16:35, alialnu at nvidia.com:
> > From: Ali Alnubani <alialnu at nvidia.com>
> >
> > Store the IDs we already fetched in a file and don't
> > run 'callcmd' again for them.
>
> We store all IDs. Should we manually remove olds one from time to time?
>
Do you have a suggestion for when should we clear this file? Maybe each time the script starts?
> We need an explanation about the strategy, why it is needed.
> I think it is because filtering by date is not enough.
> In order to not miss any patch, we should request a date earlier
> than the previous fetch and skip those already fetched.
>
The reason this change was made isn't because filtering by date is not enough, it's because
I want to avoid feeding the same ID to 'callcmd' more than once.
This can happen if a patchwork ID was created between recording date_now and fetching the API.
I don't think we are missing any IDs, even without this change.
> Where the "earlier date" is defined?
There are 2 variables, "date_now", which is recorded right before fetching from the API, and then gets written
to the file, and "since", which is the last date that was written to the file.
>
> > Signed-off-by: Ali Alnubani <alialnu at nvidia.com>
> > ---
> > tools/poll-pw | 9 +++++++++
> > 1 file changed, 9 insertions(+)
> >
> > diff --git a/tools/poll-pw b/tools/poll-pw
> > index e197a91..dacf34b 100755
> > --- a/tools/poll-pw
> > +++ b/tools/poll-pw
> > @@ -65,6 +65,11 @@ if [ ! $? -eq 0 ] ; then
> > exit 1
> > fi
> >
> > +poll_pw_ids_file=/tmp/poll_pw_${resource_type}_ids
> > +if [ ! -f "$poll_pw_ids_file" ] ; then
> > + touch $poll_pw_ids_file
> > +fi
> > +
> > URL="${URL}/${resource_type}/?project=${project}&"
> >
> > callcmd () # <patchwork id>
> > @@ -77,7 +82,11 @@ while true ; do
> > date_now=$(date '+%FT%T')
> > since=$(date -d $(cat $since_file | tr '\n' ' ') '+%FT%T')
> > for id in $(curl -s "${URL}since=${since}" | jq '.[].id') ; do
> > + if grep -q "^${id}$" $poll_pw_ids_file ; then
> > + continue
> > + fi
> > callcmd $id
> > + echo $id >>$poll_pw_ids_file
> > done
> > echo -n $date_now >$since_file
> > # pause before next check
> >
>
>
>
>
More information about the ci
mailing list