DPDK clone/fetch URL

[Adam Hassick]

Hi Thomas,

Yes, mirroring the next-* branches would help to reduce the complexity of
creating patch series artifacts.
As Patrick wrote, moving the clones for the periodic testing on DPDK main
and DPDK LTS should be a trivial change.

However, most of the load we put on that server originates from our testing
on new patch series.
I am working on a new script for creating the tarball artifacts from
incoming patch series. It only pulls down either 1 or 2 repositories rather
than main and all next-* branches for every single patch.
Once deployed, I expect this script to reduce the load on git.dpdk.org
significantly.
Also, I anticipate that I can modify this new script to pull main down from
GitHub. Then, only incoming patch series for next-* branches will produce
any load on git.dpdk.org.

Thanks,
Adam Hassick

On Mon, Nov 27, 2023 at 3:58 PM Thomas Monjalon <thomas at monjalon.net> wrote:

> 27/11/2023 17:09, Patrick Robb:
> > Hello,
> >
> > Thanks for the heads up Thomas. For the Community Lab, as far as I can
> > tell, every git clone/fetch for DPDK main and DPDK LTS should be able to
> be
> > moved to the GitHub repo.
> >
> > On the other hand, I don't think next-* branches are mirrored on GitHub
> (at
> > least for now), so unless I'm mistaken, we will still have to pull from
> > git.dpdk.org. But, between DPDK main and LTS, it should still be a
> sizable
> > reduction on the load put on your server. I'll discuss with the team
> today
> > and we will update here.
>
> We should mirror next-* trees as well I think, good point.
>
>
> > On Mon, Nov 27, 2023 at 6:21 AM Thomas Monjalon <thomas at monjalon.net>
> wrote:
> >
> > > Hello,
> > >
> > > The server git.dpdk.org is not strong enough to hold all CI requests.
> > > When a script is doing a git fetch or clone, it is consuming workload
> on
> > > the server.
> > > Please could you favor the mirror on GitHub in any CI automation?
> > >         https://github.com/DPDK/dpdk.git
> > > This mirror is updated as frequently as git.dpdk.org:
> > > We push to git.dpdk.org where a git hook is replicating the same on
> > > GitHub.
> > >
> > > Hope we can reduce the workload on our main server.
> > > Thank you everyone for collaborating.
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mails.dpdk.org/archives/ci/attachments/20231127/906ce77d/attachment.htm>