DPDK clone/fetch URL

[Patrick Robb]

Hi Thomas,

Adam and I discussed and I agree with his points. His "patchseries apply"
script rewrite will already alleviate this issue to a significant degree,
but it will be even better if we can also use next-* mirrors on GitHub. Do
you know who I can contact about this?

Also, as of today our LTS testing is now moved over to GitHub, which should
help a little.

On Mon, Nov 27, 2023 at 4:30 PM Adam Hassick <ahassick at iol.unh.edu> wrote:

> Hi Thomas,
>
> Yes, mirroring the next-* branches would help to reduce the complexity of
> creating patch series artifacts.
> As Patrick wrote, moving the clones for the periodic testing on DPDK main
> and DPDK LTS should be a trivial change.
>
> However, most of the load we put on that server originates from our
> testing on new patch series.
> I am working on a new script for creating the tarball artifacts from
> incoming patch series. It only pulls down either 1 or 2 repositories rather
> than main and all next-* branches for every single patch.
> Once deployed, I expect this script to reduce the load on git.dpdk.org
> significantly.
> Also, I anticipate that I can modify this new script to pull main down
> from GitHub. Then, only incoming patch series for next-* branches will
> produce any load on git.dpdk.org.
>
> Thanks,
> Adam Hassick
>
> On Mon, Nov 27, 2023 at 3:58 PM Thomas Monjalon <thomas at monjalon.net>
> wrote:
>
>> 27/11/2023 17:09, Patrick Robb:
>> > Hello,
>> >
>> > Thanks for the heads up Thomas. For the Community Lab, as far as I can
>> > tell, every git clone/fetch for DPDK main and DPDK LTS should be able
>> to be
>> > moved to the GitHub repo.
>> >
>> > On the other hand, I don't think next-* branches are mirrored on GitHub
>> (at
>> > least for now), so unless I'm mistaken, we will still have to pull from
>> > git.dpdk.org. But, between DPDK main and LTS, it should still be a
>> sizable
>> > reduction on the load put on your server. I'll discuss with the team
>> today
>> > and we will update here.
>>
>> We should mirror next-* trees as well I think, good point.
>>
>>
>> > On Mon, Nov 27, 2023 at 6:21 AM Thomas Monjalon <thomas at monjalon.net>
>> wrote:
>> >
>> > > Hello,
>> > >
>> > > The server git.dpdk.org is not strong enough to hold all CI requests.
>> > > When a script is doing a git fetch or clone, it is consuming workload
>> on
>> > > the server.
>> > > Please could you favor the mirror on GitHub in any CI automation?
>> > >         https://github.com/DPDK/dpdk.git
>> > > This mirror is updated as frequently as git.dpdk.org:
>> > > We push to git.dpdk.org where a git hook is replicating the same on
>> > > GitHub.
>> > >
>> > > Hope we can reduce the workload on our main server.
>> > > Thank you everyone for collaborating.
>>
>>
>>
>>

-- 

Patrick Robb

Technical Service Manager

UNH InterOperability Laboratory

21 Madbury Rd, Suite 100, Durham, NH 03824

www.iol.unh.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mails.dpdk.org/archives/ci/attachments/20231128/c550b60a/attachment.htm>