[dpdk-ci] No space left on some UNH system

[Lincoln Lavoie]

Hi Thomas,

Yes, we are adding that VM to our internal monitoring system, based on
icinga2.  We are investigating what consumed the disk space, as it is just
a Jenkins worker and should not have had disk space being consumed over
time, as the pipelines should be either deleting temporary artifacts or
moving longer lived artifacts off the workers to the headnodes (i.e. logs
and such), so those items become available through the dashboard, etc.

Our internal monitoring actually utilizes Google Chat, through our company
Gsuite account, so it can easily generate notifications to our phones, etc.

Cheers,
LIncoln

On Tue, Feb 23, 2021 at 1:06 PM Thomas Monjalon <thomas at monjalon.net> wrote:

> 23/02/2021 19:04, David Liu:
> > On Tue, Feb 23, 2021 at 12:56 PM Thomas Monjalon <thomas at monjalon.net>
> > wrote:
> >
> > > 23/02/2021 17:21, David Liu:
> > > > Hi David,
> > > >
> > > > This is an actual issue caused by the machine ran out of space.
> > > >
> > > > I had free up some spaces in the machine.
> > > >
> > > > Please let me know if this happens again.
> > >
> > > Instead of waiting for it to happen again,
> > > please could you monitor the disk space of the machines?
> > >
> > > Yes, I will also be monitoring the pipeline result and disk space.
>
> What does it mean? Will you make sure an alert is automatically sent
> to dpdklab at iol.unh.edu when a space is below a threshold?
>
>
>

-- 
*Lincoln Lavoie*
Principal Engineer, Broadband Technologies
21 Madbury Rd., Ste. 100, Durham, NH 03824
lylavoie at iol.unh.edu
https://www.iol.unh.edu
+1-603-674-2755 (m)
<https://www.iol.unh.edu>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mails.dpdk.org/archives/ci/attachments/20210223/516f69f2/attachment.htm>