[Mageia-sysadm] [sysadmin-reports] Hobbit [727252] valstar.mageia.org:disk CRITICAL (RED)

Sat Sep 15 18:21:44 CEST 2012

On Sat, 15 Sep 2012, Thomas Backlund wrote:
+
+> nicolas vigier skrev 15.9.2012 17:51:
+>> On Sat, 15 Sep 2012, root at mageia.org wrote:
+>>
+>>> red Sat Sep 15 16:26:57 CEST 2012 - Filesystems NOT ok
+>>> &red /tmp (100% used) has reached the PANIC level (95%)
+>>>
+>>> Filesystem         1024-blocks      Used Available Capacity Mounted on
+>>> /dev/md0              20152044  10375432   8752928      55% /
+>>> /dev/sda1              1004024     92956    860064      10% /boot
+>>> /dev/sdb1              1004024     92792    860228      10% /boot2
+>>> /dev/mapper/vg0-tmp   32640904  30979996      2868     100% /tmp
+>>
+>> It seems /tmp was full because of rpmlint temporary files. Maybe because
+>> we had a few kernel packages finishing build at the same time ?
+>>
+>
+> Probably yes....
+>
+> As for pushing all kernels at the ~same time was intentional this time...
+>
+> I wanted to see how the BS would handle max load now that new
+> ecosse is working too (as kernel builds do manage to max out
+>  buildnodes due to good support for parallel builds)
+>
+> But I didn't even think about valstar getting into trouble :/
+>
+> core kernel is the worst one for rpmlint as all rpms +
+> their unpacked contents need ~20+ GB diskspace (every
+> kernel-*-debug needs some 2+ GB) ....
+>
+>
+> But otoh I think it was nice side-effect to identify a SPOF
+> now (instead of release time) so we can get it fixed...
+
+Yes.
+
+>
+> question is... do we somehow need to limit how many rpmlint
+> processes is started (depending on cpu load or free disk space
+> in /rmp), or should we just hope the extra disk space is enough ?
+
+Maybe youri could check if there is enough space on /tmp before
+attempting to run rpmlint on the package.
+
+>
+>  Because load on the server was too high, I killed all rpmlint processes,
+>> and removed all rpmlint temporary files in /tmp.
+>>
+>
+> So whah happend to the packages being rpmlinted ? did they get uploaded 
+> anyway ?
+
+Yes. Actually they were already listed as uploaded on pkgsubmit when I
+killed the rpmlint processes. I don't know why there still was rpmlint
+process running when the package was already uploaded.
+
+