[Mageia-sysadm] Toys&Tools for Mageia Infrastructure

[nicolas vigier]

On Tue, 03 Jan 2012, Kamil Rytarowski wrote:

> W dniu 02.01.2012 22:40, Romain d'Alverny pisze:
>> Hi,
>>
> Hello and thank you for the response!
>> On Fri, Dec 16, 2011 at 02:20, Kamil Rytarowski<n54 at gmx.com>  wrote:
>>> 1. *Mageia packages maintainership daily statistics* - It's already working
>>> here http://kamil.mageia.org.pl/MageiaStats/
>>> 2. *SVN searcher* - Preview is working here
>>> http://kamil.mageia.org.pl/svnsearch.cgi - data is a few weeks old, twice
>>> synced
>>> 3. *Missing dependencies daily statistics*  - to-be done
>>> 4. *Statistics about our packagers* (like https://cia.vc - but more
>>> information) - to-be done
>>> 5. *Build server statistics* - to-be done
>>> And a few more ideas.
>> That sounds excellent. I started a small, progressive dashboard mockup
>> here: http://dashboard.mageia.org/ but it still needs quite some work
>> (and more data sources to be available through a Web interface).
> I like this page. My idea is similar to show different things, but divided 
> into parts, not on one page.
>>> Some of the projects above are more toys than tools, but anyway I find them
>>> handy (like the maintainership statistics).
>> What would be even better would be to make all/most of these
>> informations available from a central place (at least in a documented
>> way) so that consistent mashups can be built with time.
> I will do it.
>>   You could
>> start by adding to https://wiki.mageia.org/en/Metrics above resources
>> you compiled.
> Thank you! I will add a page when it is ready.
>
> Is it possible to move in a long-term, ready or working (certain) pages 
> from the mageia.org.pl server to a central one? This is a server 
> administrated by a friend of mine, that ships a few web-pages and it's 
> short on hardware and won't run everything.

I think we can host those statistic scripts on Mageia servers. Maybe as
part of http://dashboard.mageia.org/ ?

Source code can be hosted on svn.mageia.org/svn/soft/[something], and be
packaged in Mageia, then we can install the package on Mageia servers.

>>> So then I have two primary questions:
>>> - How to you find that, is it possible to (say daily, weekly) synchronize
>>> Mageia SVN (only Cauldron tree?)? What about fetching svn:log data? It's a
>>> few GB of data and a lot of files, directories
>> sysadmins?
> It would be excellent to synchronize the current SVN tree with rsync. Could 
> the sysadmins allow to do this please? Then keeping fresh list of all 
> .specs would be incomparably more friendly, including for the Mageia 
> server.

You can synchronize a copy of the SVN using svnsync. With something like
this :
$ svnadmin create /path/to/local/repo
$ svnsync init file:///path/to/local/repo svn://svn.mageia.org/svn/packages

And something like this in cron :
$ svnsync synchronize file:///path/to/local/repo

I don't think svnsync takes a lot of resources on the server (except
the first time when everything has to be downloaded), so if that's
useful it shouldn't be a problem to run it every 15 or 30 minutes.

You can then use your local svn mirror to get all the infos you need.

> BTW. Please also fix this bug https://bugs.mageia.org/show_bug.cgi?id=2839 
> or please tell me where is the source-code of this component.

This should now be fixed.