[Mageia-sysadm] Using SQL database for youri

[nicolas vigier]

Hello,

While looking at enabling submits to backports_testing repository on
Mageia 1, I noticed that we have the following lines in mgarepo.conf on
valstar (config file used by create-srpm, for package submit) :

[submit 1]
target = <%= sched_home_dir %>/repsys/srpms
allowed = svn://svn.<%= domain %>/svn/packages/updates/1
rpm-macros = global 1

It means all submit to mageia 1 should be done with a URL in updates/1
svn directory. And this is a problem for backports as they are usually
done using the package from cauldron directory. Unfortunatly, it's only
possible to set allowed URLs per release, not per repository, so we
cannot allow updates/1 directory for updates_testing and cauldron
directory for backports.

We also have a problem with markrelease commits always done on the
cauldron directory, even for mageia 1 updates. Youri always run
markrelease on the cauldron directory because it doesn't know the URL
that was used for the submit, this URL is only known by create-srpm
which only uses it to generate the src.rpm, before sending the src.rpm
to youri.

So I think we could do this to fix those problems :
1) setup a SQL database (with postgres on alamut), with the following columns :
   sha1 of src.rpm (as index), URL used for submit, repository and later
   other infos about build
2) change create-srpm to save the URL of the submit in the database
   before giving the src.rpm to youri
3) add a new check module in youri to check that the URL used is correct
4) update the youri Action/Markrelease.pm module to do the markerelease
   on the URL from the database, instead of the cauldron directory

All the infos about builds are currently stored by youri in the
filesystem. I think that later we can store them in the database. I
think it would be better because :
- we can keep history about previous builds longer. I think that
  currently we only keep history from the last 48h because things become
  slow with too many files to parse.
- it's easier to add new infos in the database (we only need to add a
  new column in the database)
- we can use SQL query to search infos about builds (it can be useful
  if we want to improve build system web page, to add more infos)
- build nodes can add infos in the database directly (for instance
  adding the URL of current build logs, if we want to provide access to
  the logs before the build is finished)

What do you think about this ?