[Mageia-sysadm] Update procedure on sysadmin side

[Carlos Carvalho]

Michael scherer (misc at zarb.org) wrote on 22 July 2011 23:37:
 >> Should we hardlink the rpms instead of moving them,
 >> and postpone the removal from *_testing for a few days (or ~a week)
 >> so all mirrors catch up...
 >> 
 >> That would save the mirrors from re-downloading the stuff, and make
 >> the updates available faster...
 > 
 >> It does not matter much for a simple update like logrotate, but a
 >> kernel update or a full kde update would definately benefit from it.
 >
 >I think that's negligeable when compared to the size of cauldron updates,
 >and that would be slightly more complex to develop.
 > 
 >This would also mean we keep then in updates_testing, thus having bigger 
 >hdlists, which mean more rpms to parse, and more data to download by mirror 
 >( since hdlists are redownloaded each time, since they are compressed and
 >I am doubtful about the rsync-friendliness of the whole compression ).

Depends on the type of compression you use. In general compression
produces cumulative changes and thus spoils rsync's economy. However if
you use gzip you can do gzip --rsyncable and preserve rsync
efficiency. --rsyncable is a separate patch to gzip; Debian has it.
If one wishes high compression then xz should be used, not gzip.

 >And since hdlists are downloaded by every users doing testing, it may have a
 >bigger impact.

I'm not sure what hdlists mean. If it's file lists it'll be
negligible. OTOH, if users have to download full file lists each time
you have more urgent problems than the extra hardlinks, you should
make available diffs to these lists like Debian does. Otherwise users
may end up downloading more bureaucracy stuff than what's important for
them...

For mirrors sending a little extra is no problem. The advantage of
propagating changes faster and doing less disk writes is much more
important.

 >> the downside is of course more complex code on staging/primary mirror
 >
 >Yes, and I fear this may not be so trivial to do (ie, how do we know
 >that we need to remove a rpm from update_testing, since we cannot
 >base on the creation date due to the hardlink )

How about creating a list of the names that are hard linked instead of
removed at each master update? You can simply log-rotate it and
each cycle will just remove the names in the Nth one. Doesn't look
very complex. With N=2 or 3 you'll cater for the vast majority of
mirrors.