[Mageia-dev] Mirror tree structure

Thu Oct 21 18:16:06 CEST 2010

On Thursday, 21 October 2010 06:37:37 Olivier Thauvin wrote:
+> * J.A. Magallón (jamagallon at ono.com) wrote:
+> > On Wed, 20 Oct 2010 18:34:24 +0200, Olivier Thauvin 
+<nanardon at nanardon.zarb.org> wrote:
+> > > Hi,
+> > 
+> > ...
+> > 
+> > > Now come the question: "what is a valid mirror ?", eg, what a mirror
+> > > should have as file to be valid ?
+> > 
+> > Some ideas (probably silly ;)).
+> > 
+> > Why don't you split mirrors in 2 categories:
+> > - Software (RPM) repo mirrors, available for network install or urpmi.
+> > 
+> >   (distrib+people+software).
+> > 
+> > - ISO mirrors. Avaliable for torrenting or ftp. MDV2010.1 release isos
+> > are
+> > 
+> >   about 15Gb. With 2 releases per year, thats 30Gb per year, 200 Gb on 6
+> >   years. Some places can just mirror this...you lower the 700Gb to 500,
+> >   not bad.
+> > 
+> > so mirroring can be done by version (or was this what you wanted to
+> > avoid?).
+> 
+> Yes, I tried to avoid it because I perfectly know mirrors will take care
+> to mirror new version every 6 months (ibiblio example again, nobody had
+> a look for several years, at least since 2008).
+
+This will result in fewer mirrors (those that cannot or choose not to dedicate 
+~ 1TiB). I would rather consider other solutions to this problem. On my own 
+mirror (which has a total of 820GB available at present), I use exclude 
+options to rsync in my mirror scripts (but, I have to change these after each 
+release).
+
+Maybe instead we could provide exclude-list files on the mirror, so mirror 
+maintainers can use --exclude-from=/path/to/mageia/mirror_cfg/old_releases.cfg
+
+Also, in order to try and expedite getting what most users need out to mirrors 
+first, I would prefer either:
+-development tree separate from releases
+-development release should be selected last by mirroring tools (e.g. follow 
+alphabetically all stable releases if not separate)
+
+
+> 
+> > BTW
+> > <nitpickin>
+> > - could you kill the final 's' on all the names ?
+> > 
+> >   distribs -> distrib
+> 
+> Done
+
+Maybe 'releases' or 'release' instead?
+
+> 
+> >   iso (you didnt named it 'isos')
+> >   peoples -> people
+> 
+> Done
+> 
+> >   software (not softwares)
+> 
+> Is already 'software'
+> 
+> > - could you not mix case in names (SRPMS <-> x86_64), just srpms...
+> 
+> This part depend on BS and how this team working on it will organise the
+> distrib. I'll suggest them.
+> 
+> > - could be the arch names more uniform ? in my personal scripts/setups
+> > 
+> >   I use x86-32 and x86-64.
+
+Is x86-32 a valid architecture for rpm etc.? While uniformity might be nice, 
+unfortunately vendors don't necessarily choose uniform architecure names, and 
+it might be better to match the repo structure to values that can be 
+determined directly (and not heuristcally) . 
+
+I've also never seen 'uname -m' report x86-32 or x86_32.
+
+> >   Moreover, perhaps in a not so near future some
+> >   adventurous soul builds Mageia on ARM or Sparc, so why not sort things
+> >   like
+> >   
+> >     distrib/cauldron/srpm
+> >     distrib/cauldron/x86/32/iso
+> >     
+> >                            /rpm
+> >                         
+> >                         /64
+> >                     
+> >                     /arm/32
+> >                     
+> >                         /64
+
+I don't know if memory address space is a useful differentiator here, as 
+features differ substantially in different ARM cores of the same family or 
+architecture version. E.g., Fedora has an 'armv5tel' architecture, N900 ships 
+.deb's with 'armel' as the architecture. See 
+http://en.wikipedia.org/wiki/ARM_architecture
+
+> >                     
+> >                     /sparc/32
+> >                     
+> >                           /64
+
+AFAIK, the valid architecture names for sparc are sparc,sparc64,sparcv9.
+
+> 
+> This depend again on BS part.
+> 
+> >   The drawback is that ISOs are in the same tree so no separate mirroring
+> >   is possible, but I think you didn't want this anyways.
+> 
+> I'll take care to this.
+> 
+> > All this renaming in case it is not some kind of standard for tree layout
+> > or the like...
+> > </nitpickin>
+> > 
+> > Just my 2€cent, you will judge...
+
+The readme file mentions that mirrors must sync every 2 hours. For tier1 
+mirrors, I would agree, however you may need to stagger sync intervals to 
+lower tiers (e.g. 4 hours at tier2 and 12 hours at tier3).
+
+It may be worthwhile to try and estimate the actual rsync bandwidth that must 
+be achieved to make a specific tier.
+
+(I am not able to sync every 2 hours, probably not 4, maybe 12, though my 
+mirror currently syncs daily - for Mandriva I could sync 'official' more 
+regularly, but not 'devel').
+
+
+Regards,
+Buchan
+