[Mageia-dev] Will this work for a build system?

Mon Sep 27 11:36:03 CEST 2010

2010/9/27 herman <herman at aeronetworks.ca>
+
+
+> On Sun, 2010-09-26 at 18:32 -0700, Frank Griffin wrote:
+> > Giuseppe Ghibò wrote:
+> > > IMHO one of the building problems was not massive automatic rebuilding
+> > > but avoid bottenlecks to the users when building goes wrong.
+> > I really like the concept of a distributed build system.
+>
+> The problem with a distributed system is the enormous increase in
+> complexity.  As long as a single big server with about 24 cores can
+> compile the lot in one day, then a distributed system is not really
+> needed.
+>
+>
+I agree there would be an ENORMOUS increase in complexity, but you are
+forgetting that first 24 cores someone cited were not 24 cores, but a dual
+exacore machine (I guess a dual Xeon 5650) so 12 core with hyperthreads,
+which is not exactly the same as 24 native core. Such core with 12GB of
+memory are just "peanuts" in an environment with plenty of developers. Half
+of that machine (e.g. with a core i7 985X, or AMD 1055T) is actually a
+medium/top PC which you can build in your home.
+
+So you have to distinguish automatic rebuilding of the distro, which
+operates on "working" packages from the svn, from new packages built from
+the first time. The first task of a massive rebuilding is a automated task
+which can be done sequentially (but 1 day is only for the main, then there
+is contrib, then you have 2 archs, 32 and 64bits, and backports); the second
+task instead has a lot of stop and go. If for instance a build goes wrong
+because a packager couldn't test a parallel build in his own development
+machine (or because the number of cores is different and you get race
+conditions problems), and have to redo the work, but in the meanwhile the
+building system is busy, or has other kind of problems (wait for library to
+propagate, etc.), then he have to spend a lot of time fighting against the
+system and babysitting a package, rather than concentrate on packaging or
+developing. And for sure he would fly away (packagers are not of iron with
+infinite patience).
+
+Also consider that the phase of LZMA compression of the RPM building won't
+operate in parallel.
+
+Bye
+Giuseppe.
+-------------- next part --------------
+An HTML attachment was scrubbed...
+URL: </pipermail/mageia-dev/attachments/20100927/d8e0e4e8/attachment.html>
+