[Mageia-dev] Will this work for a build system?

Sun Sep 26 18:04:16 CEST 2010

Le dimanche 26 septembre 2010 à 17:04 +0200, joris dedieu a écrit :
+> 2010/9/26 Olivier Blin <mageia at blino.org>:
+> > R James <upsnag2 at gmail.com> writes:
+> >
+> >>> BTW, I once calculated (test plus extrapolation) how long it would take
+> >>> to rebuild every package in Mandriva on a low end 2 GHz Celeron server
+> >>> that I had available and it came to about 80 days.
+> >
+> > With a reasonably good machine, we used to be able to rebuild most of
+> > "main" in about one day.
+> >
+> >> Perhaps I was naive in thinking that compiling the distro could be
+> >> done with distcc or even a simple queuing system that distributes
+> >> SRPMs to nodes in the community swarm.  As each node returns its
+> >> completed binary package, the queuing system could send it another
+> >> SRPM to build.
+> >>
+> >> It would be cool if it could be done that way.  Why pay for data
+> >> center space, hardware, electricity and big bandwidth when you could
+> >> have a community-provided "cloud" for free? :o)
+> >
+> > Because there are some authentication and integrity issues which are not
+> > simple to solve: we have to be sure that the binary packages really come
+> > from the unmodified SRPM (so that it does not contains malware).
+> 
+> This can be avoid by
+> - building every package twice (also useful for integrity check)
+
+What if a package has changed between the first build and the second in
+such a way that it impact the compilation ?
+
+This would either requires to resubmit the packages ( which will be
+quickly annoying ) or this would requires 3rd compilation, maybe a 4th
+one.
+
+what if the binary include hostname, build date and so on ?
+
+then the build will be seen as different no matter you do ( ie, md5,
+sha1 ) because it will have different contents.
+
+
+> - randomize build order
+> - timedout jobs
+> 
+> It's not a trivial problem but imho distribute tools advantages
+> (price, scalability, availability ...) should be seriously considered.
+> Has a single build system in a single datacenter should be a single
+> point of failure.
+
+there will be a single point of failure, no matter you do :
+
+There is a reference vcs, and a single job dispatcher. We can maybe
+double them or work around issues but this would lead to more complexity
+which may not really compensate a potential datacenter problem.
+
+Fedora had been compromised once and had to shut down their
+infrastructure, or had to move servers sooner this year, they coped with
+the downtime. 
+
+Debian had problem ( like security.debian.org who burned in 2002, or the
+famous openssl problem in 2008 ) too, without trouble. 
+
+Launchpad is often down for database upgrade, and still, Ubuntu is
+there.
+
+-- 
+Michael Scherer
+
+