[Mageia-dev] Mirror layout, round two

Mon Nov 29 17:08:25 CET 2010

Le lundi 29 novembre 2010 à 15:24 +0100, Samuel Verschelde a écrit :
+> Le lundi 29 novembre 2010 14:56:20, Michael Scherer a écrit :
+> > 
+> > Le lundi 29 novembre 2010 à 13:14 +0100, Samuel Verschelde a écrit :
+> > >
+> > > It was said early that you just have to look at whether the package 
+> > > has a maintainer or not to make a distinction, but this is not sufficient. 
+> > > A maintainer can be very active in cauldron but not care about maintaining 
+> > > for stable releases. A package can have no maintainer but be actively 
+> > > supported in practice (by everybody in cauldron, by security team in 
+> > > stable releases).
+> > >
+> > > The current main vs contrib approach in mandriva is very sensible. It has some (minor, to me) 
+> > > drawbacks, but when I install packages from main I *know* there will be security updates, 
+> > > bugfix updates, and a QA process that packages in contrib don't have. 
+> > 
+> > Since we pull lots of deps ( like perl module, python module ) to ensure
+> > that everything build fine without any formal check of packages, I can
+> > ensure you that packages in main do not have more QA process than
+> > anything. 
+> 
+> May be true until the final freeze. Once the freeze happens there is a formal 
+> QA process for security and bugfix updates. That's what my post revolved around.
+
+And how having 1 merged media prevent this process ?
+
+
+> > Let me remind the problems that we are having with the split, as they
+> > are not so minor :
+> > - complexity on build system side to take in account this
+> >   - packagers time lost to understand what going on when self
+> >     containment is broken. Older ones do not have the problem ( or at
+> >     least, know it ), but newer one sooner or later does.
+> 
+> That's also what allows to block easily unwanted updates on main packages for stable releases. 
+
+I think you misunderstood me.
+I speak of the fact that in cooker, people add BuildRequires in contribs
+for a package in main and get puzzled by the error message. Granted, we
+need to improve the error message, but that still make us lose time.
+( and the fact that no one did improve the message is imho sign that
+this is not annoying enough, to warrant a patch, yet annoying enough to
+lose people )
+
+> > - complexity on the users system as they need to have twice the number
+> >   of media to cope with this. This also appear in the interface of
+> >   various tools, and it is imho uneeded. Most people will say that we
+> >   already have too much medias.
+> 
+> Indeed, however it helps showing that there's a set of packages which is supported, 
+> and another one which is only on behalf of the maintainer. In a community driven 
+> distribution, this distinction may remains valid : some packages are officially 
+> supported by the distribution, others may or may not be, depending on the maintainer 
+> (or lack of maintainer).
+
+If there is a maintainer and he has a account, then it is a official
+one, and so the package is officially supported. That's all. There is no
+separation of "the organisation" and "the maintainers", we are not
+Mandriva. 
+
+So either the package is supported, and we keep, or it is not, and then
+why should we keep it ?
+
+
+> The main complexity in current media in Mandriva comes from the updates/backports/testing/debug 
+> packages which multiply the number of visible media, but this can be drastically improved on UI side.
+
+That's exactly what i told 3 mails ago....
+I have also added that we cannot do miracles on ui if we keep pushing
+for more complexity. 
+
+> 
+> > - time lost managing it. moving from contribs to main, and cleaning
+> >   main. Main cleaning that is seldomly done because :
+> >    - no one has time to do it
+> >    - people fiercly defend the inclusion of their favorite software in
+> >        main 
+> 
+> If maintaining a package in main means a real committment to support it, 
+> with proper rules and management it seems doable to have it work better 
+> than it used to at Mandriva.
+
+It was already the case at Mandriva. But it didn't work. What step do
+you propose to avoid the same pitfall ( ie, packages pushed in main
+without checking first with support team, packages never cleaned because
+that's a hard job, endless bickering on what goes in main or contribs ) 
+
+> > - time lost because packagers have to wait on overloaded sysadmin to do 
+> >   the move
+> 
+> Let's give the ability to more people to do the move ?
+
+Then they would push what they want without consideration of support.
+See the previous paragraph. 
+
+> > > If we plan to have such processes, does the merge between core and extra 
+> > > make is efficient ? I guess we don't plan to have all packages (even 
+> > > maintained ones) equally supported with a full QA process (doesn't seem realistic) ?
+> > 
+> > Again, reread the mail I sent. This is a user concern, not a mirror
+> > admin concern. As such, it should not leak on mirror organisation,
+> > neither on BS organisation. 
+> > And the filtering proposal I made could perfectly fit, provided we find
+> > a way to get the "QA note" for a package. Maybe a webapp, with 
+> > 
+> 
+> If there's a way to really flag packages as supported, so that we can 
+> query the RPM database and know if a package is supported or not, and 
+> do it easily (as a user), then I have nothing more against merging 
+> core and extras.
+
+Why in the rpm db ?
+A external file would perfectly do the trick. The same goes for
+maintainers. The only thing needed is less talk and more code.
+
+
+
+> In fact, mirror structure and processes (including QA and support) are 
+> related, 
+
+No. That's because you think that mirror structure is the only way we
+have to act on urpmi config. And that shouldn't be, because we can't
+support every possible distinction that everybody want without a
+complexity explosion. See what I said in the mail. 
+
+> and the decision to merge core and extras must be taken together 
+> with decisions on QA and support processes, 
+
+Well, everything should be supported or dropped, that's all. Easy, done
+by every other distros out there except those that place a artificial
+separation. If there is a security bug opened and no one act on it after
+a time, let's drop the package. If there is a severe bug and no one act,
+the same, let's drop it.
+
+If people want to resurrect a rpm, they can, there is a svn for that.
+
+
+And if we want to wait on the QA team for defining the mirror structure,
+then we will have a loop ( qa team requires a distro to start and be
+launched, a distro requires a BS the bs requires mirror structure, and
+so mirror can't depend on QA or we will never start ).
+
+> so that we don't end up with 
+> a situation were no one knows what's supported or not because the media 
+> separation disappeared. Without such processes I should remain against the merge.
+
+That's already the case at Mandriva. See for example
+http://lists.mandriva.com/kernel-discuss/2010-10/msg00000.php 
+
+We _already_ do not know what is supported or not at Mandriva. People
+push update to contribs ( I do push them for tor or puppet ), while some
+packages in main are not updated or buggy or simply unrebuildable ( see
+all mdk rpm still in the distro for long forgotten reasons ). See this
+old long standing pdftk bug caused by a issue in the java stack in
+main : https://qa.mandriva.com/show_bug.cgi?id=44372 . In main, and no
+one did anything. 
+
+There is also lots of duplication :
+- apache & nginx, who was moved to main likely because oden like nginx,
+- the various ftp servers, 
+- sendmail & postfix, 
+- the 4 tls/ssl implementation, 
+- the 2 html rendering library ( webkit, gecko ) with more than 1
+browser. 
+
+
+And most people tend to simply not care to the difference between main
+and contribs. If you decide to use puppet for technical reasons ( as we
+did in sysadm team ), I doubt you will switch to another tool because it
+is in contribs. People install web applications without using the distro
+rpm also because they do not care enough about official support ( this
+and maybe other reason ).
+
+And in the end, I think most people just keep contribs and main, and
+forget the difference. 
+
+Those that care more just take a support contact at Mandriva, and with
+enough money, anything can be supported. And there is not really a 1 to
+1 mapping between "supported" and "main".
+
+So the split did not help much on this regard at all. It just give a
+false sense of security and more complexity on the distro side. 
+
+> > > Packages in core get the "QA approved" stamp whereas packages in extra get none.
+> > 
+> > The self containment requirement would make it very hard to get such QA
+> > approved stamp. 
+> > 
+> > We cannot have a package being tested without doing the same for its
+> > requires first. And we cannot say that we support a package without
+> > supporting the full requires. 
+> > 
+> > Look at openoffice at Mandriva. It does requires
+> > tomcat5-servlet-2.4-api, hsqldb on 2010.1 . They are both non trivial to
+> > test, and I am quite sure that they were not much "QA approved" besides
+> > "openoffice work for basic tasks". And in fact, when eclipse was in
+> > main, this was the same.
+> > 
+> > So in order to have openoffice to have a QA approved stamp, openjdk,
+> > tomcat and hsqldb would have one. I am pretty sure they do not had
+> > extensive checks..
+> 
+> The "QA approved" stamps, in my mind, means : "there no known blocking bug or 
+> security issue, and if we find some after the release, we'll fix them rapidly".
+>  This is more a guarantee of support than a guarantee of quality.
+
+Again, I am not sure that hsqldb, being unmaintained, would be
+supported. 
+
+And there is know bugs that were not fixed rapidly in main ( see the
+pdftk example I gave for the most notorious one ).
+
+So let's be honest, let's don't start by doing promise we can't keep for
+the moment.
+
+-- 
+Michael Scherer
+
+