[Mageia-dev] bug 2317 revisited: --update option should behave like --search-media

Fri Jun 22 12:10:12 CEST 2012

On 21/06/12 22:01, AL13N wrote:
+> Op donderdag 21 juni 2012 22:21:52 schreef Sander Lepik:
+>> On Jun 21, 2012 9:10 PM, "AL13N"<alien at rmail.be>
+>>
+>>> so, there's 2 options:
+>>>
+>>> - testing i586 with backports enabled
+>>> - testing x86_64 without backports enabled
+>>>
+>>> this is still 2 tests, and this is sufficient.
+>>
+>> Are you serious?
+>> I've seen bugs were i586 and x86_64 doesn't work quite the same. Every arch
+>> + repo must be tested separately (be it tainted or release, i'm still not
+>> mixing backports with updates ... until you promise to do all the testing
+>> here and not bother QA;)).
+>
+> I see...
+>
+> however, as long as backports is installed, it could still be that due to an
+> update a new dependency from release is pulled, which could conflict (or not
+> work correctly) with some of the installed backports.
+>
+> if we want to have supported both backports and updates, these cases should
+> still be tested. And if you want to support cherrypicking backports, the
+> possibilities are even bigger.
+>
+> It seems to me that people don't really see what kind of QA resources
+> backports will need if all this need to be supported. This is completely
+> irrelevant to this solution however. whatever solution we pick for bug 2317
+> (A, B, or even doing nothing), it seems you think this solution has any effect
+> on QA resources, but it doesn't, it's only enabling backports that does it.
+>
+> Let me give you an overview on options for the amount of support in backports
+> and it's impact on QA (also with example) for only 1 release:
+>
+> (package X has update A&  backport B;
+> package Y has update C and backport D;
+> package Z has update E and backport F)
+>
+>
+> A. backports is fully supported + cherry-picking backports is also supported
+> (for only mga2)
+>
+> for update validation of package X (let's call it update A2):
+> 1. testing combination: A,C,E for arch i586
+> 2. testing combination: A,D,E for arch i586
+> 3. testing combination: A,C,F for arch i586
+> 4. testing combination: A,D,F for arch i586
+> 5. testing combination: A,C,E for arch x86_64
+> 6. testing combination: A,D,E for arch x86_64
+> 7. testing combination: A,C,F for arch x86_64
+> 8. testing combination: A,D,F for arch x86_64
+>
+> for backport validation of package X (let's call it backport B2):
+> 1. testing combination: B,C,E for arch i586
+> 2. testing combination: B,D,E for arch i586
+> 3. testing combination: B,C,F for arch i586
+> 4. testing combination: B,D,F for arch i586
+> 5. testing combination: B,C,E for arch x86_64
+> 6. testing combination: B,D,E for arch x86_64
+> 7. testing combination: B,C,F for arch x86_64
+> 8. testing combination: B,D,F for arch x86_64
+>
+> Validations required: 2*2^(number of backported packages - 1)
+> ==>  this is completely impossible
+>
+> B. backports is fully supported, but cherry-picking isn't
+>
+> for update validation of package X (let's call it update A2):
+> 1. testing combination: A,C,E for arch i586
+> 2. testing combination: A,D,F for arch i586
+> 3. testing combination: A,C,E for arch x86_64
+> 4. testing combination: A,D,F for arch x86_64
+>
+> for backport validation of package X (let's call it backport B2):
+> 1. testing combination: B,C,E for arch i586
+> 2. testing combination: B,D,F for arch i586
+> 3. testing combination: B,C,E for arch x86_64
+> 4. testing combination: B,D,F for arch x86_64
+>
+> Validations required: 4 for each package
+> ==>  this is quadrupling the QA workload
+>
+> B2. i thought perhaps that testing these on one arch would be ok:
+>
+> for update validation of package X (let's call it update A2):
+> 1. testing combination: A,C,E for arch i586
+> 2. testing combination: A,D,F for arch x86_64
+>
+> OR
+>
+> 1. testing combination: A,D,F for arch i586
+> 2. testing combination: A,C,E for arch x86_64
+>
+> for backport validation of package X (let's call it backport B2):
+> 1. testing combination: B,C,E for arch i586
+> 2. testing combination: B,D,F for arch x86_64
+>
+> OR
+>
+> 1. testing combination: B,D,F for arch i586
+> 2. testing combination: B,C,E for arch x86_64
+>
+> Validations required: 2 for each package
+> =>  this could be doable by QA, even though it's perhaps not completely tested
+>
+> C. perhaps backports being semi-supported
+>
+> for update validation of package X (let's call it update A2):
+> 1. testing combination: A,C,E for arch i586
+> 2. testing combination: A,C,E for arch x86_64
+>
+> for backport validation of package X (let's call it backport B2):
+> 1. testing combination: B,C,E for arch i586
+> 2. testing combination: B,C,E for arch x86_64
+>
+> Validations required: 2 for each package
+> =>  this could be doable by QA, but even though a package might work, it's
+> possible that an update (or backport), might not be cleanly installable and
+> give errors.
+>
+> D. not supporting backports
+>
+> for update validation of package X (let's call it update A2):
+> 1. testing combination: A,C,E for arch i586
+> 2. testing combination: A,C,E for arch x86_64
+>
+> for backport validation of package X (let's call it backport B2):
+> No testing
+>
+> Validations required: 2 for each update
+> =>  this is how it is now
+>
+> --------------------------------------------------
+> I would implore all of you to look at this above and try to understand.
+>
+> (of course, you can tell me if i'm wrong here)
+>
+>
+> It seems to me that all of you had thought that C was good enough validation
+> (except for tv, i guess he saw this from the beginning and figured D would be
+> best or even E, no backports at all).
+>
+> I'm was thinking that if C was good enough for you, then perhaps B2 would also
+> be good, as i think it doesn't give any extra load on QA.
+>
+>
+> IMPORTANT: Again, i'm stating that this does NOT matter whatever solution for
+> bug 2317 is chosen.
+>
+> even my preferred solution on bug 2317 ONLY has more testing requirement for
+> the backport packager
+>
+>
+> Is this more clear?
+
+
+All this assumes that backport media will be treated as a normal update 
+media. That is certainly not my impression. My impression of backports 
+are being able to install a new blender for example, not having a system 
+where backports are just another update media and replace everything 
+available. The QA task for that scenario would be ridiculously huge. If 
+you want to have backports which go any further than backports testing 
+then you seriously need to rethink this idea.
+
+I don't think you understand quite how short handed we are in QA. For 
+the life of Mageia 1 it was mainly just two people. This bug is a major 
+bug for QA and has been since it was first discovered almost a year ago. 
+It adds complexity and extra workload to an already severely overloaded 
+team.
+
+We have a few (very few) new people since Mageia 2 was released who are 
+beginning to help out, understand the process and how to find ways to 
+test things. Thankyou all, you know who you are. The complexity of 
+working around this bug is something they shouldn't have to learn. The 
+extra workload involved in working around this bug is something we can't 
+continue with, even in our current state with no backport medias. I 
+spent most of yesterday on it for example instead of testing updates. 
+Validating updates is now taking twice as long because there is twice as 
+much testing involved for two releases - plus the extra time spent 
+working around bug 2317 for each release. Yesterday we even found its 
+effects are different for some rpm's on i586 than they are on x86_64 
+(see p11-kit bug 6502).
+
+Anything which, at this stage, slows this process further will lead to 
+updates being further and further delayed. That is a situation nobody 
+wants to see, and personally I'd like to be able to take a day off now 
+and again too. I'm sure others feel the same.
+
+The aim of fixing this bug is to reduce the complexity and extra 
+workload of working around it for QA. This assumption and solution 
+actually has the opposite effect, dramatically increasing the complexity 
+and workload. As I've explained, that is simply not possible if we want 
+to release timely updates.
+
+I hope this makes the situation clearer. There is a workable solution 
+but I'm afraid it isn't this one, for the reasons given above.
+
+Thanks
+Claire
+