[Mageia-dev] ANN: Recent changes in QA updates validation

Wed Aug 15 15:35:51 CEST 2012

Hi,
+
+During summer QA has not stopped working, and is still trying to improve the 
+process so that updates are pushed as quickly as possible.
+
+First, the QA team now uses http://mageia.madb.org/tools/updates to have an 
+overview of current updates candidates. Major security updates are first in 
+list, in bold (or even red bold for highly critical ones). Those are usually 
+pushed very fast.
+
+Other changes include :
+
+--- Taking into account packager tests for updates they provided ---
+
+Until now, packagers could not test their own updates. We decided to change 
+this rule because packagers often know their packages better than QA team. So 
+now packagers can test their own updates, within the following conditions: 
+only one arch per Mageia version, and final update validation must still be 
+done by a QA team member.
+
+So when you provide an update, you can tell us what you have tested, on which 
+version and arch, and how (step by step instructions much appreciated!). Then 
+a QA team member will just have to test the other arch and validate the 
+update, hopefully very fast if a procedure is provided. 
+
+This policy change can be reviewed later if problems arise.
+
+For the record, here is how QA tests update candidates: 
+https://wiki.mageia.org/en/QA_process_for_validating_updates#Test
+
+--- Identification of update candidates coming with a testing procedure ---
+
+Update candidates that have a testing procedure (either provided by the 
+packager or by QA team members) are validated really faster than updates 
+candidates without a procedure. Indeed, most of QA's time is put in trying to 
+find how a package works, so we really appreciate testing instructions.
+
+We added a new "has_procedure" whiteboard marker in bugzilla to identify 
+updates that have a testing procedure. They show in 
+http://mageia.madb.org/tools/updates
+
+--- Identification of update candidates pending packager input ---
+
+We added a new "feedback" whiteboard marker in bugzilla to identify updates 
+for which packager input has been requested. They have a gray background in 
+the aforementioned page. 
+
+Update candidates with that marker are held for a few days in hope that the 
+packager will answer (a simple "please push regardless of this issue" is 
+sometimes enough as a feedback), and QA members give priority to other updates 
+in the meantime. 
+
+Note : they are not held forever, unless the problem is found to be blocking. 
+We review stalled updates regularly, although we don't like pushing updates 
+when the packager doesn't answer our questions at al, of coursel. Critical 
+security fixes are usually not held this way. When questions are answered, QA 
+team usually removes the feedback marker itself, but don't hesitate to do so 
+yourself when answering!
+
+Do you have any questions or remark?
+
+Best regards
+
+Samuel Verschelde
+