diff options
Diffstat (limited to 'zarb-ml/mageia-dev/2013-January/021124.html')
| -rw-r--r-- | zarb-ml/mageia-dev/2013-January/021124.html | 140 |
1 files changed, 140 insertions, 0 deletions
diff --git a/zarb-ml/mageia-dev/2013-January/021124.html b/zarb-ml/mageia-dev/2013-January/021124.html new file mode 100644 index 000000000..c099adbf6 --- /dev/null +++ b/zarb-ml/mageia-dev/2013-January/021124.html @@ -0,0 +1,140 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN"> +<HTML> + <HEAD> + <TITLE> [Mageia-dev] Regression testing for generated RPM + </TITLE> + <LINK REL="Index" HREF="index.html" > + <LINK REL="made" HREF="mailto:mageia-dev%40mageia.org?Subject=Re%3A%20%5BMageia-dev%5D%20Regression%20testing%20for%20generated%20RPM&In-Reply-To=%3CCABHgR8%2BRatT4c6iQuvo3Mo7NOnDufyJZJR7YWk4QZV4X7yQdjg%40mail.gmail.com%3E"> + <META NAME="robots" CONTENT="index,nofollow"> + <META http-equiv="Content-Type" content="text/html; charset=us-ascii"> + <LINK REL="Previous" HREF="021123.html"> + <LINK REL="Next" HREF="021125.html"> + </HEAD> + <BODY BGCOLOR="#ffffff"> + <H1>[Mageia-dev] Regression testing for generated RPM</H1> + <B>Jochen Breuer</B> + <A HREF="mailto:mageia-dev%40mageia.org?Subject=Re%3A%20%5BMageia-dev%5D%20Regression%20testing%20for%20generated%20RPM&In-Reply-To=%3CCABHgR8%2BRatT4c6iQuvo3Mo7NOnDufyJZJR7YWk4QZV4X7yQdjg%40mail.gmail.com%3E" + TITLE="[Mageia-dev] Regression testing for generated RPM">brejoc at gmail.com + </A><BR> + <I>Fri Jan 4 09:11:47 CET 2013</I> + <P><UL> + <LI>Previous message: <A HREF="021123.html">[Mageia-dev] [changelog] [RPM] cauldron core/release file-5.12-1.mga3 +</A></li> + <LI>Next message: <A HREF="021125.html">[Mageia-dev] Regression testing for generated RPM +</A></li> + <LI> <B>Messages sorted by:</B> + <a href="date.html#21124">[ date ]</a> + <a href="thread.html#21124">[ thread ]</a> + <a href="subject.html#21124">[ subject ]</a> + <a href="author.html#21124">[ author ]</a> + </LI> + </UL> + <HR> +<!--beginarticle--> +<PRE>Hi everyone, + +I've already posted this to the Mageia forum, but doktor5000 suggested to +also post this to the mailing list. + +I'd like to ask if it would make sense for Mageia to automatically test +generated RPM packages. The idea isn't new. Ubuntu is using regression +tests with python-unit for a lot of packages in their distribution: +<A HREF="https://wiki.ubuntu.com/MeetingLogs/devweek0909/RegressionTests">https://wiki.ubuntu.com/MeetingLogs/devweek0909/RegressionTests</A> + +I think that using automated regession tests would make the job of QA a lot +easier. Failures within packages would surface much faster and a lot of +manual checking could be avoided. Don't get me wrong. Every package that +would pass the regression tests should still be checked by a human. ;) + +The following scenario could be used for this. After the package hast been +generated, the RPM (or job) is passed to the regession test server. This +server spawns a new virtual machine with the Mageia version the package +should be checked on. When the machine is booted, the package is installed +with all the needed dependencies and the tests are executed. If everyting +went smooth, the package might be checked again by QA manually. If one or +more tests failed, there is something wrong with package, although the +package generation worked, and the maintainer/developer might want to take +a look at it again. + +To setup this up a combination of hudson (hudson-ci.org) and vagrant ( +vagrantup.com) could be used. vagrant allows to use VirtualBox to boot +serveral VM instances from base images that are thrown away after +everything is done. This way multiple versions of Mageia could be tested on +one well equipped server. One drawback with VirtualBox would be the missing +ARM support. + +Let's use something like a JSON lib for Python as an example. The package +generation was successful. But Python is a highly dynamic language, so some +submodule is missing due to a wrong install path, although "import json-xy" +of the main module works flawlessly. Without further testing this will only +surface when a poor developer uses the submodule or when a second package +that depends on that submodule is tested by QA and shows errors or +misbehaviours. +Second example, same lib. The JSON lib usually ships with a handy +executable to lint check JSON files, but upstream decided to change that. +Due to an error in the upstream repo, the executable script file is still +there but it's empty or there is gibberish in it. There is no more lint +checking possible with this executable. Automatic testing would show this +very fast and without someone sitting there, pasting some JSON into a file +to feed the executable with it. + +Testing UI would be an other story, but I guess you get the idea. ;-) + +That do you think about that? I think it would enhance the quality on +Mageia in the long run and take some work from QA to find the really tricky +bugs. + +Regards, + +Jochen +-------------- next part -------------- +An HTML attachment was scrubbed... +URL: </pipermail/mageia-dev/attachments/20130104/182edc02/attachment.html> +</PRE> + + + + + + + + + + + + + + + + + + + + + + + + + + + +<!--endarticle--> + <HR> + <P><UL> + <!--threads--> + <LI>Previous message: <A HREF="021123.html">[Mageia-dev] [changelog] [RPM] cauldron core/release file-5.12-1.mga3 +</A></li> + <LI>Next message: <A HREF="021125.html">[Mageia-dev] Regression testing for generated RPM +</A></li> + <LI> <B>Messages sorted by:</B> + <a href="date.html#21124">[ date ]</a> + <a href="thread.html#21124">[ thread ]</a> + <a href="subject.html#21124">[ subject ]</a> + <a href="author.html#21124">[ author ]</a> + </LI> + </UL> + +<hr> +<a href="https://www.mageia.org/mailman/listinfo/mageia-dev">More information about the Mageia-dev +mailing list</a><br> +</body></html> |