1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<TITLE> [Mageia-dev] Regression testing for generated RPM
</TITLE>
<LINK REL="Index" HREF="index.html" >
<LINK REL="made" HREF="mailto:mageia-dev%40mageia.org?Subject=Re%3A%20%5BMageia-dev%5D%20Regression%20testing%20for%20generated%20RPM&In-Reply-To=%3C50E692EC.20903%40gmail.com%3E">
<META NAME="robots" CONTENT="index,nofollow">
<META http-equiv="Content-Type" content="text/html; charset=us-ascii">
<LINK REL="Previous" HREF="021124.html">
<LINK REL="Next" HREF="021128.html">
</HEAD>
<BODY BGCOLOR="#ffffff">
<H1>[Mageia-dev] Regression testing for generated RPM</H1>
<B>Guillaume Rousse</B>
<A HREF="mailto:mageia-dev%40mageia.org?Subject=Re%3A%20%5BMageia-dev%5D%20Regression%20testing%20for%20generated%20RPM&In-Reply-To=%3C50E692EC.20903%40gmail.com%3E"
TITLE="[Mageia-dev] Regression testing for generated RPM">guillomovitch at gmail.com
</A><BR>
<I>Fri Jan 4 09:29:32 CET 2013</I>
<P><UL>
<LI>Previous message: <A HREF="021124.html">[Mageia-dev] Regression testing for generated RPM
</A></li>
<LI>Next message: <A HREF="021128.html">[Mageia-dev] Regression testing for generated RPM
</A></li>
<LI> <B>Messages sorted by:</B>
<a href="date.html#21125">[ date ]</a>
<a href="thread.html#21125">[ thread ]</a>
<a href="subject.html#21125">[ subject ]</a>
<a href="author.html#21125">[ author ]</a>
</LI>
</UL>
<HR>
<!--beginarticle-->
<PRE>Le 04/01/2013 09:11, Jochen Breuer a écrit :
><i> Hi everyone,
</I>><i>
</I>><i> I've already posted this to the Mageia forum, but doktor5000 suggested
</I>><i> to also post this to the mailing list.
</I>><i>
</I>><i> I'd like to ask if it would make sense for Mageia to automatically test
</I>><i> generated RPM packages. The idea isn't new. Ubuntu is using regression
</I>><i> tests with python-unit for a lot of packages in their distribution:
</I>><i> <A HREF="https://wiki.ubuntu.com/MeetingLogs/devweek0909/RegressionTests">https://wiki.ubuntu.com/MeetingLogs/devweek0909/RegressionTests</A>
</I>><i>
</I>><i> I think that using automated regession tests would make the job of QA a
</I>><i> lot easier. Failures within packages would surface much faster and a lot
</I>><i> of manual checking could be avoided. Don't get me wrong. Every package
</I>><i> that would pass the regression tests should still be checked by a human. ;)
</I>><i>
</I>><i> The following scenario could be used for this. After the package hast
</I>><i> been generated, the RPM (or job) is passed to the regession test server.
</I>><i> This server spawns a new virtual machine with the Mageia version the
</I>><i> package should be checked on. When the machine is booted, the package is
</I>><i> installed with all the needed dependencies and the tests are executed.
</I>><i> If everyting went smooth, the package might be checked again by QA
</I>><i> manually. If one or more tests failed, there is something wrong with
</I>><i> package, although the package generation worked, and the
</I>><i> maintainer/developer might want to take a look at it again.
</I>We already run existing tests suite during package build... Running them
again on a different host would just change the execution environment to
use runtime dependencies instead of build time dependencies. I don't
know if the potential results are worth the additional infrastructure
needed.
><i> To setup this up a combination of hudson (hudson-ci.org
</I>><i> <<A HREF="http://hudson-ci.org">http://hudson-ci.org</A>>) and vagrant (vagrantup.com
</I>><i> <<A HREF="http://vagrantup.com">http://vagrantup.com</A>>) could be used. vagrant allows to use VirtualBox
</I>><i> to boot serveral VM instances from base images that are thrown away
</I>><i> after everything is done. This way multiple versions of Mageia could be
</I>><i> tested on one well equipped server. One drawback with VirtualBox would
</I>><i> be the missing ARM support.
</I>><i>
</I>><i> Let's use something like a JSON lib for Python as an example. The
</I>><i> package generation was successful. But Python is a highly dynamic
</I>><i> language, so some submodule is missing due to a wrong install path,
</I>><i> although "import json-xy" of the main module works flawlessly. Without
</I>><i> further testing this will only surface when a poor developer uses the
</I>><i> submodule or when a second package that depends on that submodule is
</I>><i> tested by QA and shows errors or misbehaviours.
</I>><i> Second example, same lib. The JSON lib usually ships with a handy
</I>><i> executable to lint check JSON files, but upstream decided to change
</I>><i> that. Due to an error in the upstream repo, the executable script file
</I>><i> is still there but it's empty or there is gibberish in it. There is no
</I>><i> more lint checking possible with this executable. Automatic testing
</I>><i> would show this very fast and without someone sitting there, pasting
</I>><i> some JSON into a file to feed the executable with it.
</I>Which sounds likes "writing tests", not "executing already existing
tests", which is a whole different story...
--
BOFH excuse #139:
UBNC (user brain not connected)
</PRE>
<!--endarticle-->
<HR>
<P><UL>
<!--threads-->
<LI>Previous message: <A HREF="021124.html">[Mageia-dev] Regression testing for generated RPM
</A></li>
<LI>Next message: <A HREF="021128.html">[Mageia-dev] Regression testing for generated RPM
</A></li>
<LI> <B>Messages sorted by:</B>
<a href="date.html#21125">[ date ]</a>
<a href="thread.html#21125">[ thread ]</a>
<a href="subject.html#21125">[ subject ]</a>
<a href="author.html#21125">[ author ]</a>
</LI>
</UL>
<hr>
<a href="https://www.mageia.org/mailman/listinfo/mageia-dev">More information about the Mageia-dev
mailing list</a><br>
</body></html>
|
