[Mageia-dev] [changelog] [RPM] cauldron core/release kdenlive-0.8.2-1.mga2

[David W. Hodgins]

On Tue, 15 Nov 2011 12:34:48 -0500, Juan Luis Baptiste <juancho at mageia.org> wrote:

> I wrote about this some weeks ago but it went unnoticed[1], about a
> cross-platform tool developed by OpenSuSE that is for doing all kind
> of automated QA tests[2]. That tool looks very promising and helpful.
> [1] http://thread.gmane.org/gmane.linux.mageia.devel/8661
> [2] http://news.opensuse.org/2011/10/11/opensuse-announces-first-public-release-of-openqa/

There was brief discussion about it on the qa discuss mailing
list.

I have experience with automated testing going back to the early 80's.
This was on ibm mainframes, starting with Batch Terminal Simulator,
and many others packages, often developed in-house.

As soon as I saw that the testing scripts are using hashes of
screen shots to determine if a test was successful, I cringed.
You better not have a clock displayed, during the test.

When you know *exactly* what the expected output will be, the
tests can be automated.  Otherwise, you'll spend more time tweaking
the test cases, rather then testing the updates.

Where the qa team does have problems, or encounter delays, at
present, there are two main reasons.

Lack of hardware.  For example, mtd-utils is currently on hold
pending testing by the person who requested the package, as no
one on the qa team has an mtd flash device.

Time and knowledge required to set up a test environment.  For
example, in order to test gssftp from the krb5-appl-servers
package, the qa tester has to set up and configure a kerberos
server, with an appropriate backend such as ldap.

There are only a few cases where repetitive testing is involved,
where an automated procedure would be useful.

Regards, Dave Hodgins