jeudi 23 juin 2016

How to run multiple QTest classes?

I have a subproject where I put all my QTest unit tests and build a stand-alone test application that runs the tests (i.e. I run it from within Qt Creator). I have multiple test classes that I can execute with qExec(). However I don't know what is the proper way to execute multiple test classes.

Currently I do it in this way (MVCE):

tests.pro

QT -= gui
QT += core \
    testlib

CONFIG += console
CONFIG -= app_bundle
TEMPLATE = app
TARGET = testrunner

HEADERS += test_foo.h
SOURCES += main.cpp

main.cpp

#include <QtTest>
#include <QCoreApplication>
#include "test_foo.h"

int main(int argc, char** argv) {
    QCoreApplication app(argc, argv);

    TestFooClass testFoo;
    TestBarClass testBar;
    return QTest::qExec(&testFoo, argc, argv) || QTest::qExec(&testBar, argc, argv);
}

test_foo.h

#include <QtTest>

class TestFooClass: public QObject
{
    Q_OBJECT
private slots:
    void test_func_foo() {};
};

class TestBarClass: public QObject
{
    Q_OBJECT
private slots:
    void test_func_bar() {};
};

However the documentation for qExec() says this is the wrong way:

For stand-alone test applications, this function should not be called more than once, as command-line options for logging test output to files and executing individual test functions will not behave correctly.

The other major downside is that there is no single summary for all the test classes, only for individual classes. This is a problem when I have dozens of classes that each have dozens of tests. To check if all tests passed I have to scroll up to see all the "Totals" of what passed/failed of each class, e.g.:

********* Start testing of TestFooClass *********
PASS   : TestFooClass::initTestCase()
PASS   : TestFooClass::test_func_foo()
PASS   : TestFooClass::cleanupTestCase()
Totals: 3 passed, 0 failed, 0 skipped, 0 blacklisted
********* Finished testing of TestFooClass *********
********* Start testing of TestBarClass *********
PASS   : TestBarClass::initTestCase()
PASS   : TestBarClass::test_func_bar()
PASS   : TestBarClass::cleanupTestCase()
Totals: 3 passed, 0 failed, 0 skipped, 0 blacklisted
********* Finished testing of TestBarClass *********

I'm also surprised my qExec() || qExec() works considering that the documentation says if a test failed qExec() returns a non-zero value, which should mean all the following qExec() calls wouldn't happen, but this seems not to be the case.

What is the proper way to run multiple test classes? And so that I can see at a glance if any of the hundreds of unit tests I have have failed.

Aucun commentaire:

Enregistrer un commentaire