6.0.0-git
2019-12-07
Last Modified 2019-12-01 by Ralf Lang (B1 Systems GmbH)

Anatomy of Horde Unit Tests

Filesystem structure of a test suite

The following tree shows a hypothetical arrangement of all element types that you might find in a Horde package:

Xyz/
`-- test
    `-- Horde
        `-- Xyz
            |-- AllTests.php
            |-- Autoload.php
            |-- bootstrap.php
            |-- fixtures
            |   `-- data.xml
            |-- Integration
            |   `-- IntegrationTest.php
            |-- phpunit.xml
            |-- Server
            |   |-- conf.php
            |   |-- conf.php.dist
            |   `-- ServerTest.php
            |-- SomeTest.php
            |-- Stub
            |   `-- Object.php
            |-- TestCase.php
            `-- Unit
                `-- UnitTest.php

We will go through the different elements below but lets first look at the hierarchy from the top level directory to the level of the AllTests.php file.

The top level directory Xyz represents a horde git repository.

The majority of the test suites can be found in the framework libraries but several applications have their own test suites as well. Whether a framework package or an application: In both cases the test directory in the top-level directory of the component will contain the tests.

Below the test directory the code hierarchy of the component will usually be mirrored.

For a hypothetical framework package with name Horde_Xyz this means that where will be a directory Horde with a subdirectory Xyz. The latter one contains the tests. This matches the hierarchy in the lib directory where you will also find a Horde directory that usually contains a Xyz subdirectory. If not there will at least be an Xyz.php file.

For an application named xyz the test directory will only contain a directory Xyz.

This directory structure below test is just a convention and no functional requirement for the tests. At least at the moment. At some point the idea was that you would be able to unpack all PEAR package archives into one location without the file paths in the components conflicting with each other. But when we rely on PEAR for the installation now then the different test suites of package Horde_One and Horde_Two end up in tests/Horde_One and tests/Horde_Two respectively. So there is no chance for a conflict anyway. This might change at some point though. So we stick to the convention and ask anyone writing new test suites to do it as well.

AllTests.php

This file is a mandatory requirement for a Horde test suite. In almost every case you should use the boilerplate template found at Test/doc/Horde/Test/template/packagetest/AllTests.php.

bootstrap.php

This file is a mandatory requirement for a Horde test suite. In almost every case you should use the boilerplate template found at Test/doc/Horde/Test/template/packagetest/bootstrap.php. This file contains the code necessary to integrate the tests with the rest of the Horde suite (i.e. it sets up autoloading).

phpunit.xml

This file is a mandatory requirement for a Horde test suite. In almost every case you should use the boilerplate template found at Test/doc/Horde/Test/template/packagetest/phpunit.xml.

Autoload.php

This file is not required . It's purpose is to setup additional PHP autoloading so that all tests in the test suite automatically have access to all the classes required for executing the tests. The default Horde_Test_AllTests already loads a basic autoloading definition that works for most framework components; this file should only be used if additinal autoloading is needed.

This file will be needed if you desire to autoload any helper class contained in the Test framework, since the default autoloader does not account for these files.

The content of Autoload.php

If you derive your test cases from a central test case definition you should load this one in Autoload.php as well:

/** Load the basic test definition */
require_once dirname(__FILE__) . '/TestCase.php';

Sometimes it makes sense to pull in the definition of test helpers that may be used throughout the test suite. They are usually not available via autoloading and need to be pulled in explicitly:

/** Load stub definitions */
require_once dirname(__FILE__) . '/Stub/ListQuery.php';
require_once dirname(__FILE__) . '/Stub/DataQuery.php';

Autoload mappings for applications

In case you test a horde application that may have special class name mappings when autoloading, you can define this via the Horde_Test_Autoload::addPrefix() call:

Horde_Test_Autoload::addPrefix('IMP', dirname(__FILE__) . '/../../lib/');

The class IMP_Imap would be loaded from dirname(__FILE__) . '/../../lib/Imap.php' in the example above.

Running the test suite

The Horde_Test module must be installed on your system.

From the command line, navigate to the base of an application/package test directory (should be located in the tests/ directory at the base of the application - navigate to the level of the directory containing the AllTests.php file).

To run all tests, issue the command:

php AllTests.php

You can also run individual tests by issuing (replace /usr/share/php with the path to PEAR):

/usr/share/php/Horde/Test/vendor/phpunit/phpunit/phpunit [SubDirectory1/SubDirectory2/]UnitTest.php

Attention

Using phpunit AllTests.php will cause autoloading features to fail.

Remember to run AllTests.php with php and not phpunit.

For single or individual tests use phpunit.

In a developer checkout, unit tests can also be run by calling <code>../components/bin/horde-components qc</code> from the app's root dir.

Unit testing a simple library

I call a library simple to test which does not do IO (db, downloads, interact with devices), does not apply side effects to its input arguments and does not depend on global state.

This is quite straight forward.
For example, look at [Horde_Argv library's test suite].

The logic is easy to test as all the classes get their complete state from input to methods or the constructor. You can just insert what you want to test and look at the results. Some very minimal test setup is done in the base class Horde_Argv_TestCase from which the actual test cases are derived. Each test case may have a setUp method before the individual tests are run. This method sets up the environment with the unit under test. Look at the ParserTest. In setUp, the parser is configured with commandline switches it should recognize. Then, the individual tests interact with the parser. Individual assertions which are non-trivial and reused can be factored out into helper methods like assertRemoved.

Not so simple libraries

It gets more complicated the more libraries either mix logic with data retrieval, IO, side effects or if they are essentially adapters to backends.
For example, a lot of code in Horde_Core is essentially glue between standalone libraries to provide easy to use wrappers and derived classes for horde applications.

database access

File IO, streams, internet

Testing horde applications

Unit testing well definded subsystems of applications

Integration tests / end to end tests