Further information#

Why do we write functions in tests?#

In Tutorial we wrote all the tests using assert statements inside of functions. **Technically this is not necessary as you could write a single script with all the assert statements one after the other.

This is not recommended: by using functions we directly have a place to document the test (in the docstring) and the tests themselves are modularised. Furthermore, this is actually how to write the tests when using a more appropriate way of running tests as described in Is there a more efficient way to run tests?.

Is there a more efficient way to run tests?#

Writing tests as a script and directly running them has one immediate problem: once the first assert statement fails the rest of them are not run.

There is a Python library for running tests called pytest https://docs.pytest.org/. Importantly this is not part of the anaconda distribution and needs to be installed.

To do this type the following at the command line:

python -m pip install pytest

This uses a python tool called pip which lets you install libraries from the Python Package Index (PyPi). Figure Installing pytest from the Python Package Index. shows this.

../../../_images/main14.png

Fig. 25 Installing pytest from the Python Package Index.#

Once we have pytest installed we can actually find all files starting with test_ in the name and all functions in those files starting with test_ and run them with the following command:

python -m pytest

Figure Running pytest shows this.

../../../_images/main15.png

Fig. 26 Running pytest#

It is also possible to use pytest to test the documentation by running the doctests:

python -m pytest --doctest-glob="README.md"

Figure Running pytest with doctests shows this.

../../../_images/main16.png

Fig. 27 Running pytest with doctests#

When you tests pass or fail you will get more information and there are also a number of plugins that exists that all you to extend the capabilities of pytest.

What should be tested?#

The short answer to this is that all software should be tested and that software is compromised of documentation and code.

Note that it is often not sufficient to test a function in a single way. For example, if we have a function that does two different things depending on the parity of some input:

def feedback_on_guess(guess, chosen_number):
    """
    Returns whether or not a guess is:

    - Larger than  a chosen_number
    - Smaller than a chosen_number
    - Equal to a chosen number
    """
    if guess < chosen_number:
        return "Guess is lower than chosen number"
    if guess > chosen_number:
        return "Guess is larger than chosen number"
    return "Guess is equal to chosen number"

In this case we would need to write at least 3 tests that check the 3 behaviours.

In practice there might be some functionality that is not tested but this should be made clear and explicit and documentation as to why should be written.

There are a number of resources from which to learn how to use pytest to its full extent and [Oliveira, 2018] is a further great resource.

Why do we need doc tests?#

The purpose of doctests is to ensure that the code written in documentation is correct.

It is important to not use doctests to test the functionality of the code: this risks making the documentation unclear.

What is test driven development?#

Test driven development is the development process of writing the test before you write the code. Whilst this might seem counter-intuitive it is in fact a strong approach to writing robust code efficiently.

In practice the process is as follows:

  1. Write a test for some new functionality.

  2. Run the tests to confirm that it fails (as the functionality is not yet written).

  3. Write the functionality.

  4. Run the test.

  5. Modify the test and/or the functionality

Steps 4 and 5 might be repeated many times.

A good overview of test driven development is given in [Percival, 2014].