Testing Python Applications

In this article I will give a brief introduction to testing Python 3 applications. Testing can be done with standard Python libraries or external modules installed through PyPI.

This article is not focused on TDD (Test Driven Development), I will only show you how you can test your code. If you want to do TDD, you can but it is not a must.

Why writing tests?

You may think about development as a one-time job where you have to be as fast as you can and Python helps you to gain speed. However in the long-run you need more quality than speed and you can ensure quality with tests.

Naturally you could test your application manually after every time you change a feature. But after doing this 3-4 time you're bound to get fed-up and stop testing your application, telling yourself "I know what I'm doing". And this is the time when bugs start creeping in...

However you can take counter-measurements when you write tests. Testing has to be done even if you don't like it and it will bring you fruits if you do it right. In this article I will help you to start writing tests and give you different options you can choose from.

Module tests

You know how you can write modules in Python. To test them the simplest way is to include the tests in the module itself. This seems a bit cumbersome but it is something worth trying because after you have your tests in your module you can simply start your module from the command line and see if the tests work.

Naturally for bigger modules this kind of testing won't be ideal. For such purposes a better solution is required but for a simple task and a simple module this is the easiest way.

The solution is to put the test code inside the __main__ part of your application. Let's take a look at this basic factorial calculation example:

def factorial(n):
    """ Recursive factorial calculation """
    if n == 1:
        return 1
    return n + factorial(n-1)

As you can see it has a basic recursion to calculate the factorial of a number. If you have good eyes for this you will find the errors in the code.

Because this example is very simple you can run it some times and see if the results are as expected or not. That's fine until someone changes the code after you tested it and this is when the problems start. You will insist, that your code is perfect because you have tested it.

Now let's see how you can write tests to ensure that your module works as you expect it.

def factorial(n):
    """ Recursive factorial calculation """
    if n == 1:
        return 1
    return n + factorial(n-1)

if __name__ == '__main__':
    if factorial(5) == 120:
        print('Everything is OK, the module works as it is expected!')
    else:
        print('The tests do not seem to work. Something is wrong with the module!')

If you run the code you get the following result:

The tests do not seem to work. Something is wrong with the module!

Yes, the code has problems but as you may think this solution for testing is not the best -- this only ensures that the code works or it does not. And if you want to make more complicated test cases for a more complicated module it would be blown-up, not to speak of that you would add functions for testing which would be available when someone is using your module.

There has to be another solution!

Doctest

This module is a simpler way to add tests to your code -- as it's name suggests it contains tests in the documentation of the module-methods. Yes, you read that right: you include the test cases in the function documentation and you can execute the tests.

This solution makes tests easier because you only start the testing in the __main__ block of your code and all testing is done in the documentation. Let's change the previous example to using doctest:

def factorial(n):
    """ Recursive factorial calculation
    >>> factorial(1)
    1
    >>> factorial(2)
    2
    >>> factorial(3)
    6
    >>> factorial(5)
    120
    """
    if n == 1:
        return 1
    return n + factorial(n-1)

if __name__ == '__main__':
    import doctest
    doctest.testmod()

As you can see, I included some example runs of the factorial function into the docstring of the function and changed the __main__ block to execute the testing mode of the docstring module which is run the example runs and compares them with the expected output: >>> factorial(1) is the example call which has to result in 1.

When you run this example code you will get the following results from docstring:

**********************************************************************

File "factorial.py", line 5, in __main__.factorial
Failed example:
    factorial(2)
Expected:
    2
Got:
    3

**********************************************************************

File "factorial.py", line 9, in __main__.factorial
Failed example:
    factorial(5)
Expected:
    120
Got:
    15

**********************************************************************

1 items had failures:
   2 of   4 in __main__.factorial
***Test Failed*** 2 failures.

As you can see, with this approach you can define more tests and get more detailed reports. The drawback again is that you have to write your tests inside the module and if you have some functions and you test it thoroughly then you may end up with a big file where 80% are tests. And sometimes this is not what you want.

But there is an even better way to test!

Unit tests

Here is the next solution-- unit testing. It is independent because you write your tests in separate test files and you do not mix your code together with irrelevant tests.

Unit testing is a basic block of tests. As it's name already mentions it tests a unit of code.

The module to use for unit testing is called unittest. Now let's rewrite the example application to have a separate test file and the module itself. For this, move the code for the module into the factorial.py file:

def factorial(n):
    """ Recursive factorial calculation """
    if n == 1:
        return 1
    return n + factorial(n-1)

As you can see the module only contains the factorial function and nothing else, no __main__ block either.

The tests would look like this:

import unittest
from factorial import factorial

class FactorialTest(unittest.TestCase):

    def testFactorial(self):
        self.assertEqual(factorial(1), 1)
        self.assertEqual(factorial(2), 2)
        self.assertEqual(factorial(3), 6)
        self.assertEqual(factorial(5), 120)


if __name__ == '__main__':
    unittest.main()

As you can see, the initiator of the test execution is still in the __main__ block but you can write your tests as you like, until you extend the unittest.TestCase class and have all test methods as instance methods of the extension class.

If we run this test file we get the following result:

F
======================================================================
FAIL: testFactorial (__main__.FactorialTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "factorial_test.py", line 8, in testFactorial
    self.assertEqual(factorial(2), 2)
AssertionError: 3 != 2

----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (failures=1)

As you can see, you get the first assert in error and the test stops. Naturally there is a way to get all the tests run at once: you need to separate the tests into their own methods.

Other libraries

The modules discussed above are all included with Python 3.5.

I know currently about two modules for testing Python code: node and py.test.

I introduced the nose library in a previous article. You can refer there if you want to get an introduction.

An introduction of py.test if available in another article.

A detailed introduction of these libraries would not fit into the framework of this introductory article but if you are interested I will introduce them in more detail in a follow-up article.

Conclusion

This was a very brief introductory article on testing with Python 3.5. As you have seen you can utilize built-in modules and you do not need external libraries to write and run tests.

I have shown you different approaches and in the end we arrived at the cleanest solution: unittest where you can write your tests in separate files and leave your modules clean. Naturally the section about unittest's features is not complete, it only shows the basic usage of the module but it gives you a good start if you want to explore the path of testing.