PyUnitHowto - SeaPIG

Unit testing Python programs: PyUnit / unittest module HOWTO

What is this document?

This is meant to provide a quick introduction and "how-to" guide for PyUnit, Python's unittest unit testing module. This is not meant as a definitive guide; for more information see the Python Reference manual, or the other resources listed below.

Why do unit testing?

Before writing code, it forces you to detail your requirements in a useful fashion.
While writing code, it keeps you from over-coding. When all the test cases pass, the function is complete.
When refactoring code, it assures you that the new version behaves the same way as the old version.
When maintaining code, it helps you cover your ass when someone comes screaming that your latest change broke their old code. ("But sir, all the unit tests passed when I checked it in...")

(From the online book DiveIntoPython.)

What Is PyUnit?

PyUnit is a framework that makes it easy to write automated test suites in Python. It's based on JUnit, the definitive Java unit testing framework, which is based on Kent Beck's Smalltalk unit testing framework.

PyUnit makes it easy to write test cases that test a unit of functionality like a single method, and also makes it easy to group these testcases into test suites that would test whole classes or functional blocks. Because the tests are automated, it's easy to run them whenever you change your program...

PyUnit comes included as the unittest module in Python 2.1 and above.

Overview

PyUnit provides three main classes for us to use. You will spend most of your time using unittest.TestCase. unittest.TestSuite is used occasionally to group TestCase objects together. unittest.TextTestRunner is used to run the test suite, and once you have your framework running, you probably won't have to look at it again.

unittest.TestCase

This is the work horse that we will be using quite a bit. Here's some code:

  class CaseCheck(unittest.TestCase):                  
    def testToRomanCase(self):                       
        """toRoman should always return uppercase""" 
        for integer in range(1, 4000):               
            numeral = roman.toRoman(integer)         
            self.assertEqual(numeral, numeral.upper())     

    def testFromRomanCase(self):                     
        """fromRoman should only accept uppercase input"""
        for integer in range(1, 4000):               
            numeral = roman.toRoman(integer)         
            roman.fromRoman(numeral.upper())         
            self.assertRaises(roman.InvalidRomanNumeralError,
                              roman.fromRoman, numeral.lower()) 

  if __name__ == "__main__":
      unittest.main()

(From the online book DiveIntoPython.)

Here are a couple of things to note:

Make a class to hold a group of individual test cases.
The class derives from unittest.TestCase.
Make a new function for each test. Each test case should only test one thing!
Each test function name should start with 'test'.
If you include a docstring for the function, the first line will be printed if when there is an error. It's a simple way to document your test code.
Include a way to run this test case from the command line.
unittest.TestCase provides its own ways to assert errors- use them! Don't raise assertions yourself.
unittest.main() looks at all the classes that inherit from TestCase or TestSuite, and runs those tests cases.

Things not shown here:

setUp() and tearDown() methods provide a way to do setup or housekeeping before and after the test case is run. tearDown() is only run if setUp() succeeds (doesn't raise an exception). The test program treats errors in these methods as regular errors.
There are many ways to test error conditions, including fail(), failException(), failUnless(), failNotEqual(), and failIf().

unittest.TestSuite

Note that you probably don't have to bother with this if you use the TestRunner.py program below!

  def suite():
     suite = unittest.TestSuite()
     suite.addTest(WidgetTestCase("testDefaultSize"))
     suite.addTest(WidgetTestCase("testResize"))
     return suite

TestSuite objects are containers for TestCases and other TestSuites. Since TestSuites can hold other TestSuites, you can have a hierarchy of tests.

There are two main methods, both just add tests to the TestSuite container:

addTest() adds an object to the TestSuite. Pass in an object derived from TestCase or TestSuite.
addTests() adds a list of objects to the TestSuite. Pass in a list of objects derived from TestCase or TestSuite.

unittest.TextTestRunner

Again, you probably won't have to bother with this if you use the TestRunner.py program, below!

    suite = unittest.TestSuite()
    suite.addTest(DataStoreTest.Test())

    ttr = unittest.TextTestRunner()
    results = ttr.run(suite)

TextTestRunner is a way of presenting the results nicely in a text terminal window. You just pass in an object derived from TestSuite to its constructor, then call run, and it does the rest.

TestRunner.py - An automated unit test system

Mark Pilgrim gives a short but very useful program to run all your tests automatically in Dive Into Python Chapter 7. If your testfiles all end in "test.py", this program will import all the classes in these files, make a TestSuite out of the TestCases or TestSuites they contain, and start unittest with that TestSuite. I call this program 'TestRunner.py':

   """Regression testing framework
   This module will search for scripts in the same directory named
   XYZTest.py.  Each such script should be a test suite that tests a
   module through PyUnit.  (As of Python 2.1, PyUnit is included in
   the standard library as "unittest".)  This script will aggregate all
   found test suites into one big test suite and run them all at once.
   """

   import unittest

   import sys, os, re, unittest

   def regressionTest():
   path = os.path.split(sys.argv[0])[0] or os.getcwd()
   files = os.listdir(path)
   test = re.compile("test.py$", re.IGNORECASE)
   files = filter(test.search, files)
   filenameToModuleName = lambda f: os.path.splitext(f)[0]
   moduleNames = map(filenameToModuleName, files)
   modules = map(__import__, moduleNames)
   load = unittest.defaultTestLoader.loadTestsFromModule
   return unittest.TestSuite(map(load, modules))

   if __name__ == "__main__":
      unittest.main(defaultTest="regressionTest")

unittestgui.py - GUI interface to PyUnit

This is a small Tkinter GUI written in Python, similar to JUnit's GUI. If you like GUIs, this gives a way to hit a button and run all your tests. It shows the progress of the tests with a moving bar, which stays green as long as the tests pass, and goes red as soon as one test fails. It gives you a list of the tests that failed, and you can double-click the test to get the traceback detail.

Usage notes: you only get this if you download the PyUnit source, it doesn't come with Python. I could only get it to work correctly if I copied the unittestgui.py file to my test directory and ran it from there; for some reason, it doesn't have the Python import path set right unless you do this.

If you use the program TestRunner.py above, and you have the it, your tests, and the unittestgui.py file in your current directory, you can start the GUI like this:

   $ python unittestgui.py TestRunner.regressionTest &

(You have to give unittestgui.py the name of an object that will return a TestSuite instance.)

Resources

Python's unittest module: http://docs.python.org/lib/module-unittest.html
PyUnit homepage: http://pyunit.sourceforge.net/
DiveIntoPython book chapters on Unit testing:
- http://diveintopython.org/roman_divein.html
- http://diveintopython.org/regression_divein.html
Quixote.unittest (another unit testing framework): http://www.mems-exchange.org/software/quixote/unittest.html
JUnit (lots of good general info here): http://junit.sourceforge.net/
Good article on why to use automated unit testing: http://junit.sourceforge.net/doc/testinfected/testing.htm

Copyright and license info for this document

Permission is granted to copy, distribute, and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.

For more information see http://www.gnu.org/copyleft/fdl.html