Exercise: What is a bug?
Software testing is usually defined as the process of finding and eliminating bugs in software. Take a minute and write down your answer to the question “What is a bug?”
- Automated vs. Manual testing
- Regression bugs: bugs that appear later, breaking previously working code.
Test Cases and Automating Tests
- What is a test case?
A single, logical piece of functionality with well-defined inputs and expected outputs. It must be able to be exactly matched as “success” or “failure”.
- Automated test cases: Easy to run over and over
- Testing harness: runs a collection of test cases and produce output reports
- Assert / should
Types of Tests
- Level 0: Basic Functionality
- Level 1: Full Range of Functionality
- Level 2: Error conditions
- Level 3: Performance and Scalability
- In python (or your favorite language), write a “Fibonnaci” function that accepts an input
nand prints out the first
- Write a function that conducts a level 0 test.
- Write a set of functions that conduct as comprehensive set of level 1 tests.
- Write a set of functions taht conduct a set of level 2 tests.
- Write a function that measures the performance of your fibonnaci function (level 3 tests).
def fib(n): if type(n) is not int: raise TypeError() if n <= 0: raise IndexError() if n <= 2: return 1 else: return fib(n-2) + fib(n-1)
Example Test Code
import unittest import datetime from rick import fib class TestFib(unittest.TestCase): # *** Level 0 tests *** def test_fib4(self): self.assertEqual(fib(4), 3) def test_fib5(self): self.assertEqual(fib(5), 5) # *** Level 1 tests *** def test_fib1(self): self.assertEqual(fib(1), 1) def test_fib2(self): self.assertEqual(fib(2), 1) def test_fib3(self): self.assertEqual(fib(3), 2) # *** Level 2 tests *** def test_fibneg5(self): self.assertRaises(IndexError, fib, -5) def test_fib0(self): self.assertRaises(IndexError, fib, 0) def test_fibhi(self): self.assertRaises(Exception, fib, "hi") def test_fib2str(self): self.assertRaises(Exception, fib, "2") def test_fib2point(self): self.assertRaises(Exception, fib, 2.0) def test_fib2point5(self): self.assertRaises(Exception, fib, 2.5) # *** Level 3 tests *** def test_fibperf(self): start = datetime.datetime.now() out = fib(40) end = datetime.datetime.now() self.assertLess((end-start).total_seconds(), 10) if __name__ == '__main__': unittest.main()
Testing an Application
- Unit tests
- Integration tests
Choose an application you wrote recently. Write an integration test case – a script for a user going through a process, and the detailed output expected.
- Clean environment for each test (setup and teardown)
- Separating units: stubs and mocks
- Test plans
- Test driven development / behavior driven development
- Continuous Integration
- Code coverage