Unit Tests

The objective of unit tests is to test the functionality of a single module isolated from the system and other functional units.

• Test functionality of single software unit or module
• May require dependencies of other modules that come before in the compilation order
• Shouldn't require to run ACS
• Shouldn't depend on functional units / modules that come later in the compilation order

There are frameworks for the different programming languages and shell scripting. ACS comes with:

• shunit: Shell (sh, bash, etc.) unit testing framework
• unittest: Official Python unit testing module
• jUnit: Java unit testing library
• cppunit: C++ unit testing library

Besides these, each project is free to include their own unit testing libraries, packages, frameworks, etc.

Integration Tests

Integration tests go one step beyond, they allow to test functionality in the interactions of different functional units and can include using a partial or complete ACS runtime environment.

• Test functionality of multiple software units or modules
• Would probably require several dependencies from modules that come after in the compilation order or even that come from different projects
• Can make use of ACS

TAT (Tool for Automated Testing)

TAT is a set of Tcl-based scripts that allow to run a set of tests in a generic way

• TestList(.lite): The main TAT file containing all the test suite details
• Environment: Loads environment variables to be used by the test
•  Prologue
  • General "set-up" script to be called before all tests
  • Per-test "set-up" script to be called before execution of the test
• Epilogue
  • General "tear-down" script to be called at the end of all tests
  • Per-test "tear-down" script to be called at the end of the execution of the test
• Test
  • Command or list of commands to be executed as part of a single test
  • Special commands that can be added to the test, such as sleep to give a delay between commands
  • Commands can be concatenated in a sequential order, or let them run in parallel (for sender/receivers for example)
• Reference file
  • The tests can be of any kind, for instance unit tests, but also integration tests
  • A way to check the test results of any kind of test is to analyze their output
  • The reference file has all the output of each command run during a test; each command is stored in a different output level (1, 2, 3, etc.) depending on the order of execution
• Grep and sed files
  • TestList.grep: Is a file where each line is filtered out from the output using grep regular expressions before comparing to reference file
  • TestList.sed: Is a file where each line is used to modify the output according to the sed regular expression before comparing to reference file

Quick TAT example with shunit

Reporting TAT results as junit.xml – Example in Jenkins