writing-tests.md

Table-driven Test Cases

These are some thoughts on how to approach template-based generation of table-driven tests. Using the approach, the tests would be described using a table of expressions, with each test case defined by one row. The columns would provide values that could be substituted into a template.

As an example, consider a series of tests for a simple "factorial" problem:

n	expected
0	1
1	1
2	2
3	6

Class Template

We can use a template to define the "container" structure for all the tests. For Java, it might look like this:

import junit.framework.TestCase;
import static org.junit.Assert.*;
import java.io.*;
import org.junit.*;
{{imports}}

public class {{class_name}}Test extends codeworkout.CodeWorkoutTest
{
    {{tests}}
}

Here, {{imports}} could be an optionally provided list of additional custom imports that defaults to an empty string if not provided. The {{class_name}} could default to a specific value based on the tool, or could be set for a specific problem or test suite (a default of "Answer" as the value for the class_name is a good choice).

The {{tests}} would then be a text string generated by concatenating together all the test definitions. A separate template for a single test case could be mapped across all rows in the tabular description, concatenating them together to get {{tests}}.

Test Case Template

A separate template could be used for each test case, which would render as a separate method in XUnit-style frameworks (like JUnit, pyunit, CxxTest, etc.). For Java, we might use a template like this:

@Test
public void test_{{test_case_id}}()
{
  {{given_clauses}}
  {{when_clauses}}
  {{then_clauses}}
}

The {{given_clauses}} could be empty, or provided as part of the test description. A reasonable default might be something like:

{{class_name}} subject = new {{class_name}}();

The {{when_clauses}} might default to an empty string. However, if the data table includes a column called "stdin", we could instead use something like this for {{when_clauses}}:

setSystemIn("{{stdin}}");

The {{then_clauses}} might default to a simple equality claim:

assertEquals(
  "{{negative_feedback}}",
  {{expected}},
  {{actual}});

Here, {{expected}} might come directly from a column in the table. {{negative_feedback}} would also come from the table, if specified, or default to an empty string. The {{actual}} expression might come from the table, or default to something like:

subject.{{method_name}}({{parameters}})

If the input table defines "stdout" as a column, we could extend the default value for {{then_clauses}} with an extra statement:

assertEquals(
  "{{negative_feedback}}",
  "{{stdout}}",
  systemOut().getHistory());