Characterization Tests for Convert

[gmjgeek]

Support characterization tests for the convert process.

When characterization tests first run, they convert one or more test apps and save the result as a "snapshot." On subsequent runs, they compare their output with the saved snapshot, throwing an error if the two do not match.

Characterization tests are useful when refactoring, since the goal of refactoring is to keep the program's output unchanged.

Features

"Code without tests is broken by design." - Jacob Kaplan-Moss

• View test coverage with v8
• Support characterization tests for convert

Commands

Run each command using npm run <command>

Command	Description
test	Run all tests except characterization tests
test:snapshot	Run all tests including characterization tests. The first run takes a snapshot of the convert output, and subsequent runs test against this snapshot.
test:new-snapshot	Delete existing snapshots so they can be regenerated. Then run all tests.

Note: Snapshots are not source controlled.

To view the code coverage for the tests, append -- --coverage to any of the commands above, like so:

npm run test -- --coverage

Example Usage

Suppose you are refactoring convertBooks.ts. You may consider the following workflow:

• Check test coverage
  • Run the characterization tests with coverage enabled: npm run test:new-snapshot -- --coverage
  • Open the local file coverage/index.html in your browser. Select convert > convertBooks.ts
  • Inspect the code coverage
• If significant portions of convertBooks.ts are untested,
  • Add features to the test app in convert/test/characterization/test_app1/input that will cause the code to run
  • Alternatively, create a new test app (test_app2/input) that will cause the code to run. Add a test case to characterization.test.ts that converts this app, following the example from test_app1.
  • Rerun tests and ensure the code is covered.
• Make refactoring changes.
• Rerun tests to ensure the output has not changed.

What About Unit Tests?

Unit tests are better than characterization tests in several ways. Because they test isolated units of code, they are usually faster and less fragile than characterization tests. They also play a key role in Test-Driven Development (TDD), a coding style that that improves code quality through short development cycles and fast feedback.

However, unit tests are hard to write for existing code. For convert, adding unit tests would require significant refactoring. In the meantime, characterization tests offer quality control to facilitate refactoring.

Viewing Test Coverage

When enabled, v8 reports test coverage through the terminal and through HTML. The following screenshot shows part of the terminal output from npm run test -- --coverage.

To view the HTML report, open the local file coverage/index.html in a browser. In Visual Studio Code, select the file in the explorer tab and use the "Copy Path" command (Shift + Alt + C) to copy the absolute path. Paste this path into a browser, as shown below.