Turns:
Test Case '-[ProjectTests.TimelineSnapshotTests testTimelineView]' started.
2016-10-09 13:24:47.405 Project[66668:4288774] If you have Kaleidoscope installed you can run this command to see an image diff:
ksdiff "/Users/user/Library/Developer/CoreSimulator/Devices/D44D7153-EC5A-4F31-BAAE-1879B8C830E7/data/Containers/Data/Application/C2792144-5257-43B6-BD9D-9FFE0D1C5FD2/tmp/ProjectTests.TimelineSnapshotTests/[email protected]" "/Users/user/Library/Developer/CoreSimulator/Devices/D44D7153-EC5A-4F31-BAAE-1879B8C830E7/data/Containers/Data/Application/C2792144-5257-43B6-BD9D-9FFE0D1C5FD2/tmp/ProjectTests.TimelineSnapshotTests/[email protected]"
/Users/user/Documents/code/Project/ProjectTests/TimelineTest.swift:54: error: -[ProjectTests.TimelineSnapshotTests testTimelineView] : failed - Snapshot comparison failed: Optional(Error Domain=FBSnapshotTestControllerErrorDomain Code=4 "Images different" UserInfo={NSLocalizedFailureReason=image pixels differed by more than 0.00% from the reference image, FBDiffedImageKey=<UIImage: 0x600000487e40>, {375, 50}, FBReferenceImageKey=<UIImage: 0x608000097e80>, {375, 50}, FBCapturedImageKey=<UIImage: 0x60800009a810>, {375, 50}, NSLocalizedDescription=Images different})
Test Case '-[ProjectTests.TimelineSnapshotTests testTimelineView]' failed (0.029 seconds).
Test Suite 'TimelineSnapshotTests' failed at 2016-10-09 13:24:47.413.
Executed 1 test, with 1 failures (0 unexpected) in 0.029 (0.030) seconds
Into: Example HTML output
FBSnapshotTestCase is an excellent tool. Unfortunately, its output requires the use of expensive software and is not particularly useful for detecting what broke in a snapshot test whilst running CI. This tool will parse xcodebuild output, detect any failed snapshot tests, and generate human-consumable html output. This output is stored locally so your CI scripts will be able to host in any way they wish, or just store locally.
| Tool | Description |
|---|---|
| FBSnapshotTestCase | Outputs only a Kaleidoscope command, doesn't really do much beyond detecting a failure |
| ashfurrow/second_curtain | A fantastic ruby gem with the exact same purpose as snapdiff, who's only deficiency is that it forces an S3 upload without providing the raw output files for you to host by any other means or inspect locally |
| orta/Snapshots | No longer functions after Xcode deprecated plugins. |
xcodebuildcp ./build/Release/snapdiff /usr/local/bin/snapdiff
$ gem install rake
$ gem install xcpretty$ rake make installsnapdiff will accept input from stdin. This means you can either pipe from xcodebuild directly:
$ xcodebuild | snapdiff Or output to your home directory, with a custom filename
$ xcodebuild | snapdiff --output-dir=~/ --output-filename=failed-snapshots --colorOr output with debugging logs to see why the tool isn't working
$ xcodebuild | snapdiff --color --debugOr cat your build output log into snapdiff if you'd prefer to run snapdiff outside of your actual build process
$ xcodebuild > output.log
$ cat output.log | snapdiffIf you are using xcpretty, you must take care to pipe into snapdiff before xcpretty.
| Arg | Description | Default Value |
|---|---|---|
--output-dir |
Output directory for generated files | ./ |
--output-file |
The filename for the actual html file to be output | snapdiff-YYYY-MM-DD--hh-ss |
--debug |
Output debugging information | disabled |
--color |
Output debugging and stderr output with ANSI coloring | disabled |
--no-timestamp |
Disable output directory timestamp suffix. | timestamp enabled |
--help |
Print usage information |
Special thanks to:
- Facebook for creating FBSnapshotTestCase, an indispensible tool for reliable iOS development.
- Ash Furrow for creating second_curtain, a large inspiration for this project and an excellent tool.
- Orta for creating the excellent snapshots Xcode plugin.