Skip to content

Commit

Permalink
Expose resemble's options in config file and don't scaleToSameSize by…
Browse files Browse the repository at this point in the history 
… default (#176)

* Set scaleToSameSize to false when comparing images.

We should not scale to the same size when comparing images because if they have different sizes, they are not equal by definition. The scaling introduces distortions that make it hard to find the real problem.

* Update reference image.

* Update reference image.

* Print diff percentage of images in error messages to help fine tune threshold config.

* Expose resemble options in config file, so it can be tweaked as needed.

* Add resembleOptions config to the readme file.
  • Loading branch information
@rafaeldalsanto
rafaeldalsanto authored May 10, 2023
1 parent 5c4bfd5 commit 3af271f
Show file tree
Hide file tree
Showing 5 changed files with 85 additions and 23 deletions.
25 changes: 13 additions & 12 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -113,18 +113,19 @@ Now let's talk about **how to test your application!**.
## Setting up
After going through all the steps of the getting started section, you are able to configure robot-eyes to your needs. `robot-eyes.json` is the file that contains the general settings, which are valid for all tests. Here's a list of the available options:

| Property | Description | Default value
| ------------- |-------------| -------------|
| `baseURL` | Link to the main page of your application (shouldn't contain '/'). Example: https://github.com | null |
| `paths.testImages` | relative path where temporary test image files will be saved. Example './test_images/' | './images/test_images' |
| `paths.diffImages` | relative path where temporary diff image files will be saved. Example './diff_images/' | './images/diff_images' |
| `paths.referenceImages` | relative path where the baseline/reference images will be saved. Example './reference_images/' | './images/reference_images' |
| `viewports` | Array of objects containing the width and height that will be tested | `{width: 1920, height: 1080}` |
| `timeout` | Mocha timeout | 40000 |
| `headless` | Chrome browsing mode. Is important to know that, headless and headed generate different images. | true |
| `threshold` | Maximum percentage of different pixels for the test to pass | 0.01 |
| `waitForResourceTimeout` | Maximum time (in ms) to wait for baseURL to be available | 60000 |

| Property | Description | Default value
| ------------- |-------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `baseURL` | Link to the main page of your application (shouldn't contain '/'). Example: https://github.com | null |
| `paths.testImages` | relative path where temporary test image files will be saved. Example './test_images/' | './images/test_images' |
| `paths.diffImages` | relative path where temporary diff image files will be saved. Example './diff_images/' | './images/diff_images' |
| `paths.referenceImages` | relative path where the baseline/reference images will be saved. Example './reference_images/' | './images/reference_images' |
| `viewports` | Array of objects containing the width and height that will be tested | `{width: 1920, height: 1080}` |
| `timeout` | Mocha timeout | 40000 |
| `headless` | Chrome browsing mode. Is important to know that, headless and headed generate different images. | true |
| `threshold` | Maximum percentage of different pixels for the test to pass | 0.01 |
| `waitForResourceTimeout` | Maximum time (in ms) to wait for baseURL to be available | 60000 |
| `resembleOptions`| Config options for [Resemble.js](https://github.com/rsmbl/Resemble.js). This object is passed directly to Resemble's `compareImages` function. You can set the parameters it will use to compare the images, for instance, change the default `ignore: "nothing"` to `ignore: "antialiasing"` to make it more tolerant to small differences. | <pre><code>{</code><br><code> ignore: 'nothing',</code><br><code> output: {</code><br><code> largeImageThreshold: 0,</code><br><code> transparency: 0.3</code><br><code> }</code><br><code>}</code></pre> |
### Example robot-eyes.json file
```javascript
{
Expand Down
11 changes: 2 additions & 9 deletions compare/compareImages.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,21 +12,14 @@ const compareImages = (config, fileName, viewport) => {
const result = await resembleCompare(
referenceImage,
testImage,
{
ignore: 'nothing',
scaleToSameSize: true,
output: {
largeImageThreshold: 0,
transparency: 0.3
}
}
config.resembleOptions
)

if (result.rawMisMatchPercentage > config.threshold) {
await createFolder(`${config.paths.diffImages}/${fileName}`)
const diffImagePath = getFilePath(config.paths.diffImages, fileName, viewport)
fs.writeFileSync(diffImagePath, result.getBuffer())
reject(new Error(`Images are not the same. See difference at ${diffImagePath}.`))
reject(new Error(`Images are not the same (${result.rawMisMatchPercentage}%). See difference at ${diffImagePath}.`))
} else {
resolve()
}
Expand Down
26 changes: 24 additions & 2 deletions core/getConfig.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ const getConfig = () => {
const args = minimist(process.argv)
const configFile = fs.readFileSync(process.cwd() + '/robot-eyes.json')
const userConfig = JSON.parse(configFile)
const config = Object.assign({
const config = mergeDeep({
baseURL: null,
paths: {
testImages: './images/test_images',
Expand All @@ -22,7 +22,14 @@ const getConfig = () => {
timeout: 40000,
headless: true,
threshold: 0.01,
waitForResourceTimeout: 60000
waitForResourceTimeout: 60000,
resembleOptions: {
ignore: 'nothing',
output: {
largeImageThreshold: 0,
transparency: 0.3
}
}
}, userConfig)
Object.keys(config.paths)
.forEach(v => {
Expand All @@ -36,4 +43,19 @@ const getConfig = () => {
return config
}

const mergeDeep = (obj1, obj2) => {
const result = {};
Object.assign(result, obj1);

for (const prop of Object.keys(obj2)) {
if (result.hasOwnProperty(prop) && typeof obj2[prop] === 'object' && !Array.isArray(obj2[prop])) {
result[prop] = mergeDeep(result[prop], obj2[prop]);
} else {
result[prop] = obj2[prop];
}
}

return result;
}

module.exports = getConfig
42 changes: 42 additions & 0 deletions core/tests/getConfig.spec.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,48 @@ describe('getConfig', function () {
compareResolvedAndUnresolvedPath(config.paths.referenceImages, paths.referenceImages)
})

it('default resemble options', function () {
createEmptyConfigFile()

const config = getConfig()

expect(config.resembleOptions.ignore).to.eql('nothing')
expect(config.resembleOptions.output.largeImageThreshold).to.eql(0)
expect(config.resembleOptions.output.transparency).to.eql(0.3)
})

it('default resemble options should be overrided by config file', function () {
const resembleOptions = {
ignore: 'antialiasing',
output: {
largeImageThreshold: 1000,
transparency: 0.5
}
}
createConfigFile({ resembleOptions })

const config = getConfig()

expect(config.resembleOptions).to.eql(resembleOptions)
})

it('should override only the specified resemble options preserving defaults', function () {
const resembleOptions = {
output: {
transparency: 0.5,
errorType: "movement"
}
}
createConfigFile({ resembleOptions })

const config = getConfig()

expect(config.resembleOptions.ignore).to.eql('nothing')
expect(config.resembleOptions.output.largeImageThreshold).to.eql(0)
expect(config.resembleOptions.output.transparency).to.eql(0.5)
expect(config.resembleOptions.output.errorType).to.eql("movement")
})

const compareResolvedAndUnresolvedPath = (resolved, unresolved) => {
expect(resolved).to.satisfy(path => path.endsWith(unresolved.replace('.', '')))
}
Expand Down
4 changes: 4 additions & 0 deletions integrationTests/robot-eyes.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,9 @@
"testImages": "./images/test",
"diffImages": "./images/diff",
"referenceImages": "./images/reference"
},
"threshold": 0.02,
"resembleOptions": {
"ignore": ["antialiasing"]
}
}

0 comments on commit 3af271f

Please sign in to comment.