-
-
Notifications
You must be signed in to change notification settings - Fork 201
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
ba73759
commit 198a684
Showing
9 changed files
with
429 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,68 @@ | ||
# Toxiproxy Module | ||
|
||
Testcontainers module for Shopify's [Toxiproxy](https://github.com/Shopify/toxiproxy). | ||
This TCP proxy can be used to simulate network failure conditions. | ||
|
||
You can simulate network failures: | ||
|
||
* between NodeJS code and containers, ideal for testing resilience features of client code | ||
* between containers, for testing resilience and emergent behaviour of multi-container systems | ||
* if desired, between NodeJS code/containers and external resources (non-Dockerized!), for scenarios where not all dependencies can be/have been dockerized | ||
|
||
Testcontainers Toxiproxy support allows resilience features to be easily verified as part of isolated dev/CI testing. This allows earlier testing of resilience features, and broader sets of failure conditions to be covered. | ||
|
||
## Install | ||
```bash | ||
npm install @testcontainers/toxiproxy --save-dev | ||
``` | ||
|
||
## Usage example | ||
|
||
A Toxiproxy container can be placed in between test code and a container, or in between containers. | ||
In either scenario, it is necessary to create a `ToxiProxyContainer` instance on the same Docker network. | ||
|
||
Next, it is necessary to instruct Toxiproxy to start proxying connections. | ||
Each `ToxiProxyContainer` can proxy to many target containers if necessary. | ||
|
||
A proxy is created by calling `createProxy` on the `ToxiProxyContainer` instance. | ||
|
||
The client connecting to the proxied endpoint then needs to use the exposed port from the returned proxy. | ||
|
||
All of this is done as follows: | ||
<!--codeinclude--> | ||
[Creating, starting and using the container:](../../packages/modules/toxiproxy/src/toxiproxy-container.test.ts) inside_block:create_proxy | ||
<!--/codeinclude--> | ||
|
||
!!! note | ||
Currently, `ToxiProxyContainer` will reserve 31 ports, starting at 8666. After this, trying to create a new proxy instance will throw an error. | ||
|
||
|
||
Having done all of this, it is possible to trigger failure conditions ('Toxics') through the `proxy.instance.addToxic<TPClient.TOXIC_TYPE>()` object: | ||
|
||
`TPClient` is the internal `toxiproxy-node-client` re-exported in this package. | ||
|
||
* `bandwidth` - Limit a connection to a maximum number of kilobytes per second. | ||
* `latency` - Add a delay to all data going through the proxy. The delay is equal to `latency +/- jitter`. | ||
* `slicer` - Slices TCP data up into small bits, optionally adding a delay between each sliced "packet". | ||
* `slowClose` - Delay the TCP socket from closing until `delay` milliseconds has elapsed. | ||
* `timeout` - Stops all data from getting through, and closes the connection after `timeout`. If `timeout` is `0`, the connection won't close, and data will be delayed until the toxic is removed. | ||
* `limitData` - Closes connection when transmitted data exceeded limit. | ||
|
||
Please see the [Toxiproxy documentation](https://github.com/Shopify/toxiproxy#toxics) and the [toxiproxy-node-client](https://github.com/ihsw/toxiproxy-node-client) for full details on the available Toxics. | ||
|
||
As one example, we can introduce latency and random jitter to proxied connections as follows: | ||
|
||
<!--codeinclude--> | ||
[Adding latency to a connection](../../packages/modules/toxiproxy/src/toxiproxy-container.test.ts) inside_block:adding_toxic | ||
<!--/codeinclude--> | ||
|
||
There is also a helper method to enable / disable the proxy (so you can simulate disconnections). This can also be done by calling the `proxy.instance.update` method, however | ||
you'll need to supply the upstream again and the internal listening port. | ||
|
||
<!--codeinclude--> | ||
[Enable and disable the proxy:](../../packages/modules/toxiproxy/src/toxiproxy-container.test.ts) inside_block:enabled_disabled | ||
<!--/codeinclude--> | ||
|
||
## Acknowledgements | ||
|
||
This module was inspired by the Java implementation, and under the hood uses the [toxiproxy-node-client](https://github.com/ihsw/toxiproxy-node-client). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
import type { Config } from "jest"; | ||
import * as path from "path"; | ||
|
||
const config: Config = { | ||
preset: "ts-jest", | ||
moduleNameMapper: { | ||
"^testcontainers$": path.resolve(__dirname, "../../testcontainers/src"), | ||
}, | ||
}; | ||
|
||
export default config; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,39 @@ | ||
{ | ||
"name": "@testcontainers/toxiproxy", | ||
"version": "10.16.0", | ||
"license": "MIT", | ||
"keywords": [ | ||
"toxiproxy", | ||
"testing", | ||
"docker", | ||
"testcontainers" | ||
], | ||
"description": "Toxiproxy module for Testcontainers", | ||
"homepage": "https://github.com/testcontainers/testcontainers-node#readme", | ||
"repository": { | ||
"type": "git", | ||
"url": "https://github.com/testcontainers/testcontainers-node" | ||
}, | ||
"bugs": { | ||
"url": "https://github.com/testcontainers/testcontainers-node/issues" | ||
}, | ||
"main": "build/index.js", | ||
"files": [ | ||
"build" | ||
], | ||
"publishConfig": { | ||
"access": "public" | ||
}, | ||
"scripts": { | ||
"prepack": "shx cp ../../../README.md . && shx cp ../../../LICENSE .", | ||
"build": "tsc --project tsconfig.build.json" | ||
}, | ||
"dependencies": { | ||
"testcontainers": "^10.16.0", | ||
"toxiproxy-node-client": "^4.0.0" | ||
}, | ||
"devDependencies": { | ||
"@testcontainers/redis": "^10.16.0", | ||
"redis": "^4.7.0" | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
export { ToxiProxyContainer, StartedToxiProxyContainer, CreatedProxy } from "./toxiproxy-container"; |
183 changes: 183 additions & 0 deletions
183
packages/modules/toxiproxy/src/toxiproxy-container.test.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,183 @@ | ||
import { ToxiProxyContainer, TPClient } from "./toxiproxy-container"; | ||
import { GenericContainer, Network } from "testcontainers"; | ||
import { createClient } from "redis"; | ||
|
||
describe("ToxiProxyContainer", () => { | ||
jest.setTimeout(240_000); | ||
|
||
// Helper to connect to redis | ||
async function connectTo(url: string) { | ||
const client = createClient({ | ||
url, | ||
}); | ||
client.on("error", () => {}); // Ignore errors | ||
await client.connect(); | ||
expect(client.isOpen).toBeTruthy(); | ||
return client; | ||
} | ||
|
||
// create_proxy { | ||
it("Should create a proxy to an endpoint", async () => { | ||
const containerNetwork = await new Network().start(); | ||
const redisContainer = await new GenericContainer("redis:7.2") | ||
.withNetwork(containerNetwork) | ||
.withNetworkAliases("redis") | ||
.start(); | ||
|
||
const toxiproxyContainer = await new ToxiProxyContainer().withNetwork(containerNetwork).start(); | ||
|
||
// Create the proxy between Toxiproxy and Redis | ||
const redisProxy = await toxiproxyContainer.createProxy({ | ||
name: "redis", | ||
upstream: "redis:6379", | ||
}); | ||
|
||
const url = `redis://${redisProxy.host}:${redisProxy.port}`; | ||
const client = await connectTo(url); | ||
await client.set("key", "val"); | ||
expect(await client.get("key")).toBe("val"); | ||
|
||
await client.disconnect(); | ||
await toxiproxyContainer.stop(); | ||
await redisContainer.stop(); | ||
}); | ||
// } | ||
|
||
// enabled_disabled { | ||
it("Should enable and disable a proxy", async () => { | ||
const containerNetwork = await new Network().start(); | ||
const redisContainer = await new GenericContainer("redis:7.2") | ||
.withNetwork(containerNetwork) | ||
.withNetworkAliases("redis") | ||
.start(); | ||
|
||
const toxiproxyContainer = await new ToxiProxyContainer().withNetwork(containerNetwork).start(); | ||
|
||
// Create the proxy between Toxiproxy and Redis | ||
const redisProxy = await toxiproxyContainer.createProxy({ | ||
name: "redis", | ||
upstream: "redis:6379", | ||
}); | ||
|
||
const url = `redis://${redisProxy.host}:${redisProxy.port}`; | ||
const client = await connectTo(url); | ||
|
||
await client.set("key", "val"); | ||
expect(await client.get("key")).toBe("val"); | ||
|
||
// Disable any new connections to the proxy | ||
await redisProxy.setEnabled(false); | ||
|
||
await expect(client.ping()).rejects.toThrow(); | ||
|
||
// Enable the proxy again | ||
await redisProxy.setEnabled(true); | ||
|
||
expect(await client.ping()).toBe("PONG"); | ||
|
||
await client.disconnect(); | ||
await toxiproxyContainer.stop(); | ||
await redisContainer.stop(); | ||
}); | ||
// } | ||
|
||
// adding_toxic { | ||
it("Should add a toxic to a proxy and then remove", async () => { | ||
const containerNetwork = await new Network().start(); | ||
const redisContainer = await new GenericContainer("redis:7.2") | ||
.withNetwork(containerNetwork) | ||
.withNetworkAliases("redis") | ||
.start(); | ||
|
||
const toxiproxyContainer = await new ToxiProxyContainer().withNetwork(containerNetwork).start(); | ||
|
||
// Create the proxy between Toxiproxy and Redis | ||
const redisProxy = await toxiproxyContainer.createProxy({ | ||
name: "redis", | ||
upstream: "redis:6379", | ||
}); | ||
|
||
const url = `redis://${redisProxy.host}:${redisProxy.port}`; | ||
const client = await connectTo(url); | ||
|
||
// See https://github.com/ihsw/toxiproxy-node-client for details on the instance interface | ||
const toxic = await redisProxy.instance.addToxic<TPClient.Latency>({ | ||
attributes: { | ||
jitter: 50, | ||
latency: 1500, | ||
}, | ||
name: "upstream-latency", | ||
stream: "upstream", | ||
toxicity: 1, // 1 is 100% | ||
type: "latency", | ||
}); | ||
|
||
const before = Date.now(); | ||
await client.ping(); | ||
const after = Date.now(); | ||
expect(after - before).toBeGreaterThan(1000); | ||
|
||
await toxic.remove(); | ||
|
||
await client.disconnect(); | ||
await toxiproxyContainer.stop(); | ||
await redisContainer.stop(); | ||
}); | ||
// } | ||
|
||
it("Should create multiple proxies", async () => { | ||
const containerNetwork = await new Network().start(); | ||
const redisContainer = await new GenericContainer("redis:7.2") | ||
.withNetwork(containerNetwork) | ||
.withNetworkAliases("redis") | ||
.start(); | ||
|
||
const toxiproxyContainer = await new ToxiProxyContainer().withNetwork(containerNetwork).start(); | ||
|
||
// Create the proxy between Toxiproxy and Redis | ||
const redisProxy = await toxiproxyContainer.createProxy({ | ||
name: "redis", | ||
upstream: "redis:6379", | ||
}); | ||
|
||
// Create the proxy between Toxiproxy and Redis | ||
const redisProxy2 = await toxiproxyContainer.createProxy({ | ||
name: "redis2", | ||
upstream: "redis:6379", | ||
}); | ||
|
||
const url = `redis://${redisProxy.host}:${redisProxy.port}`; | ||
const client = await connectTo(url); | ||
await client.set("key", "val"); | ||
expect(await client.get("key")).toBe("val"); | ||
|
||
const url2 = `redis://${redisProxy2.host}:${redisProxy2.port}`; | ||
const client2 = await connectTo(url2); | ||
expect(await client2.get("key")).toBe("val"); | ||
|
||
await client.disconnect(); | ||
await client2.disconnect(); | ||
await toxiproxyContainer.stop(); | ||
await redisContainer.stop(); | ||
}); | ||
|
||
it("Throws an error when too many proxies are created", async () => { | ||
const toxiproxyContainer = await new ToxiProxyContainer().start(); | ||
|
||
for (let i = 0; i < 32; i++) { | ||
await toxiproxyContainer.createProxy({ | ||
name: "test-" + i, | ||
upstream: `google.com:80`, | ||
}); | ||
} | ||
|
||
await expect( | ||
toxiproxyContainer.createProxy({ | ||
name: "test-32", | ||
upstream: `google.com:80`, | ||
}) | ||
).rejects.toThrow(); | ||
|
||
await toxiproxyContainer.stop(); | ||
}); | ||
}); |
Oops, something went wrong.