Skip to content

Commit

Permalink
fixing
Browse files Browse the repository at this point in the history
  • Loading branch information
@Simplychee
Simplychee committed Dec 2, 2024
1 parent 28633d8 commit 36201d8
Showing 1 changed file with 38 additions and 42 deletions.
80 changes: 38 additions & 42 deletions docs/shipping/Containers/docker.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,33 +19,29 @@ drop_filter: []

### Docker collector

Docker is a set of platform as a service products that deliver software in containers. This integration is a Docker container that uses Filebeat to collect logs
from other Docker containers and forward them to your Logz.io account.
Docker is a set of platform-as-a-service products that deliver software in containers. This integration is a Docker container that can use either Filebeat or Fluent Bit to collect logs from other Docker containers and forward them to your Logz.io account.

To use docker-collector-logs, you'll set environment variables when you run the container. The Docker logs directory and docker.sock are mounted to the container, enabling Filebeat or Fluent Bit to collect logs and metadata.

To use docker-collector-logs, you'll set environment variables when you run the container.
The Docker logs directory and docker.sock are mounted to the container, allowing Filebeat to collect the logs and metadata.

:::note
[Project's GitHub repo](https://github.com/logzio/docker-collector-logs/)
:::


##### Upgrading to a newer version

* Upgrading to a newer version of docker-collector-logs while it is already running
will cause it to resend logs that are within the `ignoreOlder` timeframe.
You can minimize log duplicates
by setting the `ignoreOlder` parameter of the new docker
to a lower value (for example, `20m`).

* Version 0.1.0 of docker-collector-logs includes breaking changes. Please see the project's [change log](https://github.com/logzio/docker-collector-logs#change-log) for further information.

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

<Tabs>
<TabItem value="filebeat-docker" label="Filebeat" default>

##### Upgrading to a newer version

* When upgrading to a newer version of docker-collector-logs while it is already running, logs within the `ignoreOlder` timeframe may be resent. To minimize log duplicates, configure the `ignoreOlder` parameter of the new container to a lower value (e.g., `20m`).

* Version 0.1.0 of docker-collector-logs includes breaking changes. Refer to the [change log](https://github.com/logzio/docker-collector-logs#change-log) for further details.


#### Pull the Docker image

Download the logzio/docker-collector-logs image.
Expand Down Expand Up @@ -81,20 +77,20 @@ docker service create --name docker-collector-logs \

#### Parameters

| Parameter | Description | Required/Default |
| Parameter | Description | Field Type |
|---|---|---|
| LOGZIO_TOKEN | Your Logz.io account token. {@include: ../../_include/log-shipping/log-shipping-token.html} | Required |
| LOGZIO_REGION | Logz.io region code to ship the logs to. This region code changes depending on the region your account is hosted in. For example, accounts in the EU region have region code `eu`. If you don't specify this parameter, the default value will be used. For more information, see [Account region](https://docs.logz.io/docs/user-guide/admin/hosting-regions/account-region/) on the Logz.io Docs. | (US region) |
| LOGZIO_TYPE | The log type you'll use with this Docker. {@include: ../../_include/log-shipping/type.md} | Docker image name |
| LOGZIO_CODEC | Set to `json` if shipping JSON logs. Otherwise, set to `plain` for plain text format. | `plain` |
| ignoreOlder | Set a time limit on back shipping logs. Upgrading to a newer version of docker-collector-logs while it is already running will cause it to resend logs that are within the `ignoreOlder` timeframe. You can minimize log duplicates by setting the `ignoreOlder` parameter of the new docker to a lower value (for example, `20m`). | `3h` |
| LOGZIO_URL | URL for your account listener host. The URL changes depending on the region your account is hosted in. You can skip this parameter if you specify `LOGZIO_REGION`. If neither `LOGZIO_URL` nor `LOGZIO_REGION` is specified, the default value will be used. For more information, see [Account region](https://docs.logz.io/docs/user-guide/admin/hosting-regions/account-region/) on the Logz.io Docs. | `listener.logz.io:5015` |
| additionalFields | Include additional fields with every message sent, formatted as `"fieldName1=fieldValue1;fieldName2=fieldValue2"`. To use an environment variable, format as `"fieldName1=fieldValue1;fieldName2=$ENV_VAR_NAME"`. In that case, the environment variable should be the only value in the field. If the environment variable can't be resolved, the field is omitted. | -- |
| matchContainerName | Comma-separated list of containers you want to collect the logs from. If a container's name partially matches a name on the list, that container's logs are shipped. Otherwise, its logs are ignored. **Note: Can't be used with skipContainerName** | -- |
| skipContainerName | Comma-separated list of containers you want to ignore. If a container's name partially matches a name on the list, that container's logs are ignored. Otherwise, its logs are shipped. **Note: Can't be used with matchContainerName** | -- |
| includeLines | Comma-separated list of regular expressions to match the lines that you want to include. **Note**: Regular expressions in this list should not contain commas. | -- |
| excludeLines | Comma-separated list of regular expressions to match the lines that you want to exclude. **Note**: Regular expressions in this list should not contain commas. | -- |
| renameFields | Rename fields with every message sent, formatted as `"oldName,newName;oldName2,newName2"`. To use an environment variable, format as `"oldName,newName;oldName2,$ENV_VAR_NAME"`. When using an environment variable, it should be the only value in the field. If the environment variable can't be resolved, the field will be omitted. | -- |
| additionalFields | Include additional fields with every message sent, formatted as `"fieldName1=fieldValue1;fieldName2=fieldValue2"`. To use an environment variable, format as `"fieldName1=fieldValue1;fieldName2=$ENV_VAR_NAME"`. In that case, the environment variable should be the only value in the field. If the environment variable can't be resolved, the field is omitted. | Optional |
| matchContainerName | Comma-separated list of containers you want to collect the logs from. If a container's name partially matches a name on the list, that container's logs are shipped. Otherwise, its logs are ignored. **Note: Can't be used with skipContainerName** | Optional |
| skipContainerName | Comma-separated list of containers you want to ignore. If a container's name partially matches a name on the list, that container's logs are ignored. Otherwise, its logs are shipped. **Note: Can't be used with matchContainerName** | Optional |
| includeLines | Comma-separated list of regular expressions to match the lines that you want to include. **Note**: Regular expressions in this list should not contain commas. | Optional |
| excludeLines | Comma-separated list of regular expressions to match the lines that you want to exclude. **Note**: Regular expressions in this list should not contain commas. | Optional |
| renameFields | Rename fields with every message sent, formatted as `"oldName,newName;oldName2,newName2"`. To use an environment variable, format as `"oldName,newName;oldName2,$ENV_VAR_NAME"`. When using an environment variable, it should be the only value in the field. If the environment variable can't be resolved, the field will be omitted. | Optional |
| HOSTNAME | Include your host name to display it for the field `agent.name`. If no value is entered, `agent.name`displays the container id.| `''` |
| multilinePattern | Include your regex pattern. See [Filebeat's official documentation](https://www.elastic.co/guide/en/beats/filebeat/7.12/multiline-examples.html#multiline) for more information. | `''` |
| multilineNegate |Include `'true'` to negate the pattern. **Note**: Cannot be used without multilinePattern. See [Filebeat's official documentation](https://www.elastic.co/guide/en/beats/filebeat/7.12/multiline-examples.html#multiline) for more information.| `'false'` |
Expand All @@ -103,6 +99,9 @@ docker service create --name docker-collector-logs \
| INPUT_ENCODING | Here is a full list of [valid encodings](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-container.html#_encoding) you can use. | `utf-8` |


:::note
By default, logs from docker-collector-logs and docker-collector-metrics containers are ignored.
:::


</TabItem>
Expand Down Expand Up @@ -133,25 +132,25 @@ logzio/docker-logs-collector:latest
```
#### Parameters

| Parameter | Description | Required/Default |
| Parameter | Description | Field Type |
|--|--|--|
| **LOGZIO_LOGS_TOKEN**| Your Logz.io account logs token. Replace `<LOGS-SHIPPING-TOKEN>` with the [token](https://app.logz.io/#/dashboard/settings/general) of the account you want to ship to. | Required |
| **LOGZIO_URL** | `https://listener.logz.io:8071`. The full URL to send logs to, including your region if needed. For example, for the EU region, use `https://listener-eu.logz.io:8071`. to.| Default |
| **LOGZIO_TYPE** | `logzio-docker-logs`. Sets the log type.| Default|
| **MATCH_CONTAINER_NAME** | Specify a container to collect logs from. If the container's name matches, its logs are shipped; otherwise, its logs are ignored. **Note**: This option cannot be used with SKIP_CONTAINER_NAMES. Use regular expressions to keep records that match a specific field.|--|
| **SKIP_CONTAINER_NAMES** | Comma-separated list of containers to ignore. If a container's name matches a name on this list, its logs are ignored; otherwise, its logs are shipped. **Note**: This option cannot be used with MATCH_CONTAINER_NAME. Use regular expressions to exclude records matching a specific field. |--|
| **MATCH_IMAGE_NAME** | Specify an image to collect logs from. If the image's name matches, its logs are shipped; otherwise, its logs are ignored. **Note**: This option cannot be used with SKIP_IMAGE_NAMES. Use regular expressions to keep records that matching a specific field.|--|
| **SKIP_IMAGE_NAMES** | Comma-separated list of images to ignore. If an image's name matches a name on this list, its logs are ignored; otherwise, its logs are shipped. **Note**: This option cannot be used with MATCH_IMAGE_NAME. Use regular expressions to exclude records that match a specific field.|--|
| **INCLUDE_LINE** | Regular expression to match which lines Fluent Bit should include.|--|
| **EXCLUDE_LINES** | Regular expression to match which lines Fluent Bit should exclude.|--|
| **ADDITIONAL_FIELDS** | Include additional fields with every message sent, formatted as `"fieldName1:fieldValue1,fieldName2:fieldValue2"`.|--|
| **SET_FIELDS** | Set fields with every message sent, formatted as `"fieldName1:fieldValue1,fieldName2:fieldValue2"`.|--|
| **LOG_LEVEL** | `info`. Set log level for Fluent Bit. Allowed values: `debug`, `info`, `warning`, `error`.|Default|
| **MULTILINE_START_STATE_RULE** | Regular expression for the start state rule of multiline parsing. See [Fluent Bit's official documentation](https://docs.fluentbit.io/manual/administration/configuring-fluent-bit/multiline-parsing#rules-definition) for further info.|--|
| **MULTILINE_CUSTOM_RULES** | Custom rules for multiline parsing, separated by semicolons `;`.||
| **READ_FROM_HEAD** | `true`. Specify if Fluent Bit should read logs from the beginning.|Default|
| **OUTPUT_ID** | `output_id`. Specify the output ID for Fluent Bit logs.|Default|
| **HEADERS** | Custom headers for Fluent Bit logs.|--|
| LOGZIO_LOGS_TOKEN| Your Logz.io logs account token. Replace `<LOGS-SHIPPING-TOKEN>` with the [token](https://app.logz.io/#/dashboard/settings/general) associated with the account you want to use for shipping logs. | Required |
| LOGZIO_URL | The full URL to send logs to, including your region if needed. For example, for the EU region, use `https://listener-eu.logz.io:8071`. | `https://listener.logz.io:8071` |
| LOGZIO_TYPE | Sets the log type.| `logzio-docker-logs` |
| MATCH_CONTAINER_NAME | Specify a container to collect logs from. If the container's name matches, its logs are shipped; otherwise, its logs are ignored. **Note**: This option cannot be used with SKIP_CONTAINER_NAMES. Use regular expressions to keep records that match a specific field.| Optional |
| SKIP_CONTAINER_NAMES | Comma-separated list of containers to ignore. If a container's name matches a name on this list, its logs are ignored; otherwise, its logs are shipped. **Note**: This option cannot be used with MATCH_CONTAINER_NAME. Use regular expressions to exclude records matching a specific field. | Optional |
| MATCH_IMAGE_NAME | Specify an image to collect logs from. If the image's name matches, its logs are shipped; otherwise, its logs are ignored. **Note**: This option cannot be used with SKIP_IMAGE_NAMES. Use regular expressions to keep records that matching a specific field. | Optional |
| SKIP_IMAGE_NAMES | Comma-separated list of images to ignore. If an image's name matches a name on this list, its logs are ignored; otherwise, its logs are shipped. **Note**: This option cannot be used with MATCH_IMAGE_NAME. Use regular expressions to exclude records that match a specific field. | Optional |
| INCLUDE_LINE | Regular expression to match which lines Fluent Bit should include. | Optional |
| EXCLUDE_LINES | Regular expression to match which lines Fluent Bit should exclude. | Optional |
| ADDITIONAL_FIELDS | Include additional fields with every message sent, formatted as `"fieldName1:fieldValue1,fieldName2:fieldValue2"`.| Optional |
| SET_FIELDS | Set fields with every message sent, formatted as `"fieldName1:fieldValue1,fieldName2:fieldValue2"`. | Optional |
| LOG_LEVEL | Set log level for Fluent Bit. Allowed values: `debug`, `info`, `warning`, `error`.| `info` |
| MULTILINE_START_STATE_RULE** | Regular expression for the start state rule of multiline parsing. See [Fluent Bit's official documentation](https://docs.fluentbit.io/manual/administration/configuring-fluent-bit/multiline-parsing#rules-definition) for further info.| Optional |
| MULTILINE_CUSTOM_RULES | Custom rules for multiline parsing, separated by semicolons `;`.| Optional |
| READ_FROM_HEAD | Specify if Fluent Bit should read logs from the beginning.|`true`|
| OUTPUT_ID | Specify the output ID for Fluent Bit logs.|`output_id`|
| HEADERS | Custom headers for Fluent Bit logs.| Optional |


</TabItem>
Expand All @@ -161,9 +160,6 @@ logzio/docker-logs-collector:latest



:::note
By default, logs from docker-collector-logs and docker-collector-metrics containers are ignored.
:::


#### Check Logz.io for your logs
Expand Down

0 comments on commit 36201d8

Please sign in to comment.