Skip to content

Latest commit

 

History

History
142 lines (113 loc) · 8.07 KB

error-log-logger.md

File metadata and controls

142 lines (113 loc) · 8.07 KB
title
error-log-logger

Description

error-log-logger is a plugin which pushes the log data of APISIX's error.log to TCP servers or Apache SkyWalking.

This plugin will provide the ability to send the log data which selected by the level to Monitoring tools and other TCP servers, and SkyWalking over HTTP.

This plugin provides the ability as a batch to push the log data to your external TCP servers or monitoring tools. If not receive the log data, don't worry, it will automatically send the logs after the timer function expires in our Batch Processor.

For more info on Batch-Processor in Apache APISIX please refer. Batch-Processor

Attributes

Name Type Requirement Default Valid Description
tcp.host string required IP address or the Hostname of the TCP server.
tcp.port integer required [0,...] Target upstream port.
tcp.tls boolean optional false Control whether to perform SSL verification.
tcp.tls_server_name string optional The server name for the new TLS extension SNI.
skywalking.endpoint_addr string optional http://127.0.0.1:12900/v3/logs the http endpoint of Skywalking.
skywalking.service_name string optional APISIX service name for skywalking reporter
skywalking.service_instance_name String optional APISIX Instance Name Service instance name for skywalking reporter, set it to $hostname to get local hostname directly.
clickhouse.endpoint_addr String optional http://127.0.0.1:8213 clickhouse HTTP endpoint, default http://127.0.0.1:8213
clickhouse.user String optional default clickhouse user
clickhouse.password String optional clickhouse password
clickhouse.database String optional clickhouse for error log DB name
clickhouse.logtable String optional clickhouse for error log table name
host string optional (Deprecated, use tcp.host instead) IP address or the Hostname of the TCP server.
port integer optional [0,...] (Deprecated, use tcp.port instead) Target upstream port.
tls boolean optional false (Deprecated, use tcp.tls instead) Control whether to perform SSL verification.
tls_server_name string optional (Deprecated, use tcp.tls_server_name instead) The server name for the new TLS extension SNI.
timeout integer optional 3 [1,...] Timeout for the upstream to connect and send, unit: second.
keepalive integer optional 30 [1,...] Time for keeping the cosocket alive, unit: second.
level string optional WARN The filter's log level, default warn, choose the level in ["STDERR", "EMERG", "ALERT", "CRIT", "ERR", "ERROR", "WARN", "NOTICE", "INFO", "DEBUG"], the value ERR equals ERROR.

The plugin supports the use of batch processors to aggregate and process entries(logs/data) in a batch. This avoids frequent data submissions by the plugin, which by default the batch processor submits data every 5 seconds or when the data in the queue reaches 1000. For information or custom batch processor parameter settings, see Batch-Processor configuration section.

How To Enable And Disable

The error-log-logger is a global plugin of APISIX.

Enable plugin

Enable the plug-in error-log-logger in conf/config.yaml, then this plugin can work fine. It does not need to be bound in any route or service.

Here is an example of conf/config.yaml:

plugins:                          # plugin list
  ... ...
  - request-id
  - hmac-auth
  - api-breaker
  - error-log-logger              # enable plugin `error-log-logger

Disable plugin

Remove or comment out the plugin error-log-logger from conf/config.yaml.

plugins:                          # plugin list
  ... ...
  - request-id
  - hmac-auth
  - api-breaker
  #- error-log-logger              # enable plugin `error-log-logger

How to set the TCP server address

Step: update the attributes of the plugin

curl http://127.0.0.1:9080/apisix/admin/plugin_metadata/error-log-logger -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
  "tcp": {
    "host": "127.0.0.1",
    "port": 1999
  },
  "inactive_timeout": 1
}'

How to set the SkyWalking OAP server address

Step: update the attributes of the plugin

curl http://127.0.0.1:9080/apisix/admin/plugin_metadata/error-log-logger -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
  "skywalking": {
    "endpoint_addr":"http://127.0.0.1:12800/v3/logs"
  },
  "inactive_timeout": 1
}'

How to set the clickhouse

The plugin sends the error log as a string to the data field of the clickhouse table.

TODO Here save error log as a whole string to clickhouse 'data' column. We will add more columns in the future.

Step: update the attributes of the plugin

curl http://127.0.0.1:9080/apisix/admin/plugin_metadata/error-log-logger -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
  "clickhouse": {
      "user": "default",
      "password": "a",
      "database": "error_log",
      "logtable": "t",
      "endpoint_addr": "http://127.0.0.1:8123"
  }
}'