forked from influxdata/telegraf
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Bigquery output Plugin (influxdata#8634)
- Loading branch information
1 parent
c905116
commit 5085f59
Showing
6 changed files
with
477 additions
and
22 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| # BigQuery Google Cloud Output Plugin | ||
|
|
||
| This plugin writes to the [Google Cloud BigQuery][bigquery] and requires [authentication][] | ||
| with Google Cloud using either a service account or user credentials | ||
|
|
||
| This plugin accesses APIs which are [chargeable][pricing]; you might incur | ||
| costs. | ||
|
|
||
| Requires `project` to specify where BigQuery entries will be persisted. | ||
|
|
||
| Requires `dataset` to specify under which BigQuery dataset the corresponding metrics tables reside. | ||
|
|
||
| Each metric should have a corresponding table to BigQuery. | ||
| The schema of the table on BigQuery: | ||
| * Should contain the field `timestamp` which is the timestamp of a telegraph metrics | ||
| * Should contain the metric's tags with the same name and the column type should be set to string. | ||
| * Should contain the metric's fields with the same name and the column type should match the field type. | ||
|
|
||
| ### Configuration | ||
|
|
||
| ```toml | ||
| [[outputs.bigquery]] | ||
| ## GCP Project | ||
| project = "erudite-bloom-151019" | ||
|
|
||
| ## The BigQuery dataset | ||
| dataset = "telegraf" | ||
|
|
||
| ## Timeout for BigQuery operations. | ||
| # timeout = "5s" | ||
|
|
||
| ## Character to replace hyphens on Metric name | ||
| # replace_hyphen_to = "_" | ||
| ``` | ||
|
|
||
| ### Restrictions | ||
|
|
||
| Avoid hyphens on BigQuery tables, underlying SDK cannot handle streaming inserts to Table with hyphens. | ||
|
|
||
| In cases of metrics with hyphens please use the [Rename Processor Plugin](https://github.com/influxdata/telegraf/tree/master/plugins/processors/rename). | ||
|
|
||
| In case of a metric with hyphen by default hyphens shall be replaced with underscores (_). | ||
| This can be altered using the `replace_hyphen_to` configuration property. | ||
|
|
||
| Available data type options are: | ||
| * integer | ||
| * float or long | ||
| * string | ||
| * boolean | ||
|
|
||
| All field naming restrictions that apply to BigQuery should apply to the measurements to be imported. | ||
|
|
||
| Tables on BigQuery should be created beforehand and they are not created during persistence | ||
|
|
||
| Pay attention to the column `timestamp` since it is reserved upfront and cannot change. | ||
| If partitioning is required make sure it is applied beforehand. |
Oops, something went wrong.