forked from smetj/logstash
-
Notifications
You must be signed in to change notification settings - Fork 0
/
CHANGELOG
260 lines (236 loc) · 13.2 KB
/
CHANGELOG
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
1.1 (??? ??, 2012)
- Bugs fixed
- [LOGSTASH-181] - output/statsd: set sender properly
- feature: grok filter: named_captures_only now defaults to true
## NEW
- upgrade: elasticsearch 0.18.6 now included
- upgrade: now using jruby 1.9
- feature: [LOGSTASH-158] - MRI-1.9 compatible (except for some
plugins/functions which will throw a compatibility exception)
- bugfix, feature: [LOGSTASH-40,65]: much smarter file watching for file
inputs. now supports globs, keeps state between runs, can handle
truncate, log rotation, etc. no more inotify is required, either
(file input now works on all platforms)
- feature: [LOGSTASH-66,150]: libgrok re-written in pure ruby (no more
FFI / external libgrok.so dependency!)
- feature: new file output plugin
- feature: [LOGSTASH-316] - allow specifying "type" and "tags" for filters
- feature: [LOGSTASH-168] - add new GELF input plugin
- feature: [LOGSTASH-267,-254] - add DNS filter for doing forward or
reverse DNS on an event field
- feature: [LOGSTASH-57] - add mutate filter to help with manipulating
event field content and type
- upgrade: now using cabin library for all internal logging
- feature: add json filter to turn an event field with JSON into the
expanded data structure
## general
- bugfix: initialize log4j appenders. silences warnings, provides logging
for in-process elasticsearch server
- bugfix: AMQP overhaul (input & output), please see docs for updated
config parameters
- bugfix: [LOGSTASH-162,177] - make sure plugin-contained global actions
happen serially across all plugins (with a mutex)
## inputs
- bugfix: [LOGSTASH-317] - fix file input not to crash when a file
is unreadable
- feature: [LOGSTASH-172] - syslog input accepts ISO8601 timestamps
- feature: [LOGSTASH-159] - TCP input lets you configure what identifies
an input stream to the multiline filter (unique per host, or connection)
- bugfix: [LOGSTASH-8] - fix stomp input
## filters
- bugfix: [LOGSTASH-285] - allow dropping empty grok matches
- feature: [LOGSTASH-219] - support parsing unix epoch times
## outputs
- bugfix: embedded elasticsearch now acts as a full ES server; previously
embedded was only accessible from within the logstash process.
- bugfix: many gelf output enhancements and bugfixes
- feature: [LOGSTASH-281] - add https support to loggly output
- bugfix: [LOGSTASH-167] - limit number of in-flight requests to the
elasticsearch node to avoid creating too many threads (one thread per
pending write request)
- misc: [LOGSTASH-160] - switch from rake to make
- test: use minitest
1.0.17 (Aug 12, 2011)
- Bugs fixed
- [LOGSTASH-147] - grok filter incorrectly adding fields when a match failed
- [LOGSTASH-151] - Fix bug in routing keys on AMQP
- [LOGSTASH-156] - amqp issue with 1.0.16?
- Improvement
- [LOGSTASH-148] - AMQP input should allow queue name to be specified separately from exchange name
- [LOGSTASH-157] - Plugin doc generator should make regexp config names more readable
- New Feature
- [LOGSTASH-153] - syslog input: make timestamp an optional field
- [LOGSTASH-154] - Make error reporting show up in the web UI
1.0.16 (Aug 18, 2011)
- Fix elasticsearch client problem with 1.0.15 - jruby-elasticsearch gem
version required is now 0.0.10 (to work with elasticsearch 0.17.6)
1.0.15 (Aug 18, 2011)
- IMPORTANT: Upgraded to ElasticSearch 0.17.6 - this brings a number of bug
fixes including an OOM error caused during high index rates in some
conditions.
NOTE: You *must* use same main version of elasticsearch as logstash does,
so if you are still using elasticsearch server 0.16.x - you need to upgrade
your server before the elasticsearch output will work. If you are using
the 'embedded' elasticsearch feature of logstash, you do not need to make
any changes.
- feature: tcp input and output plugins can now operate in either client
(connect) or server (listen) modes.
- feature: new output plugin "statsd" which lets you increment or record
timings from your logs to a statsd agent
- feature: new redis 'pattern_channel' input support for PSUBSCRIBE
- feature: new output plugin "graphite" for taking metrics from events and
shipping them off to your graphite/carbon server.
- feature: new output plugin "ganglia" for shipping metrics to ganglia
gmond server.
- feature: new output plugin "xmpp" for shipping events over jabber/xmpp
- feature: new input plugin "xmpp" for receiving events over jabber/xmpp
- feature: amqp input now supports routing keys.
https://logstash.jira.com/browse/LOGSTASH-122
- feature: amqp output now supports setting routing key dynamically.
https://logstash.jira.com/browse/LOGSTASH-122
- feature: amqp input/output both now support SSL.
https://logstash.jira.com/browse/LOGSTASH-131
- feature: new input plugin "exec" for taking events from executed commands
like shell scripts or other tools.
- feature: new filter plugin "split" for splitting one event into multiple.
It was written primarily for the new "exec" input to allow you to split
the output of a single command run by line into multiple events.
- misc: upgraded jar releases to use JRuby 1.6.3
- bugfix: syslog input shouldn't crash anymore on weird network behaviors
like portscanning, etc.
https://logstash.jira.com/browse/LOGSTASH-130
1.0.14 (Jul 1, 2011)
- feature: new output plugin "loggly" which lets you ship logs to loggly.com
- feature: new output plugin "zabbix" - similar to the nagios output, but
works with the Zabbix monitoring system. Contributed by Johan at
Mach Technology.
- feature: New agent '-e' flag which lets you specify a config in a string.
If you specify no 'input' plugins, default is stdin { type => stdin }
If you specify no 'output' plugins, default is stdout { debug => true }
This is intended to be used for hacking with or debugging filters, but
you can specify an entire config here if you choose.
- feature: Agent '-f' flag now supports directories and globs. If you specify
a directory, all files in that directory will be loaded as a single config.
If you specify a glob, all files matching that glob will be loaded as a
single config.
- feature: gelf output now allows you to override the 'sender'. This defaults
to the source host originating the event, but can be set to anything now.
It supports dynamic values, so you can use fields from your event as the
sender. Contributed by John Vincent
Issue: https://github.com/logstash/logstash/pull/30
- feature: added new feature to libgrok that allows you to define patterns
in-line, like "%{FOO=\d+}" defines 'FOO' match \d+ and captures as such.
To use this new feature, you must upgrade libgrok to at least 1.20110630
Issue: https://logstash.jira.com/browse/LOGSTASH-94
- feature: grok filter now supports 'break_on_match' defaulting to true
(this was the original behavior). If you set it to false, it will attempt
to match all patterns and create new fields as normal. If left default
(true), it will break after the first successful match.
- feature: grok filter now supports parsing any field. You can do either of
these: grok { match => [ "fieldname", "pattern" ] }
or this: grok { fieldname => "pattern" }
The older 'pattern' attribute still means the same thing, and is equivalent
to this: grok { match => [ "@message", "pattern" ] }
Issue: https://logstash.jira.com/browse/LOGSTASH-101
- feature: elasticsearch - when embedded is true, you can now set the
'embedded_http_port' to configure which port the embedded elasticsearch
server listens on. This is only valid for the embedded elasticsearch
configuration. https://logstash.jira.com/browse/LOGSTASH-117
- bugfix: amqp input now reconnects properly when the amqp broker restarts.
- bugfix: Fix bug in gelf output when a fields were not arrays but numbers.
Issue: https://logstash.jira.com/browse/LOGSTASH-113
- bugfix: Fix a bug in syslog udp input due to misfeatures in Ruby's URI
class. https://logstash.jira.com/browse/LOGSTASH-115
- misc: jquery and jquery ui now ship with logstash; previously they were
loaded externally
- testing: fixed some bugs in the elasticsearch test itself, all green now.
- testing: fixed logstash-test to now run properly
1.0.12 (Jun 9, 2011)
- misc: clean up some excess debugging output
- feature: for tcp input, allow 'data_timeout => -1' to mean "never time out"
1.0.11 (Jun 9, 2011)
- deprecated: The redis 'name' and 'queue' options for both input and output
are now deprecated. They will be removed in a future version.
- feature: The redis input and output now supports both lists and channels.
- feature: Refactor runner to allow you to run multiple things in a single
process. You can end each instance with '--' flag. For example, to run one
agent and one web instance:
% java -jar logstash-blah.jar agent -f myconfig -- web
- feature: Add 'embedded' option to the elasticsearch output:
elasticsearch { embedded => true }
Default is false. If true, logstash will run an elasticsearch server
in the same process as logstash. This is really useful if you are just
starting out or only need one one elasticsearch server.
- feature: Added a logstash web backend feature for elasticsearch that tells
logstash to use the 'local' (in process) elasticsearch:
--backend elasticsearch:///?local
- feature: Added 'named_captures_only' option to grok filter. This will have
logstash only keep the captures you give names to - for example %{NUMBER}
won't be kept, but %{NUMBER:bytes} will be.
- feature: Add 'bind_host' option to elasticsearch output. This lets you choose the
address ElasticSearch client uses to bind to - useful if you have a
multihomed server.
- feature: The mongodb output now supports authentication
- bugfix: Fix bug in GELF output that caused the gelf short_message to be set as an
array if it came from a grok value. The short_message field should only
now be a string properly.
- bugfix: Fix bug in grep filter that would drop/cancel events if you had
more than one event type flowing through filters and didn't have a grep
filter defined for each type.
- misc: Updated gem dependencies (tests still pass)
- misc: With the above two points, you can now run a single logstash process
that includes elasticsearch server, logstash agent, and logstash web.
1.0.10 (May 23, 2011)
- Fix tcp input bug (LOGSTASH-88) that would drop connections.
- Grok patterns_dir (filter config) and --grok-patterns-dir (cmdline opt)
are now working.
- GELF output now properly sends extra fields from the log event (prefixed
with a "_") and sets timestamp to seconds-since-epoch (millisecond
precision and time zone information is lost, but this is the format GELF
asks for).
- Inputs support specifying the format of input data (see "format" and
"message_format" input config parameters).
- Grok filter no longer incorrectly tags _grokparsefailure when more than
one grok filter is enabled (for multiple types) or when an event has
no grok configuration for it's type.
- Fix bug where an invalid HTTP Referer: would break grok parsing of the
log line (used to expect %{URI}). Since Referer: is not sanitized in
the HTTP layer, we cannot assume it will be a well formed %{URI}.
1.0.9 (May 18, 2011)
- Fix crash bug caused by refactoring that left 'break' calls in code
that no longer used loops.
1.0.8 (May 17, 2011)
- Remove beanstalk support because the library (beanstalk-client) is GPL3. I
am not a lawyer, but I'm not waiting around to have someone complain about
license incompatibilities.
- fix bug in jar build
1.0.7 (May 16, 2011)
- logstash 'web' now allows you to specify the elasticsearch clustername;
--backend elasticsearch://[host[:port]]/[clustername]
- GELF output now supports dynamic strings for level and facility
https://logstash.jira.com/browse/LOGSTASH-83
- 'amqp' output supports persistent messages over AMQP, now. Tunable.
https://logstash.jira.com/browse/LOGSTASH-81
- Redis input and output are now supported. (Contributed by dokipen)
- Add shutdown processing. Shutdown starts when all inputs finish (like
stdin) The sequence progresses using the same pipeline as the
inputs/filters/outputs, so all in-flight events should finish getting
processed before the final shutdown event makes it's way to the outputs.
- Add retries to unhandled input exceptions (LOGSTASH-84)
1.0.6 (May 11, 2011)
* Remove 'sigar' from monolithic jar packaging. This removes a boatload of
unnecessary warning messages on startup whenever you use elasticsearch
output or logstash-web.
Issue: https://logstash.jira.com/browse/LOGSTASH-79
1.0.5 (May 10, 2011)
* fix queues when durable is set to true
1.0.4 (May 9, 2011)
* Fix bugs in syslog input
1.0.2 (May 8, 2011)
* Fix default-value handling for configs when the validation type is
'password'
1.0.1 (May 7, 2011)
* Fix password auth for amqp and stomp (Reported by Luke Macken)
* Fix default elasticsearch target for logstash-web (Reported by Donald Gordon)
1.0.0 (May 6, 2011)
* First major release.