CHANGELOG

0.1.0

First stable release. A custom Flume source for tailing files from local file system and ftp.The main job is done DefaultFileMonitor tool from Apache commons vfs2. Once the file is processed succesfully by flume, it is moved to a target directory. In this version this action is acomplished by commons.io with FileUtils.

0.2.0

Important changes and an interesting new capabality is added to the source. Instead using FileUtils for moving files we try out commons vfs2 for moving files, this seems more natural, and also, it will allow do the operation of moving under the supported file systems by apache commons vfs2. Another important feature is the "timeout" or delay for processing files. Once a file is in the directory been monitorized, it is send to be processed by flume. We faced and issue when the file, small o huge, is still been downloaded when the file has already started to be process. So delaying the file to be discovered, we allow the file to be fully of content before stargin to process. For huge files it may not be necessary because if a new line is appended after the previous one have already been sent to process, that new one is added and the event is tagged as "modification". But what we really worried about network latency causing issues like truncated data.

0.2.1

We face a new requirement about what to do when the file has been processed. We introduce a new configurable parameter for seeting an action to acomplished when the file has been succesfully processed. Choose between move or delete.