METRICS ANALYSIS

Environment:

• Flume 1.6.0-cdh5.12.0
• Options for java: -Xms512m -Xmx2048m
• Flume config file:

# www.keedio.com

# example configuration for VFS source

#ACTIVE LIST
agent.sources = stest1
agent.channels = ch3
agent.sinks = ktest1



agent.sources.stest1.type = org.keedio.flume.source.vfs.source.SourceVFS
agent.sources.stest1.work.dir = /mnt/fileserver/etl/source_file/ocsusr/incoming
agent.sources.stest1.includePattern = Generic.*.
agent.sources.stest1.post.process.file = delete 
agent.sources.stest1.status.file.dir = /tmp/saved
agent.sources.stest1.timeout.start.post.process = 180
agent.sources.stest1.delay.between.runs = 20

###to file roll
agent.sources.stest1.channels = ch3

agent.sinks.ktest1.type = file_roll
agent.sinks.ktest1.sink.directory = /var/log/flume-vfs
agent.sinks.ktest1.sink.rollInterval = 360

agent.channels.ch3.type = memory
agent.channels.ch3.capacity = 10000000
agent.channels.ch3.transactionCapacity = 1000000

agent.sinks.ktest1.channel = ch3

Files are being written while flume processing.

Files are written before dumping to flumes's incoming

Files exist in flume's incoming before agent starts, dicover.