Add an option to delete docker/kubernetes attributes that are also tags

[jonmoter]

Describe what happened:

The plugin helpfully adds tags like image_name and container_name to log entries, as described in the docs and in this source code.

fluent-plugin-datadog/lib/fluent/plugin/out_datadog.rb

Lines 380 to 412 in 01f90b0

	# Collect docker and kubernetes tags for your logs using `filter_kubernetes_metadata` plugin,
	# for more information about the attribute names, check:
	# https://github.com/fabric8io/fluent-plugin-kubernetes_metadata_filter/blob/master/lib/fluent/plugin/filter_kubernetes_metadata.rb#L265
	def get_container_tags(record)
	[
	get_kubernetes_tags(record),
	get_docker_tags(record)
	].compact.join(",")
	end
	def get_kubernetes_tags(record)
	if record.key?('kubernetes') and not record.fetch('kubernetes').nil?
	kubernetes = record['kubernetes']
	tags = Array.new
	tags.push("image_name:" + kubernetes['container_image']) unless kubernetes['container_image'].nil?
	tags.push("container_name:" + kubernetes['container_name']) unless kubernetes['container_name'].nil?
	tags.push("kube_namespace:" + kubernetes['namespace_name']) unless kubernetes['namespace_name'].nil?
	tags.push("pod_name:" + kubernetes['pod_name']) unless kubernetes['pod_name'].nil?
	return tags.join(",")
	end
	nil
	end
	def get_docker_tags(record)
	if record.key?('docker') and not record.fetch('docker').nil?
	docker = record['docker']
	tags = Array.new
	tags.push("container_id:" + docker['container_id']) unless docker['container_id'].nil?
	return tags.join(",")
	end
	nil
	end

However, now in the datadog logs UI, I have those values in two places. Once as a tag called container_name, and once as a facet called @kubernetes.container_name. This causes some confusion among users of the logs UI, on which one they should use.

Describe what you'd like to see:

I'd like an option like delete_tag_attributes. Where if it's set, and a tag is created for the log entry, it deletes the value from the record itself. Then I'd only see the tag in the logs UI, rather than two copies of the same information.

The option should default to false, to preserve existing behavior.

I could probably configure Datadog's log pipelines to strip out the extra attribute on the server, but it would be nice to avoid duplicating data sent to the server.

Steps to reproduce the issue:

n/a

Additional environment details (Operating System, Cloud provider, etc):

Fluentd 1.6.3, running on a Kubernetes cluster, version 1.18.6