casa_concat capabilities and errors

[Fil8]

casa_concat embedded in the transform worker has some limitations, the most important is that it cannot concatenate SPWs of different sizes.

Another property is that 'If concatvis already exists (e.g., it is the same as the first input data set), then the other input data sets will be appended to the concatvis data set.'

Within the transform worker the latter is not necessarily what the user wants (and it can easily generate errors because of the previous point), hence I suggest to add a keyword to delete previously existing files or not. This can be added here:

caracal/caracal/workers/transform_worker.py

Lines 338 to 381 in 346405b

	if pipeline.enable_task(config, 'concat'):
	concat_labels = label_in.split(',')
	step = 'concat-ms{0:d}-{1:d}'.format(i, target_iter)
	concat_ms = [from_ms.replace('.ms', '-{0:s}.ms'.format(cl)) for cl in concat_labels]
	recipe.add('cab/casa_concat', step,
	{
	"vis": concat_ms,
	"concatvis": 'tobedeleted-' + to_ms,
	},
	input=pipeline.input,
	output=pipeline.output,
	label='{0:s}:: Concatenate {1:}'.format(step, concat_ms))
	# If the output of this run of mstransform exists, delete it first
	if os.path.exists('{0:s}/{1:s}'.format(pipeline.msdir, to_ms)) or \
	os.path.exists('{0:s}/{1:s}'.format(pipeline.msdir, flagv)):
	os.system(
	'rm -rf {0:s}/{1:s} {0:s}/{2:s}'.format(pipeline.msdir, to_ms, flagv))
	step = 'singlespw-ms{0:d}-{1:d}'.format(i, target_iter)
	recipe.add('cab/casa_mstransform', step,
	{
	"vis": 'tobedeleted-' + to_ms,
	"outputvis": to_ms,
	"datacolumn": config['concat']['col'],
	"combinespws": True,
	},
	input=pipeline.input,
	output=pipeline.output,
	label='{0:s}:: Single SPW {1:}'.format(step, concat_ms))
	substep = 'save-{0:s}-ms{1:d}'.format('caracal_legacy', target_iter)
	manflags.add_cflags(pipeline, recipe, 'caracal_legacy', to_ms,
	cab_name=substep, overwrite=False)
	# Delete the tobedeleted file, but first we need to have created it, thus...
	recipe.run()
	# Empty job que after execution
	recipe.jobs = []
	os.system(
	'rm -rf {0:s}/tobedeleted-{1:s}'.format(pipeline.msdir, to_ms))
	obsinfo_msname = to_ms