Key tag not in schema

[iparask]

I define a task in radical.entk as follows:

    task1 = Task()
    task1.name = '%s-T1' % stage1.name
    task1.pre_exec = ['module load anaconda3/2019.03',
                      'source activate keras-gpu']
    task1.executable = 'python3'   # Assign executable to the task
    # Assign arguments for the task executable
    task1.arguments = ['predict.py',
                       '--input', '$NODE_LFS_PATH/%s/%s-multipage.tif' %
                       (task0.name, image.split('/')[-1].split('.')[0]),
                       '--output_folder', '$NODE_LFS_PATH/%s' % task1.name,
                       '--weights_path', 'weights/unet_weights.hdf5']
    task1.link_input_data = ['$SHARED/unet_weights.hdf5 >' +
                             'weights/unet_weights.hdf5']
    task1.upload_input_data = [os.path.abspath('../classification/predict.py'),
                               os.path.abspath('../classification/' +
                                               'gen_patches.py'),
                               os.path.abspath('../classification/' +
                                               'train_unet.py'),
                               os.path.abspath('../classification/' +
                                               'unet_model.py')]
    task1.cpu_reqs = {'processes': 1, 'threads_per_process': 1,'process_type': None,
                      'thread_type': 'OpenMP'}
    task1.gpu_reqs = {'processes': 1, 'threads_per_process': 1,'process_type': None,
                      'thread_type': 'OpenMP'}
    task1.tag = task0.name

When the task is getting submitted I get the following error:

1582837692.001 : radical.entk.task_manager.0000 : 24100 : 140437612377856 : ERROR    : task_manager.0000 failed with key tag not in schema
Traceback (most recent call last):
  File "/home/iparask/iceberg/lib/python3.7/site-packages/radical/entk/execman/rp/task_manager.py", line 324, in _process_tasks
    umgr.submit_units(bulk_cuds)
  File "/home/iparask/iceberg/lib/python3.7/site-packages/radical/pilot/unit_manager.py", line 698, in submit_units
    unit = ComputeUnit(umgr=self, descr=ud)
  File "/home/iparask/iceberg/lib/python3.7/site-packages/radical/pilot/compute_unit.py", line 61, in __init__
    descr.verify()
  File "/home/iparask/iceberg/lib/python3.7/site-packages/radical/utils/description.py", line 234, in verify
    return super(Description, self).verify(self._schema)
  File "/home/iparask/iceberg/lib/python3.7/site-packages/radical/utils/description.py", line 187, in verify
    if k not in schema: raise TypeError('key %s not in schema' % k)
TypeError: key tag not in schema

Apparently in radical.pilot schema tag, has been renamed to tags.

@andre-merzky can you confirm that the change happened? I think this is critical and needs to be fixed immediately.

I can create a hot fix and open the required PR, but I do not know EnTK's release procedure to finalize it.

[andre-merzky]

Hmm, RP introduced tags in this commit in August 2018, but that did not exist as tag before, AFAICT. Well, that's what I saw from a quick check. Either way though, we can add tag as an alias to tags until EnTK is sorted out?

[iparask]

I remember that it worked with 0.72, so I made a few spot checks. EnTK had it as tag, but RP registered only tag as an attribute and could set both tag and tags.

I will sort EnTK now, run a test based on a use case and let you know.

[iparask]

Furthermore, RP's scheduler used the tag key to schedule units, which apparently is still true in devel

[mturilli]

@iparask any update on this?

[iparask]

Yes, this is work in progress. Currently, I am trying to understand how the colocation scheduler works exactly, because I am not sure if it schedules units. Also, there is this issue: radical-cybertools/radical.pilot/issues/2258

[andre-merzky]

@iparask : can you please give the RP branch hotfix/tags a try? I should address this problem for the time being. Sorry that this took so long...

[iparask]

The hotfix/tags branch works! Thank you! Also, the tags PR is open again!