Enhancement: Extended name filtering capabilities for the name_filter method.

[hongyi-zhao]

Current Behavior

The name_filter parameter currently only supports simple string matching:

# Current functionality - simple string containment matching
flow.update_maker_kwargs(settings, name_filter="relax")

This basic matching is implemented using string containment (name_filter in job.name), which limits filtering capabilities to substring matching only.

Proposed Enhancement

Extend the name_filter functionality to support:

1. List-based filtering for multiple patterns:

# Match jobs containing either "relax" or "static"
flow.update_maker_kwargs(settings, name_filter=["relax", "static"])

2. Negative filtering (exclusion):

# Match jobs that don't contain "relax"
flow.update_maker_kwargs(settings, name_filter="!relax")
# Or alternatively with explicit parameter
flow.update_maker_kwargs(settings, name_filter="relax", exclude=True)

Implementation Suggestion

def update_maker_kwargs(
    self,
    settings: dict,
    name_filter: Union[str, List[str], None] = None,
    exclude: bool = False,
    ...
) -> Flow:
    def matches_filter(job_name: str) -> bool:
        if name_filter is None:
            return True
            
        if isinstance(name_filter, list):
            # For list filters, any match satisfies (OR logic)
            matches = any(nf in job_name for nf in name_filter)
        else:
            # For string filters, simple containment
            matches = name_filter in job_name
            
        # Handle exclusion
        return not matches if exclude else matches

Benefits

• More flexible job filtering patterns
• Ability to update multiple job types in one operation
• Support for exclusion patterns
• Maintains backward compatibility with existing code

Questions for Discussion

1. Should we support AND logic for list filters (matching all patterns) in addition to OR logic?
2. Should we consider supporting regex patterns for more advanced matching?
3. For negative filtering, which approach is preferred:
   • Prefix notation (!pattern)
   • Explicit parameter (exclude=True)

Related Work

• Current implementation in jobflow/core/job.py
• Similar filtering patterns in other workflow management systems

Let me know your thoughts on the proposed enhancements and implementation approach.

[utf]

Thanks very much for the suggestion. I wonder whether an alternative option is to allow a filter function. This could the be extremely flexible about how the filtering is performed.

For example:

• List-based filtering for multiple patterns:

# Match jobs containing either "relax" or "static"
flow.update_maker_kwargs(settings, filter_func=lambda job: job.name in ["relax", "static"])

• Negative filtering (exclusion):

# Match jobs that don't contain "relax"
flow.update_maker_kwargs(settings, filter_func=lambda job: "relax" not in job.name)

What do you think?

[hongyi-zhao]

@utf Your idea is much better than mine, it is extremely flexible and powerful.