Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PodioInput: Read all collections by default, allow to limit them with the collections property #162

Merged
merged 4 commits into from
Dec 4, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 10 additions & 6 deletions doc/PodioInputOutput.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,19 +68,23 @@ evtSvc = k4DataSvc("EventDataSvc")
evtSvc.input = "/path/to/your/input-file.root"

podioInput = PodioInput()
podioInput.collections = [
# the complete list of collection names to read
]
```

**Note that currently only the collections that are inside the `collections`
list will be read and become available for later algorithms.**

It is possible to change the input file from the command line via
```bash
k4run <your-options-file> --EventDataSvc.input=<input-file>
```

By default the `PodioInput` will read all collections that are available from
the input file. It is possible to limit the collections that should become
available via the `collections` option

```python
podioInput.collections = [
# List of collection names that should be made available
]
```

## Writing events

To write events you will need to use the `PodioOutput` algorithm in addition to
Expand Down
25 changes: 24 additions & 1 deletion k4FWCore/components/PodioInput.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -186,15 +186,38 @@ PodioInput::PodioInput(const std::string& name, ISvcLocator* svcLoc) : Consumer(
fillReaders();
}

StatusCode PodioInput::initialize() {
// If someone uses the collections property from the command line and passes
// an empty string we assume they want all collections (as a simple way to
// override whatever is in the options file)
if (m_collectionNames.size() == 1 && m_collectionNames[0].empty()) {
m_collectionNames.clear();
}

return StatusCode::SUCCESS;
}

void PodioInput::operator()() const {
if (m_podioDataSvc->getEventFrame().get(edm4hep::EventHeaderName)) {
m_readers[edm4hep::EventHeaderCollection::typeName](edm4hep::EventHeaderName);
} else {
info() << "No EventHeader collection found in the event. Not reading it" << endmsg;
}

for (auto& collName : m_collectionNames) {
const auto& collsToRead = [&]() {
if (m_collectionNames.empty()) {
return m_podioDataSvc->getEventFrame().getAvailableCollections();
} else {
return m_collectionNames.value();
}
}();
Comment on lines +207 to +213
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my c++ knowledge is not as advanced but why do we need this as a lambda? 🥴

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pattern is called an immediately invoked function expression (IIFE) and it is mainly used for "complicated" initialization logic, where it can help to

  • reduce the scope of temporary variables that are necessary for the initialization
  • Allows to declare the initialized object as const, instead of having it mutable and dealing with the logic outside.

So in this case really only to have a const& instead of just a &.


for (const auto& collName : collsToRead) {
debug() << "Registering collection to read " << collName << endmsg;
if (!m_podioDataSvc->getEventFrame().get(collName)) {
warning() << "Collection " << collName << " is not available from file." << endmsg;
continue;
}
auto type = m_podioDataSvc->getCollectionType(collName);
if (m_readers.find(type) != m_readers.end()) {
m_readers[type](collName);
Expand Down
5 changes: 4 additions & 1 deletion k4FWCore/components/PodioInput.h
Original file line number Diff line number Diff line change
Expand Up @@ -42,11 +42,14 @@ class PodioInput final : public Gaudi::Functional::Consumer<void(), BaseClass_t>
PodioInput(const std::string& name, ISvcLocator* svcLoc);
void operator()() const override;

StatusCode initialize() final;

private:
template <typename T> void maybeRead(std::string_view collName) const;
void fillReaders();
// Name of collections to read. Set by option collections (this is temporary)
Gaudi::Property<std::vector<std::string>> m_collectionNames{this, "collections", {}, "Places of collections to read"};
Gaudi::Property<std::vector<std::string>> m_collectionNames{
this, "collections", {}, "Collections that should be read (default all)"};
// Data service: needed to register objects and get collection IDs. Just an observing pointer.
PodioDataSvc* m_podioDataSvc;
mutable std::map<std::string_view, std::function<void(std::string_view)>> m_readers;
Expand Down
1 change: 1 addition & 0 deletions k4FWCore/src/PodioDataSvc.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,7 @@ const std::string_view PodioDataSvc::getCollectionType(const std::string& collNa
const auto coll = m_eventframe.get(collName);
if (coll == nullptr) {
error() << "Collection " << collName << " does not exist." << endmsg;
return "";
}
return coll->getTypeName();
}
Expand Down
1 change: 1 addition & 0 deletions test/k4FWCoreTest/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ endfunction()
add_test_with_env(CreateExampleEventData options/createExampleEventData.py)

add_test_with_env(CheckExampleEventData options/checkExampleEventData.py PROPERTIES DEPENDS CreateExampleEventData)
add_test_with_env(CheckExampleEventData_noCollections options/checkExampleEventData.py --collections= PROPERTIES DEPENDS CreateExampleEventData)
add_test_with_env(CheckExampleEventData_toolong -n 999 options/checkExampleEventData.py PROPERTIES PASS_REGULAR_EXPRESSION
"Application Manager Terminated successfully with a user requested ScheduledStop" DEPENDS CreateExampleEventData)
add_test_with_env(CheckExampleEventData_unbounded options/checkExampleEventData.py -n 999 PROPERTIES PASS_REGULAR_EXPRESSION
Expand Down
19 changes: 12 additions & 7 deletions test/k4FWCoreTest/options/checkExampleEventData.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,14 +26,19 @@

from Configurables import PodioInput

from k4FWCore.parseArgs import parser

parser.add_argument(
"--collections",
action="extend",
nargs="?",
help="The input collections to read",
default=["VectorFloat", "MCParticles", "SimTrackerHits", "TrackerHits", "Tracks"],
)
my_args = parser.parse_known_args()[0]

inp = PodioInput()
inp.collections = [
"VectorFloat",
"MCParticles",
"SimTrackerHits",
"TrackerHits",
"Tracks",
]
inp.collections = my_args.collections

from Configurables import k4FWCoreTest_CheckExampleEventData

Expand Down
Loading