feat: Add buffered reader #4018
Conversation
WalkthroughA new Changes
Possibly related PRs
Suggested reviewers
Poem
Finishing Touches
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (3)
Examples/Framework/include/ActsExamples/Framework/BufferedReader.hpp (1)
32-34: Document the impact of selectionSeed, we should.The purpose and impact of selectionSeed on event sampling, explain we must. Help future maintainers understand the randomization process, this will.
Examples/Framework/src/Framework/WhiteBoard.cpp (1)
88-97: Exception safety improve, we must.If exception thrown during sharing, partial state possible it is. Consider:
- Transaction-like approach for atomic operations
- Document sharing semantics (deep vs shallow)
- Roll back on failure
void ActsExamples::WhiteBoard::shareDataWith(WhiteBoard &other) const { + // Create temporary map for atomic operation + std::map<std::string, std::any> temp; for (auto &[key, val] : m_store) { - auto [it, success] = other.m_store.insert({key, val}); + auto [it, success] = temp.insert({key, val}); if (!success) { throw std::runtime_error("Cannot share key '" + key + "', is already present"); } } + // Atomic operation + other.m_store.insert(temp.begin(), temp.end()); }Examples/Python/tests/test_reader.py (1)
440-483: Good start with testing, but more coverage needed, I sense!Well structured, the basic test is. But additional test cases, beneficial they would be:
- Different buffer sizes, test you should
- Error handling for invalid configurations, verify you must
- Behavior with empty files, understand we need
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
Examples/Framework/CMakeLists.txt(1 hunks)Examples/Framework/include/ActsExamples/Framework/BufferedReader.hpp(1 hunks)Examples/Framework/include/ActsExamples/Framework/SequenceElement.hpp(1 hunks)Examples/Framework/include/ActsExamples/Framework/WhiteBoard.hpp(1 hunks)Examples/Framework/src/Framework/BufferedReader.cpp(1 hunks)Examples/Framework/src/Framework/WhiteBoard.cpp(1 hunks)Examples/Python/src/Input.cpp(2 hunks)Examples/Python/tests/test_reader.py(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (6)
- GitHub Check: macos
- GitHub Check: missing_includes
- GitHub Check: linux_ubuntu_extra (ubuntu2204_clang, 20)
- GitHub Check: linux_ubuntu_extra (ubuntu2204, 20)
- GitHub Check: linux_ubuntu
- GitHub Check: build_debug
🔇 Additional comments (7)
Examples/Framework/include/ActsExamples/Framework/SequenceElement.hpp (1)
54-54: Careful with friend declarations, we must be! Hmmmm.Access to private members through friendship, this grants. A powerful tool it is, but use it wisely we must. Consider documenting why this friendship necessary it is.
Examples/Framework/include/ActsExamples/Framework/BufferedReader.hpp (1)
59-63: Empty initialize and finalize methods, concerning they are.Consider if initialization or cleanup needed it is. At minimum, document why these methods empty they are.
Examples/Framework/src/Framework/BufferedReader.cpp (1)
43-51: Memory management concerns, I sense.Reserve buffer size upfront, good practice it is. But consider:
- Large buffer sizes, out of memory errors cause they might
- Progress logging for large buffers, helpful it would be
- Error handling for failed reads, improve we should
Examples/Python/src/Input.cpp (1)
11-11: Approve the registration of BufferedReader, I do!Properly declared, the new BufferedReader is. Follow the established pattern for reader declarations, it does. Clear and well-structured, the parameters are.
Also applies to: 43-46
Examples/Framework/include/ActsExamples/Framework/WhiteBoard.hpp (2)
44-45: Wise addition of move semantics, this is!Efficient transfer of WhiteBoard resources, it enables. Default implementations, sufficient they are, given the member types we have.
49-54: Elegant solution for data sharing, you have created!Share data between WhiteBoard instances with minimal overhead, this method does. Well documented, the behavior is. Through shared pointers, efficiency achieved it has.
Examples/Framework/CMakeLists.txt (1)
18-18: Correctly integrated into build system, the new source file is!Follow project conventions for source organization, it does. Simple yet effective, the change is.
Examples/Framework/include/ActsExamples/Framework/WhiteBoard.hpp
Outdated
Show resolved
Hide resolved
Examples/Framework/include/ActsExamples/Framework/BufferedReader.hpp
Outdated
Show resolved
Hide resolved
Examples/Framework/include/ActsExamples/Framework/BufferedReader.hpp
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
Examples/Python/tests/test_reader.py (2)
447-460: Hmm. Verify the event count for data generation, we must.Only
eventsInBufferevents are written, buteventsToProcessevents we expect to read. Document this intentional design choice in comments, we should, to prevent confusion for future padawans.# need to write out some particles first +# Note: We intentionally write fewer events than we process +# to demonstrate the BufferedReader's resampling capability eventsInBuffer = 5 eventsToProcess = 10
479-486: Incomplete, the test coverage is. Additional assertions, we need.Only verify the number of events seen, we do. Add assertions to verify the content of resampled events and proper buffer management, we should.
s2.run() assert alg.events_seen == eventsToProcess + # Verify that events are properly resampled from buffer + events = [ev.particles for ev in s2.events()] + assert len(set(str(ev) for ev in events)) <= eventsInBuffer
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
Examples/Python/tests/test_reader.py(1 hunks)
🧰 Additional context used
📓 Learnings (1)
📓 Common learnings
Learnt from: benjaminhuth
PR: acts-project/acts#4018
File: Examples/Framework/include/ActsExamples/Framework/BufferedReader.hpp:51-54
Timestamp: 2025-01-10T10:35:16.967Z
Learning: The BufferedReader in ActsExamples is designed to serve an unlimited number of events by reusing/resampling from its fixed-size buffer, which helps reduce I/O overhead in timing measurements. Therefore, its availableEvents() correctly returns std::numeric_limits<std::size_t>::max() as the upper bound.
⏰ Context from checks skipped due to timeout of 90000ms (5)
- GitHub Check: linux_physmon
- GitHub Check: linux_examples_test
- GitHub Check: missing_includes
- GitHub Check: linux_ubuntu_extra (ubuntu2204_clang, 20)
- GitHub Check: build_debug
🔇 Additional comments (2)
Examples/Python/tests/test_reader.py (2)
440-446: Clear and descriptive test setup, it is!Well-defined variables for buffer size and total events to process, I see. Good practice for test readability and maintainability, this is.
471-477: Strong with the Force, this configuration is!Correctly configured, the BufferedReader is. Match the retrieved learning about unlimited event serving through resampling, it does.
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (1)
Examples/Python/tests/test_reader.py (1)
440-486: 🛠️ Refactor suggestionMore test cases, we need!
Strong in basic testing this function is, but edge cases it lacks. Consider these scenarios, you must:
- Buffer size of 1, test the minimal case it will
- Buffer size equal to events to process, boundary conditions it shall verify
- Invalid buffer sizes (0 or negative), proper error handling they shall ensure
Help you write these additional test cases, shall I?
🧹 Nitpick comments (1)
Examples/Framework/src/Framework/WhiteBoard.cpp (1)
88-98: Consider reserving space in advance, we should!If performance critical this operation becomes, reserve space in m_store before insertion, we could. Prevent multiple reallocations during insertion, this would.
void ActsExamples::WhiteBoard::copyFrom(const WhiteBoard &other) { + m_store.reserve(m_store.size() + other.m_store.size()); for (auto &[key, val] : other.m_store) { auto [it, success] = m_store.insert({key, val});
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (6)
Examples/Framework/include/ActsExamples/Framework/BufferedReader.hpp(1 hunks)Examples/Framework/include/ActsExamples/Framework/WhiteBoard.hpp(1 hunks)Examples/Framework/src/Framework/BufferedReader.cpp(1 hunks)Examples/Framework/src/Framework/WhiteBoard.cpp(1 hunks)Examples/Python/src/Input.cpp(2 hunks)Examples/Python/tests/test_reader.py(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
- Examples/Framework/src/Framework/BufferedReader.cpp
- Examples/Framework/include/ActsExamples/Framework/BufferedReader.hpp
🧰 Additional context used
📓 Learnings (1)
📓 Common learnings
Learnt from: benjaminhuth
PR: acts-project/acts#4018
File: Examples/Framework/include/ActsExamples/Framework/BufferedReader.hpp:51-54
Timestamp: 2025-01-10T10:35:16.967Z
Learning: The BufferedReader in ActsExamples is designed to serve an unlimited number of events by reusing/resampling from its fixed-size buffer, which helps reduce I/O overhead in timing measurements. Therefore, its availableEvents() correctly returns std::numeric_limits<std::size_t>::max() as the upper bound.
⏰ Context from checks skipped due to timeout of 90000ms (19)
- GitHub Check: linux_ubuntu_extra (ubuntu2204_clang, 20)
- GitHub Check: linux_ubuntu_extra (ubuntu2204, 20)
- GitHub Check: macos
- GitHub Check: missing_includes
- GitHub Check: linux_ubuntu
- GitHub Check: CI Bridge / lcg_106a: [alma9, clang16]
- GitHub Check: CI Bridge / lcg_106a: [alma9, gcc14]
- GitHub Check: CI Bridge / lcg_105: [alma9, gcc13]
- GitHub Check: CI Bridge / lcg_105: [alma9, clang16]
- GitHub Check: CI Bridge / lcg_106a: [alma9, gcc13]
- GitHub Check: CI Bridge / linux_ubuntu_2204_clang
- GitHub Check: CI Bridge / linux_ubuntu_2204
- GitHub Check: CI Bridge / build_linux_ubuntu
- GitHub Check: build_debug
- GitHub Check: CI Bridge / clang_tidy
- GitHub Check: CI Bridge / build_exatrkx
- GitHub Check: CI Bridge / build_exatrkx
- GitHub Check: CI Bridge / clang_tidy
- GitHub Check: CI Bridge / build_exatrkx_cpu
🔇 Additional comments (4)
Examples/Framework/include/ActsExamples/Framework/WhiteBoard.hpp (2)
43-44: Wise addition of move semantics, this is!Efficient resource transfer between WhiteBoard instances, it enables. Default implementations, sufficient they are.
48-53: Well-documented and properly designed, this method is!Clear purpose and behavior, the documentation provides. Const-correctness and exception handling, properly addressed they are.
Examples/Python/src/Input.cpp (2)
11-11: Properly placed, this include is!With its fellow reader includes, it resides.
43-46: Follows the ways of the ACTS Python bindings, this declaration does!Proper parameters and established patterns, it maintains.
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
Examples/Framework/src/Framework/WhiteBoard.cpp (1)
118-124: Unnecessary 'if (success)' check, there is.Since exception thrown if insertion fails, 'success' always true here is. Remove redundant 'if' condition, you can.
Apply this diff to simplify the code:
- if (success) { if (auto it = m_objectAliases.find(name); it != m_objectAliases.end()) { m_store[it->second] = storeIt->second; ACTS_VERBOSE("Added alias object '" << it->second << "'"); } - }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
Examples/Framework/include/ActsExamples/Framework/WhiteBoard.hpp(2 hunks)Examples/Framework/src/Framework/BufferedReader.cpp(1 hunks)Examples/Framework/src/Framework/WhiteBoard.cpp(2 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- Examples/Framework/src/Framework/BufferedReader.cpp
🧰 Additional context used
📓 Learnings (1)
📓 Common learnings
Learnt from: benjaminhuth
PR: acts-project/acts#4018
File: Examples/Framework/include/ActsExamples/Framework/BufferedReader.hpp:51-54
Timestamp: 2025-01-10T10:35:16.967Z
Learning: The BufferedReader in ActsExamples is designed to serve an unlimited number of events by reusing/resampling from its fixed-size buffer, which helps reduce I/O overhead in timing measurements. Therefore, its availableEvents() correctly returns std::numeric_limits<std::size_t>::max() as the upper bound.
⏰ Context from checks skipped due to timeout of 90000ms (8)
- GitHub Check: merge-sentinel
- GitHub Check: unused_files
- GitHub Check: missing_includes
- GitHub Check: linux_ubuntu_extra (ubuntu2204_clang, 20)
- GitHub Check: linux_ubuntu_extra (ubuntu2204, 20)
- GitHub Check: macos
- GitHub Check: build_debug
- GitHub Check: linux_ubuntu
🔇 Additional comments (2)
Examples/Framework/include/ActsExamples/Framework/WhiteBoard.hpp (2)
43-44: Ensure all members are safely movable, you must.Added default move constructor and move assignment operator, you have. Verify that all member variables are safely movable and that default implementations suffice.
48-53: Consider exception safety of 'copyFrom', you should.Ensure that 'copyFrom' handles exceptions properly and maintains consistent state, you must.
|
@andiwand could you have a look at this one? I also have a commit on the branch now. |
Examples/Framework/include/ActsExamples/Framework/WhiteBoard.hpp
Outdated
Show resolved
Hide resolved
Co-authored-by: Andreas Stefl <[email protected]>
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (1)
Examples/Framework/src/Framework/WhiteBoard.cpp (1)
93-98:⚠️ Potential issueProtect against the dark side of exceptions, we must.
Strong exception guarantee, this implementation lacks. If exception during copy occurs, in inconsistent state the whiteboard may remain.
Safer approach, implement you should:
void ActsExamples::WhiteBoard::copyFrom(const WhiteBoard &other) { + // Validate all keys before modification + for (const auto &[key, val] : other.m_store) { + if (m_store.contains(key)) { + throw std::invalid_argument("Object '" + key + "' already exists"); + } + } + for (auto &[key, val] : other.m_store) { addHolder(key, val); ACTS_VERBOSE("Copied key '" << key << "' to whiteboard"); } }
🧹 Nitpick comments (2)
Examples/Framework/src/Framework/WhiteBoard.cpp (2)
118-123: Redundant check of 'success', I sense.Already verified at line 112, the success condition was. Unnecessary this second check becomes.
Simplify the code, we can:
- if (success) { - if (auto it = m_objectAliases.find(name); it != m_objectAliases.end()) { - m_store[it->second] = storeIt->second; - ACTS_VERBOSE("Added alias object '" << it->second << "'"); - } - } + if (auto it = m_objectAliases.find(name); it != m_objectAliases.end()) { + m_store[it->second] = storeIt->second; + ACTS_VERBOSE("Added alias object '" << it->second << "'"); + }
102-108: Enhanced logging for troubleshooting, suggest I do.Debug logs before validation checks, helpful they would be.
Add debug logging, you should:
if (name.empty()) { + ACTS_DEBUG("Attempt to add object with empty name"); throw std::invalid_argument("Object can not have an empty name"); } if (holder == nullptr) { + ACTS_DEBUG("Attempt to add nullptr object with name '" << name << "'"); throw std::invalid_argument("Object '" + name + "' is nullptr"); }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
Examples/Framework/src/Framework/WhiteBoard.cpp(2 hunks)
🧰 Additional context used
📓 Learnings (1)
📓 Common learnings
Learnt from: benjaminhuth
PR: acts-project/acts#4018
File: Examples/Framework/include/ActsExamples/Framework/BufferedReader.hpp:51-54
Timestamp: 2025-01-10T10:35:16.967Z
Learning: The BufferedReader in ActsExamples is designed to serve an unlimited number of events by reusing/resampling from its fixed-size buffer, which helps reduce I/O overhead in timing measurements. Therefore, its availableEvents() correctly returns std::numeric_limits<std::size_t>::max() as the upper bound.
⏰ Context from checks skipped due to timeout of 90000ms (20)
- GitHub Check: CI Bridge / lcg_106a: [alma9, clang16]
- GitHub Check: CI Bridge / lcg_106a: [alma9, gcc14]
- GitHub Check: CI Bridge / lcg_106a: [alma9, gcc13]
- GitHub Check: unused_files
- GitHub Check: missing_includes
- GitHub Check: linux_ubuntu_extra (ubuntu2204_clang, 20)
- GitHub Check: linux_ubuntu_extra (ubuntu2204, 20)
- GitHub Check: macos
- GitHub Check: linux_ubuntu
- GitHub Check: build_debug
- GitHub Check: CI Bridge / lcg_105: [alma9, clang16]
- GitHub Check: CI Bridge / lcg_105: [alma9, gcc13]
- GitHub Check: CI Bridge / linux_ubuntu_2204_clang
- GitHub Check: CI Bridge / build_linux_ubuntu
- GitHub Check: CI Bridge / linux_ubuntu_2204
- GitHub Check: CI Bridge / build_exatrkx
- GitHub Check: CI Bridge / build_exatrkx_cpu
- GitHub Check: CI Bridge / clang_tidy
- GitHub Check: CI Bridge / clang_tidy
- GitHub Check: docs
🔇 Additional comments (1)
Examples/Framework/src/Framework/WhiteBoard.cpp (1)
70-74: Wise addition of alias handling to name similarity search, this is.Consistent with existing implementation, the handling of aliases remains. Good practice in maintaining unified approach to name matching, you show.
|
acts-project-service
added
the
Breaks Athena build
This reader can wrap an exisiting reader, preload some events in a buffer, and randomly picks events upon execution in the sequencer. Should help to mitigate I/O bottlenecks in throughput measurements.
--- END COMMIT MESSAGE ---
Any further description goes here, @-mentions are ok here!
feat,fix,refactor,docs,choreandbuildtypes.Summary by CodeRabbit
New Features
BufferedReaderfor enhanced event data handling with buffering capabilities.WhiteBoard.Improvements
WhiteBoard.WhiteBoardmethods.BufferedReader.Tests
BufferedReaderfunctionality.