Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

More improvements to ExclusivePages memory allocator #445

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

More improvements to ExclusivePages memory allocator #445

wants to merge 7 commits into from

Conversation

ArthurBrussee
Copy link
Contributor

Follow up to #419 as I was still seeing some bad memory behiavour:

  • Prefer picking slices that aren't about to be freed
  • Get rid of neighbour search again, too unpredictable
    • As part of that, use simpler max_alloc_size again instead of can_handle_alloc & brute force search. "minumum" allocation was a bit confusing.
  • Instead of hardcoded over allocation keep a moving average of the expected alloc size. If smaller, alloc the average, if bigger, allocate that size exactly.
  • Tweak pool sizes / counts / dealloc frequencies a bit

@ArthurBrussee ArthurBrussee changed the title Mem partt 2 More improvements to ExclusivePages memory allocator Jan 22, 2025
@ArthurBrussee ArthurBrussee mentioned this pull request Jan 25, 2025
Closed
nathanielsimard
nathanielsimard reviewed Jan 25, 2025
View reviewed changes
crates/cubecl-runtime/src/memory_management/memory_pool/exclusive_pool.rs
self.pages.push(MemoryPage {
slice,
alloc_size,
free_count: 0,
free_count: ALLOC_AFTER_FREE - 1,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I get that setting

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some comments! It's to sort of 'suspect' allocations as unused

crates/cubecl-runtime/src/memory_management/memory_pool/exclusive_pool.rs Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nathanielsimard nathanielsimard nathanielsimard left review comments

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ArthurBrussee @nathanielsimard