forked from rails/rails
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
The goal of the commit is to make #includes slimmer -
only join the table required to satisfy the referenced tables and
preload all other associations independently.
Context
The Active Record #includes method results in different behaviour
depending on the other parts of the query:
it may execute multiple queries (one per included association) or
build a single JOIN query to load everything at once.
Before the commit, it was "all or nothing" behind an eager load flag -
one referenced table was enough to transform the whole query into one big JOIN.
However, it's not always necessary to join all the tables
except the ones with conditions.
Normally, we could avoid joining everything (which could be performance-heavy),
and execute a few queries instead.
Idea
The main idea is to remove referenced tables
(which are #references, #where, #order or copied by #and, #or, #merge)
from the final join query.
So we would LEFT OUTER JOINed only includes tables with conditions on the query.
At the same time,
we have to bring non-referenced tables back to separate query preloads.
Backward compatibility
1) Includes and table names don't always match so we use reflections.
We try to match all possible includes table names against references
including inner and left outer joins.
If there's an unreliable match (like no reflection for association),
we're inclined to JOIN the associations in order to not break user applications.
All current tests are green without changes inside them.
2) The commit certainly breaks user code with the plain string where or select:
```
Author.includes(:company, :books).where(company: { title: "The Office" })
.where("books.title = ?", "Design Patterns")
.select("books.title")
```
We do not want to introduce any flags or scopes to support this behaviour
because in docs there're mentions to prohibit string causes without references:
https://api.rubyonrails.org/v7.0.4.2/classes/ActiveRecord/QueryMethods.html#method-i-includes
https://api.rubyonrails.org/v7.0.4.2/classes/ActiveRecord/QueryMethods.html#method-i-references
People who don't know it have to start using it.
Future improvements
1) Includes are not always plain - they may contain several levels inside a hash.
In the commit, we handle several includes
as separate trees starting from the top-level root.
If there's any reference to possible includes table,
we'll mark this whole tree as needed to JOINed.
It's a safe bet which could be improved in future
with a more fine-grained reference marking algorithm.
2) The commit's tree traversal is average performant at best.
Several intermediate structures and tree walks could be removed -
we prefer a simple algorithm at the moment to show the idea.- Loading branch information
Showing
7 changed files
with
397 additions
and
7 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
87 changes: 87 additions & 0 deletions
87
activerecord/lib/active_record/relation/includes_tracker.rb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,87 @@ | ||
| # frozen_string_literal: true | ||
|
|
||
| module ActiveRecord | ||
| # Determine includes which are used / not used in refererences and joins. | ||
| module IncludesTracker # :nodoc: | ||
| def includes_values_referenced | ||
| select_includes_values_with_references(:any?, :present?) | ||
| end | ||
|
|
||
| def includes_values_non_referenced | ||
| select_includes_values_with_references(:all?, :blank?) | ||
| end | ||
|
|
||
| private | ||
| def select_includes_values_with_references(matcher, intersect_matcher) | ||
| all_references = (references_values + joins_values + left_outer_joins_values).map(&:to_s) | ||
|
|
||
| normalized_includes_values.select do |includes_value| | ||
| includes_tree = ActiveRecord::Associations::JoinDependency.make_tree(includes_value) | ||
|
|
||
| includes_values_reflections(includes_tree).public_send(matcher) do |reflection| | ||
| next true unless reliable_reflection_match?(reflection) | ||
|
|
||
| (possible_includes_tables(reflection) & all_references).public_send(intersect_matcher) | ||
| end | ||
| end | ||
| end | ||
|
|
||
| def includes_values_reflections(includes_tree) | ||
| includes_reflections = [] | ||
|
|
||
| traverse_tree_with_model(includes_tree, self) do |association, model| | ||
| reflection = model.reflect_on_association(association) | ||
|
|
||
| includes_reflections << reflection | ||
|
|
||
| reliable_reflection_match?(reflection) ? reflection.klass : model | ||
| end | ||
|
|
||
| includes_reflections | ||
| end | ||
|
|
||
| def possible_includes_tables(reflection) | ||
| all_possible_includes_tables = | ||
| reflection.collect_join_chain.map(&:table_name) << | ||
| reflection.alias_candidate(reflection.table_name) << | ||
| reflection.name.to_s | ||
|
|
||
| if inferable_reflection_table_name?(reflection) | ||
| all_possible_includes_tables << | ||
| reflection.join_table << | ||
| join_table_with_postfix_alias(reflection) | ||
| end | ||
|
|
||
| all_possible_includes_tables | ||
| end | ||
|
|
||
| def reliable_reflection_match?(reflection) | ||
| reflection && inferable_reflection_klass?(reflection) | ||
| end | ||
|
|
||
| def inferable_reflection_klass?(reflection) | ||
| !reflection.polymorphic? | ||
| end | ||
|
|
||
| def inferable_reflection_table_name?(reflection) | ||
| !reflection.through_reflection? | ||
| end | ||
|
|
||
| def join_table_with_postfix_alias(reflection) | ||
| [reflection.join_table, reflection.alias_candidate(:join)].sort.join("_") | ||
| end | ||
|
|
||
| def normalized_includes_values | ||
| includes_values.map do |element| | ||
| element.is_a?(Hash) ? element.map { |key, value| { key => value } } : element | ||
| end.flatten | ||
| end | ||
|
|
||
| def traverse_tree_with_model(object, model, &block) | ||
| object.each do |key, value| | ||
| next_model = yield(key, model) | ||
| traverse_tree_with_model(value, next_model, &block) if next_model | ||
| end | ||
| end | ||
| end | ||
| end |
Oops, something went wrong.