Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(sql): use last_course_visit mv for learner summary query #953

Merged
merged 2 commits into from
Sep 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
with
latest_emission_time as (
select course_key, actor_id, MAX(emission_time) as last_visited
from {{ DBT_PROFILE_TARGET_DATABASE }}.fact_navigation
select org, course_key, actor_id, MAX(emission_time) as last_visited
from {{ ASPECTS_XAPI_DATABASE }}.fact_learner_last_course_visit
where
1 = 1
{% include 'openedx-assets/queries/common_filters.sql' %}
{% include 'openedx-assets/queries/user_filters.sql' %}
group by course_key, actor_id
group by org, course_key, actor_id
),
enrollment_status as (
select course_key, actor_id, MAX(emission_time) as max_emission_time
select org, course_key, actor_id, MAX(emission_time) as max_emission_time
from {{ DBT_PROFILE_TARGET_DATABASE }}.fact_enrollment_status
where
1 = 1
{% include 'openedx-assets/queries/common_filters.sql' %}
{% include 'openedx-assets/queries/user_filters.sql' %}
group by course_key, actor_id
group by org, course_key, actor_id
),
student_status as (
select
Expand Down Expand Up @@ -58,9 +58,11 @@ select
from student_status fss
left join
enrollment_status fes
on fss.course_key = fes.course_key
on fss.org = fes.org
and fss.course_key = fes.course_key
and fss.actor_id = fes.actor_id
left join
latest_emission_time let
on fss.course_key = let.course_key
on fss.org = let.org
and fss.course_key = let.course_key
and fss.actor_id = let.actor_id
Original file line number Diff line number Diff line change
@@ -1,3 +1,62 @@
-- select one record per (learner, problem, course, org) tuple
-- contains either the first successful attempt
-- or the most recent unsuccessful attempt
-- find the timestamp of the earliest successful response
-- this will be used to pick the xAPI event corresponding to that submission
with
full_responses as (
select
events.emission_time as emission_time,
events.org as org,
events.course_key as course_key,
events.problem_id as problem_id,
events.object_id as object_id,
events.actor_id as actor_id,
events.responses as responses,
events.success as success,
events.attempts as attempts,
events.interaction_type as interaction_type
from {{ ASPECTS_XAPI_DATABASE }}.responses
join
{{ ASPECTS_XAPI_DATABASE }}.problem_events events using (
org, course_key, problem_id, actor_id, emission_time
)
where 1 = 1 {% include 'openedx-assets/queries/common_filters.sql' %}
),
int_problem_results as (
select
full_responses.emission_time as emission_time,
full_responses.org as org,
full_responses.course_key as course_key,
blocks.course_name as course_name,
blocks.course_run as course_run,
full_responses.problem_id as problem_id,
blocks.block_name as problem_name,
blocks.display_name_with_location as problem_name_with_location,
blocks.course_order as course_order,
concat(
'<a href="',
full_responses.object_id,
'" target="_blank">',
blocks.display_name_with_location,
'</a>'
) as problem_link,
full_responses.actor_id as actor_id,
full_responses.responses as responses,
full_responses.success as success,
full_responses.attempts as attempts,
full_responses.interaction_type as interaction_type,
blocks.graded
from full_responses
join
{{ DBT_PROFILE_TARGET_DATABASE }}.dim_course_blocks blocks
on (
full_responses.course_key = blocks.course_key
and full_responses.problem_id = blocks.block_id
)
where 1 = 1 {% include 'openedx-assets/queries/common_filters.sql' %}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you need this filter again?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure, I was quick testing it, I will benchmark it to be sure

)

select
org,
course_key,
Expand All @@ -9,6 +68,6 @@ select
course_grade,
approving_state
from {{ DBT_PROFILE_TARGET_DATABASE }}.fact_student_status
left join {{ DBT_PROFILE_TARGET_DATABASE }}.int_problem_results
left join int_problem_results
using org, course_key, course_run, actor_id, course_name
where 1 = 1 {% include 'openedx-assets/queries/common_filters.sql' %}