Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: Remove JSON column from xapi_events_all #376

Closed
3 tasks done
bmtcril opened this issue Sep 12, 2023 · 4 comments
Closed
3 tasks done

Feat: Remove JSON column from xapi_events_all #376

bmtcril opened this issue Sep 12, 2023 · 4 comments

Comments

@bmtcril
Copy link
Contributor

bmtcril commented Sep 12, 2023

When the project started, ClickHouse was working on the JSON column type and expected to have it out of "experimental" status by now. Last month I harassed them in a webinar and they admitted the engineer working on it "implemented it halfway and got bored" so it's stuck in limbo while they find a more interested engineer.

  • It is only used in the Ralph side of the implementation
  • It does not yet work through materialized views, so is unusable downstream in Aspects
  • Performs worse than just parsing the JSON string in many cases
  • Needs to be turned on via support ticket in ClickHouse Cloud due to being "experimental"

In order to remove the column we will need to get permission to update Ralph and make the first set of changes there. They're small changes, but if anyone else is using ClickHouse with Ralph already it might mean a migration. Maybe they can leave the table the same and we can just ignore the column. Then we can remove the column and setting the experimental json flag from our codebase.

  • Remove xapi_events_all.event from Ralph and rename event_str to event
  • Create migration to remove xapi_events_all.event and rename event_str to event in Aspects, and update all references in plugin, dbt, and docs
  • Remove all references to allow_experimental_object_type from plugin, dbt, and docs
@bmtcril
Copy link
Contributor Author

bmtcril commented Oct 25, 2023

openfun/ralph#482 has been opened to discuss this with FUN

@bmtcril
Copy link
Contributor Author

bmtcril commented Nov 2, 2023

openfun/ralph#493 is where the work is being done

@bmtcril
Copy link
Contributor Author

bmtcril commented Jan 30, 2024

Ralph 4 has been released, with this changes, and upgraded here: #572

All that remains is to remove the references.

@bmtcril
Copy link
Contributor Author

bmtcril commented Jan 30, 2024

Reference removal:
#585
openedx/xapi-db-load#85

@bmtcril bmtcril closed this as completed Feb 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant