Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Return some room data in Sliding Sync /sync #17320

Merged
merged 75 commits into from
Jul 2, 2024
Merged

Return some room data in Sliding Sync /sync #17320

merged 75 commits into from
Jul 2, 2024

Commits on Jun 17, 2024

  1. Return some room timeline data in Sliding Sync

    @MadLittleMods
    MadLittleMods committed Jun 17, 2024
    Configuration menu
    Copy the full SHA
    079194c View commit details
    Browse the repository at this point in the history

  2. Strip invite/knock event itself and avoid mutating event unsigned 

    Make sure we don't run into
#14919
(matrix-org/synapse#14919)
    @MadLittleMods
    MadLittleMods committed Jun 17, 2024
    Configuration menu
    Copy the full SHA
    3e0f759 View commit details
    Browse the repository at this point in the history

  3. Add changelog

    @MadLittleMods
    MadLittleMods committed Jun 17, 2024
    Configuration menu
    Copy the full SHA
    5e2fd4e View commit details
    Browse the repository at this point in the history

  4. Fix sort being lost

    @MadLittleMods
    MadLittleMods committed Jun 17, 2024
    Configuration menu
    Copy the full SHA
    8ce06f1 View commit details
    Browse the repository at this point in the history

Commits on Jun 18, 2024

  1. Start on required_state

    @MadLittleMods
    MadLittleMods committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    aa5f54a View commit details
    Browse the repository at this point in the history

  2. Add some notes from pairing

    @MadLittleMods
    MadLittleMods committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    5c175d5 View commit details
    Browse the repository at this point in the history

  3. Remove required_state for now

    @MadLittleMods
    MadLittleMods committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    9089bfe View commit details
    Browse the repository at this point in the history

  4. Clean up knock_state comments

    @MadLittleMods
    MadLittleMods committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    9427991 View commit details
    Browse the repository at this point in the history

  5. Calculate num_live

    @MadLittleMods
    MadLittleMods committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    19b2297 View commit details
    Browse the repository at this point in the history

  6. Add tests for limited

    @MadLittleMods
    MadLittleMods committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    81d36f3 View commit details
    Browse the repository at this point in the history

  7. Add more tests

    @MadLittleMods
    MadLittleMods committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    9791209 View commit details
    Browse the repository at this point in the history

Commits on Jun 19, 2024

  1. Fix lint

    @MadLittleMods
    MadLittleMods committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    70ecd4d View commit details
    Browse the repository at this point in the history

  2. Make room name optional

    @MadLittleMods
    MadLittleMods committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    71eabe5 View commit details
    Browse the repository at this point in the history

  3. Update comments

    @MadLittleMods
    MadLittleMods committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    39b4f10 View commit details
    Browse the repository at this point in the history

  4. Add bundled aggregations

    @MadLittleMods
    MadLittleMods committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    9883b0f View commit details
    Browse the repository at this point in the history

  5. Determine limited before filtering

    @MadLittleMods
    MadLittleMods committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    1c06153 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'develop' into madlittlemods/sliding-sync-room-data

    @MadLittleMods
    MadLittleMods committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    57ba033 View commit details
    Browse the repository at this point in the history

  7. Add better support for leave/ban

    @MadLittleMods
    MadLittleMods committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    c81f300 View commit details
    Browse the repository at this point in the history

  8. Fix lints

    @MadLittleMods
    MadLittleMods committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    d801db0 View commit details
    Browse the repository at this point in the history

  9. Merge branch 'develop' into madlittlemods/sliding-sync-room-data

    @MadLittleMods
    MadLittleMods committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    6942b64 View commit details
    Browse the repository at this point in the history

  10. Update some wording

    @MadLittleMods
    MadLittleMods committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    884b448 View commit details
    Browse the repository at this point in the history

Commits on Jun 20, 2024

  1. Remove unused IncludeOldRooms class

    @MadLittleMods
    MadLittleMods committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    0eb0294 View commit details
    Browse the repository at this point in the history

Commits on Jun 24, 2024

  1. Merge branch 'develop' into madlittlemods/sliding-sync-room-data

    @MadLittleMods
    MadLittleMods committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    b1b4231 View commit details
    Browse the repository at this point in the history

  2. Fix lints 

    See #17320 (comment)

```
synapse/federation/federation_server.py:677: error: Cannot determine type of "_join_rate_per_room_limiter"  [has-type]
synapse/federation/federation_server.py:720: error: Cannot determine type of "_join_rate_per_room_limiter"  [has-type]
```
    @MadLittleMods
    MadLittleMods committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    87fac19 View commit details
    Browse the repository at this point in the history

  3. Add TODO for filtering call invites in public rooms

    @MadLittleMods
    MadLittleMods committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    0e71a2f View commit details
    Browse the repository at this point in the history

  4. newly_joined vs limited already being tracked in a discussion 

    See #17320 (comment)
if anything comes out of it.
    @MadLittleMods
    MadLittleMods committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    21ca02c View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2024

  1. Fix spelling typo

    @MadLittleMods
    MadLittleMods committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    3568311 View commit details
    Browse the repository at this point in the history

  2. Just stripped_state for invite rooms

    @MadLittleMods
    MadLittleMods committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    7aea406 View commit details
    Browse the repository at this point in the history

  3. Finish up stripped_state for invite rooms 

    See #17320 (comment)
    @MadLittleMods
    MadLittleMods committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    e3e431f View commit details
    Browse the repository at this point in the history

  4. Add tracking discussion for not optional in the future

    @MadLittleMods
    MadLittleMods committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    303d834 View commit details
    Browse the repository at this point in the history

  5. Start testing for the correct room membership (failing)

    @MadLittleMods
    MadLittleMods committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    4c22131 View commit details
    Browse the repository at this point in the history

  6. Describe current_state_delta_stream better

    @MadLittleMods
    MadLittleMods committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    83d6f76 View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2024

  1. Add get_current_state_delta_membership_changes_for_user(...) (using… 

    … `current_state_delta_stream`)

(still need to add newly_left rooms back)
    @MadLittleMods
    MadLittleMods committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    fbd92e1 View commit details
    Browse the repository at this point in the history

  2. WIP: Add back newly_left

    @MadLittleMods
    MadLittleMods committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    6c791a8 View commit details
    Browse the repository at this point in the history

  3. Iterate

    @MadLittleMods
    MadLittleMods committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    27d74b0 View commit details
    Browse the repository at this point in the history

  4. Just fetch full events for `get_current_state_delta_membership_change… 

    …s_for_user(...)`

Makes downstream logic simpler and although we may look-up some events
we don't use, the lookup is all done in one go instead of fetching events
from event_ids in a couple different places.
    @MadLittleMods
    MadLittleMods committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    fb8fbd4 View commit details
    Browse the repository at this point in the history

  5. Remove extras

    @MadLittleMods
    MadLittleMods committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    d91aa00 View commit details
    Browse the repository at this point in the history

  6. Add docstring

    @MadLittleMods
    MadLittleMods committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    daa7e36 View commit details
    Browse the repository at this point in the history

  7. Refactor back to not pulling out full events

    @MadLittleMods
    MadLittleMods committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    cccbd15 View commit details
    Browse the repository at this point in the history

  8. Add newly_joined support to get_sync_room_ids_for_user(...)

    @MadLittleMods
    MadLittleMods committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    62c6a4e View commit details
    Browse the repository at this point in the history

  9. Join both tables with stream_ordering

    @MadLittleMods
    MadLittleMods committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    39259f6 View commit details
    Browse the repository at this point in the history

  10. Merge branch 'develop' into madlittlemods/sliding-sync-room-data 

    Conflicts:
	tests/rest/client/test_sync.py
    @MadLittleMods
    MadLittleMods committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    5c21315 View commit details
    Browse the repository at this point in the history

  11. Fix clause change

    @MadLittleMods
    MadLittleMods committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    c60aca7 View commit details
    Browse the repository at this point in the history

  12. Remove debug log

    @MadLittleMods
    MadLittleMods committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    11db1be View commit details
    Browse the repository at this point in the history

  13. Fix `builtins.SyntaxError: EOL while scanning string literal (test_sy… 

    …nc.py, line 1885)`

See https://github.com/element-hq/synapse/actions/runs/9675073109/job/26692003103?pr=17320#step:9:5552

Worked fine locally but failed in CI with Python 3.8
    @MadLittleMods
    MadLittleMods committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    7395e10 View commit details
    Browse the repository at this point in the history

  14. Add some tests for `get_current_state_delta_membership_changes_for_us… 

    …er(...)`
    @MadLittleMods
    MadLittleMods committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    2bf3923 View commit details
    Browse the repository at this point in the history

  15. Create events using helper

    @MadLittleMods
    MadLittleMods committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    ec2d8dc View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2024

  1. Add test that remotely joins room

    @MadLittleMods
    MadLittleMods committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    0b9a903 View commit details
    Browse the repository at this point in the history

  2. Actually test `get_current_state_delta_membership_changes_for_user(..… 

    ….)` in remote join test
    @MadLittleMods
    MadLittleMods committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    48d0acf View commit details
    Browse the repository at this point in the history

  3. Add state of the db in each situation

    @MadLittleMods
    MadLittleMods committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    2a944ff View commit details
    Browse the repository at this point in the history

  4. Remove redundant instance_name column

    @MadLittleMods
    MadLittleMods committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    8df39d1 View commit details
    Browse the repository at this point in the history

  5. Add skipped test for state resets

    @MadLittleMods
    MadLittleMods committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    b7914e7 View commit details
    Browse the repository at this point in the history

  6. Fix lints

    @MadLittleMods
    MadLittleMods committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    7eb1806 View commit details
    Browse the repository at this point in the history

  7. All get_current_state_delta_membership_changes_for_user(...) tests … 

    …passing
    @MadLittleMods
    MadLittleMods committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    935b98c View commit details
    Browse the repository at this point in the history

  8. Remove need for topological_ordering

    @MadLittleMods
    MadLittleMods committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    f163fcf View commit details
    Browse the repository at this point in the history

  9. (currently failing) Add test to make sure membership changes don't re… 

    …-appear if the server leaves the room later
    @MadLittleMods
    MadLittleMods committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    956f20e View commit details
    Browse the repository at this point in the history

  10. Grab prev_membership to see whether the server left the room (fixes… 

    … tests)

See #17320 (comment)

`prev_membership` helps determine whether we should include the `event_id=null` row because
we can check whether we have already left.

 - When we leave the room causing the server to leave the room, the `prev_event_id` will be our join event
 - When the server leaves the room after us, the `prev_event_id` will be leave event
 - In the state reset case, `prev_event_id` will be our join event
    @MadLittleMods
    MadLittleMods committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    830e09d View commit details
    Browse the repository at this point in the history

  11. Slight clean-up

    @MadLittleMods
    MadLittleMods committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    15fcead View commit details
    Browse the repository at this point in the history

  12. Detect state resets

    @MadLittleMods
    MadLittleMods committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    81c06be View commit details
    Browse the repository at this point in the history

  13. Don't worry about state_reset for now 

    See:

 - Why no `COALESCE` #17320 (comment)
 - Don't worry about `state_reset` for now, #17320 (comment)
    @MadLittleMods
    MadLittleMods committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    eb159c1 View commit details
    Browse the repository at this point in the history

  14. Passing current tests

    @MadLittleMods
    MadLittleMods committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    ba56350 View commit details
    Browse the repository at this point in the history

  15. Add better comments

    @MadLittleMods
    MadLittleMods committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    f774032 View commit details
    Browse the repository at this point in the history

  16. Inclusive ranges

    @MadLittleMods
    MadLittleMods committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    325856e View commit details
    Browse the repository at this point in the history

  17. (doesn't work) Add test for batch persisting multiple member events f… 

    …or the same user
    @MadLittleMods
    MadLittleMods committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    63c7b50 View commit details
    Browse the repository at this point in the history

  18. Opt for tackling more batch scenarios in future PRs

    @MadLittleMods
    MadLittleMods committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    1158058 View commit details
    Browse the repository at this point in the history

  19. Add TODO to handle state resets 

    See #17320 (comment)
    @MadLittleMods
    MadLittleMods committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    32b8b68 View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2024

  1. Merge branch 'develop' into madlittlemods/sliding-sync-room-data

    @MadLittleMods
    MadLittleMods committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    6045e11 View commit details
    Browse the repository at this point in the history

  2. Avoid fetching full events for prev_event_ids 

    See #17320 (comment)
    @MadLittleMods
    MadLittleMods committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    9e53336 View commit details
    Browse the repository at this point in the history

  3. Update stream tests with prev event info

    @MadLittleMods
    MadLittleMods committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    a4263bf View commit details
    Browse the repository at this point in the history

Commits on Jul 2, 2024

  1. Protect for non-existent prev events

    @MadLittleMods
    MadLittleMods committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    10d78d6 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'develop' into madlittlemods/sliding-sync-room-data

    @MadLittleMods
    MadLittleMods committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    0061561 View commit details
    Browse the repository at this point in the history

  3. Select to_key if to_key else from_key 

    See #17320 (comment)
    @MadLittleMods
    MadLittleMods committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    b8687e7 View commit details
    Browse the repository at this point in the history

  4. Add missing test description

    @MadLittleMods
    MadLittleMods committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    7c9513c View commit details
    Browse the repository at this point in the history

  5. Trigger CI again

    @MadLittleMods
    MadLittleMods committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    8b73185 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'develop' into madlittlemods/sliding-sync-room-data

    @erikjohnston
    erikjohnston authored Jul 2, 2024
    Configuration menu
    Copy the full SHA
    126ce1e View commit details
    Browse the repository at this point in the history