Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

many syncv3 optimisations; database, RocksDB, and misc performance improvements; refactoring and cleanup; fix private read receipt support; and some more bug fixes #636

Merged
merged 71 commits into from
Dec 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
8e8c6bf
abstract supported room versions apis
jevolk Dec 5, 2024
de3b137
eliminate future wrapping stream for all_pdus()
jevolk Dec 6, 2024
23cf2b2
add is_err() to TryFuture extension
jevolk Dec 6, 2024
b77a1eb
move cork to remove_to_device_events fn
jevolk Dec 6, 2024
b7a41f2
optimize sync v3 asynchronicity
jevolk Dec 6, 2024
da984d4
remove unnecessary tracing of pdu conversions
jevolk Dec 7, 2024
d921b82
add options for console tracing EnvFilter details
jevolk Dec 7, 2024
aa6d0fc
add unwrap_or_default() to future TryExt extension
jevolk Dec 7, 2024
65a3708
derive specific ReadOptions for iterators
jevolk Dec 8, 2024
139a1ac
configure the tokio runtime tunables
jevolk Dec 8, 2024
6aa3526
misc database options tweaks
jevolk Dec 8, 2024
ff56ee7
add back the consume_budget on cache hits
jevolk Dec 8, 2024
82133ee
bump rust-rocksdb
jevolk Dec 8, 2024
f70fdca
remove jemalloc_stats from main default features
jevolk Dec 8, 2024
a752fb9
rename mod with tracing inits due to macro issues
jevolk Dec 8, 2024
35e9d9b
fix duplicate timezone keys being sent on profile lookup requests
girlbossceo Dec 7, 2024
c070edc
fix profile updates reusing old membership content + small parallelis…
girlbossceo Dec 7, 2024
a1b0369
reduce line width on banned_room_check
girlbossceo Dec 7, 2024
cf71aee
fix+improve membership endpoint UX, dont reuse old membership reasons…
girlbossceo Dec 7, 2024
1606441
disable admin_room_notices for complement, update welcome text
girlbossceo Nov 21, 2024
6167037
add rocksdb paranoid_file_checks config option, add some more config …
girlbossceo Dec 8, 2024
9d59f77
refactor fed membership endpoints, add missing checks, some cleanup, …
girlbossceo Dec 7, 2024
ab4a283
update complement config
girlbossceo Dec 9, 2024
faf4840
bump rocksdb to v9.8.4, enable track_and_verify_wals_in_manifest by d…
girlbossceo Dec 9, 2024
e1bd6a7
nix: try bumping to latest complement
girlbossceo Dec 9, 2024
b6ac364
use fork of complement
girlbossceo Dec 9, 2024
0ff37b8
update default systemd units to use network-online.target fully
girlbossceo Dec 9, 2024
9cb71e2
nix: bump flake.lock finally after 3 months
girlbossceo Dec 9, 2024
cd67ca6
fix .gitignore to allow /target to be a symlink
jevolk Dec 10, 2024
5fc8e90
fix runtime metrics command
jevolk Dec 10, 2024
aba88cc
misc tracing span tweaks
jevolk Dec 8, 2024
34f9e32
additional sync v3 refactoring/optimizations and tracing instruments
jevolk Dec 8, 2024
badb834
fix private read receipt support
girlbossceo Dec 11, 2024
1e3bb02
bump cargo.lock, bump some deps
girlbossceo Dec 11, 2024
ef8392c
allow empty POST request bodies (synapse behaviour) to improve compat…
girlbossceo Dec 11, 2024
80f9536
rename config.rs to account_data.rs, refactor, add some missing checks
girlbossceo Dec 11, 2024
ac0545f
always set limited true on left timeline events in sync (for now)
girlbossceo Dec 11, 2024
39abd9a
(temp) drop sccache version down in CI to 0.8.2
girlbossceo Dec 11, 2024
9ca964a
update stored default push rules when removing legacy mentions
girlbossceo Dec 11, 2024
af08d3f
document 'DNS No connections available' troubleshooting
girlbossceo Dec 11, 2024
0740d72
document modifying systemd unit if changing database path
girlbossceo Dec 11, 2024
5d51adc
update complement test results
girlbossceo Dec 12, 2024
3e64a5c
delete pointless bad_query_ratelimiter harming e2ee performance
girlbossceo Dec 12, 2024
66a8244
log hostname in DNS error tracing spans
girlbossceo Dec 12, 2024
353dc17
construct private read receipt properly
girlbossceo Dec 12, 2024
c9bb943
mark device list updates when a device is deleted/removed
girlbossceo Dec 12, 2024
a26b1c5
send private read receipt on sliding stink (sync)
girlbossceo Dec 12, 2024
e044618
remove unnecessary args in pack_receipts
girlbossceo Dec 12, 2024
76a5a67
fix a couple rust 2024 lints introduced in rust 1.82.0
girlbossceo Dec 12, 2024
b73d558
start producing haswell target optimised x86 binaries for rocksdb fas…
girlbossceo Dec 13, 2024
c5aca80
add auto upload assets, upload binary to webserver
AsenHu Dec 13, 2024
17273b1
upload CI binaries to webserver for easy direct download
girlbossceo Dec 13, 2024
1b9c331
use database::Json() to serialise MSC4133 profile value before insert
girlbossceo Dec 13, 2024
a9f6bb6
remove jemalloc_prof and jemalloc_stats from all-features builds (exc…
girlbossceo Dec 13, 2024
e4489a5
bump rust-rocksdb fork
girlbossceo Dec 14, 2024
c6bf8f5
improve gh actions security using zizmor
girlbossceo Dec 15, 2024
0317cc8
rename conduit to conduwuit finally
girlbossceo Dec 15, 2024
77e0b76
apply new rustfmt.toml changes, fix some clippy lints
girlbossceo Dec 15, 2024
52693db
hide messages from users servers on the forbidden_remote_server_names…
girlbossceo Dec 15, 2024
6c96acc
use fast binary_search for some const slices
girlbossceo Dec 15, 2024
c834998
ping online presence on typing updates like synapse
girlbossceo Dec 15, 2024
9c6908b
bump ruwuma, implement MSC4076, add more pusher HTTP URL validation
girlbossceo Dec 15, 2024
4680713
ping online presence on read updates like synapse
girlbossceo Dec 15, 2024
2ba0400
return proper error for attempting to update m.room.create
girlbossceo Dec 15, 2024
74c5bfd
dont allow m.room.server_acl to be redacted
girlbossceo Dec 15, 2024
aeae67a
bump ruwuma to stop sending empty content rulesets
girlbossceo Dec 15, 2024
004671b
dont copy join_authorized_via_users_server on leaving rooms
girlbossceo Dec 15, 2024
c8c9d73
enable edns0 on hickory resolver config
girlbossceo Dec 15, 2024
7ad710d
bump complement
girlbossceo Dec 15, 2024
8dcdb4b
update some more docs
girlbossceo Dec 16, 2024
5813419
free up ci runner space on both jobs
girlbossceo Dec 16, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
1 change: 1 addition & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,4 @@ indent_size = 2

[*.rs]
indent_style = tab
max_line_length = 98
395 changes: 303 additions & 92 deletions .github/workflows/ci.yml

Large diffs are not rendered by default.

8 changes: 6 additions & 2 deletions .github/workflows/documentation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ concurrency:
group: "pages"
cancel-in-progress: false

permissions: {}

jobs:
docs:
name: Documentation and GitHub Pages
Expand All @@ -61,6 +63,8 @@ jobs:

- name: Sync repository
uses: actions/checkout@v4
with:
persist-credentials: false

- name: Setup GitHub Pages
if: (startsWith(github.ref, 'refs/tags/v') || github.ref == 'refs/heads/main') && (github.event_name != 'pull_request')
Expand Down Expand Up @@ -109,8 +113,8 @@ jobs:
if: ${{ (env.ATTIC_ENDPOINT != '') && (env.ATTIC_PUBLIC_KEY != '') }}
run: |
sudo tee -a "${XDG_CONFIG_HOME:-$HOME/.config}/nix/nix.conf" > /dev/null <<EOF
extra-substituters = ${{ env.ATTIC_ENDPOINT }}
extra-trusted-public-keys = ${{ env.ATTIC_PUBLIC_KEY }}
extra-substituters = ${ATTIC_ENDPOINT}
extra-trusted-public-keys = ${ATTIC_PUBLIC_KEY}
EOF

- name: Prepare build environment
Expand Down
114 changes: 114 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
name: Upload Release Assets

on:
release:
types: [published, prereleased]
workflow_dispatch:
inputs:
tag:
description: 'Tag to release'
required: true
type: string
action_id:
description: 'Action ID of the CI run'
required: true
type: string

permissions: {}

jobs:
publish:
runs-on: ubuntu-latest
permissions:
contents: write
env:
GH_EVENT_NAME: ${{ github.event_name }}
GH_EVENT_INPUTS_ACTION_ID: ${{ github.event.inputs.action_id }}
GH_EVENT_INPUTS_TAG: ${{ github.event.inputs.tag }}
GH_REPOSITORY: ${{ github.repository }}
GH_SHA: ${{ github.sha }}
GH_TAG: ${{ github.event.release.tag_name }}

steps:
- name: get latest ci id
id: get_ci_id
env:
GH_TOKEN: ${{ github.token }}
run: |
if [ "${GH_EVENT_NAME}" == "workflow_dispatch" ]; then
id="${GH_EVENT_INPUTS_ACTION_ID}"
tag="${GH_EVENT_INPUTS_TAG}"
else
# get all runs of the ci workflow
json=$(gh api "repos/${GH_REPOSITORY}/actions/workflows/ci.yml/runs")

# find first run that is github sha and status is completed
id=$(echo "$json" | jq ".workflow_runs[] | select(.head_sha == \"${GH_SHA}\" and .status == \"completed\") | .id" | head -n 1)

if [ ! "$id" ]; then
echo "No completed runs found"
echo "ci_id=0" >> "$GITHUB_OUTPUT"
exit 0
fi

tag="${GH_TAG}}"
fi

echo "ci_id=$id" >> "$GITHUB_OUTPUT"
echo "tag=$tag" >> "$GITHUB_OUTPUT"

- name: get latest ci artifacts
if: steps.get_ci_id.outputs.ci_id != 0
uses: actions/download-artifact@v4
env:
GH_TOKEN: ${{ github.token }}
with:
merge-multiple: true
run-id: ${{ steps.get_ci_id.outputs.ci_id }}
github-token: ${{ github.token }}

- run: |
ls

- name: upload release assets
if: steps.get_ci_id.outputs.ci_id != 0
env:
GH_TOKEN: ${{ github.token }}
TAG: ${{ steps.get_ci_id.outputs.tag }}
run: |
for file in $(find . -type f); do
echo "Uploading $file..."
gh release upload $TAG "$file" --clobber --repo="${GH_REPOSITORY}" || echo "Something went wrong, skipping."
done

- name: upload release assets to website
if: steps.get_ci_id.outputs.ci_id != 0
env:
TAG: ${{ steps.get_ci_id.outputs.tag }}
run: |
mkdir -p -v ~/.ssh

echo "${{ secrets.WEB_UPLOAD_SSH_KNOWN_HOSTS }}" >> ~/.ssh/known_hosts
echo "${{ secrets.WEB_UPLOAD_SSH_PRIVATE_KEY }}" >> ~/.ssh/id_ed25519

chmod 600 ~/.ssh/id_ed25519

cat >>~/.ssh/config <<END
Host website
HostName ${{ secrets.WEB_UPLOAD_SSH_HOSTNAME }}
User ${{ secrets.WEB_UPLOAD_SSH_USERNAME }}
IdentityFile ~/.ssh/id_ed25519
StrictHostKeyChecking yes
AddKeysToAgent no
ForwardX11 no
BatchMode yes
END

echo "Creating tag directory on web server"
ssh -q website "rm -rf /var/www/girlboss.ceo/~strawberry/conduwuit/releases/$TAG/"
ssh -q website "mkdir -v /var/www/girlboss.ceo/~strawberry/conduwuit/releases/$TAG/"

for file in $(find . -type f); do
echo "Uploading $file to website"
scp $file website:/var/www/girlboss.ceo/~strawberry/conduwuit/releases/$TAG/$file
done
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ modules.xml
.nfs*

# Rust
/target/
/target

### vscode ###
.vscode/*
Expand Down
Loading
Loading