Bug: client crashes after downloading deltas and database optimization

[htw-ikt-noelle]

Describe the bug

It appears that the client crashes after downloading the delta from a personal shared folder and database optimization. The problem occurred recently, while the same configuration (client version, system setup,...) worked fine before.

Operating System Details

6.11.0-17-generic #17~24.04.2-Ubuntu SMP PREEMPT_DYNAMIC Mon Jan 20 22:48:29 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

Client Installation Method

From 3rd Party Source (PPA, OpenSuSE Build Service etc)

OneDrive Account Type

Personal

What is your OneDrive Application Version

onedrive v2.5.4-1+np1+1.1

What is your OneDrive Application Configuration

Reading configuration file: /home/***/.config/onedrive/config
Configuration file successfully loaded

WARNING: Your cURL/libcurl version (8.5.0) has known operational bugs that impact the use of this client.
         Please report this to your distribution, requesting an update to a newer cURL version, or consider upgrading it yourself for optimal stability.

Application version                          = onedrive v2.5.4-1+np1+1.1
Compiled with                                = LDC 2106
Curl version                                 = libcurl/8.5.0 OpenSSL/3.0.13 zlib/1.3 brotli/1.1.0 zstd/1.5.5 libidn2/2.3.7 libpsl/0.21.2 (+libidn2/2.3.7) libssh/0.10.6/openssl/zlib nghttp2/1.59.0 librtmp/2.3 OpenLDAP/2.6.7
User Application Config path                 = /home/***/.config/onedrive
System Application Config path               = /etc/onedrive
Applicable Application 'config' location     = /home/***/.config/onedrive/config
Configuration file found in config location  = true - using 'config' file values to override application defaults
Applicable 'sync_list' location              = /home/***/.config/onedrive/sync_list
Applicable 'items.sqlite3' location          = /home/***/.config/onedrive/items.sqlite3
Config option 'drive_id'                     = 
Config option 'sync_dir'                     = ~/OneDrive
Config option 'enable_logging'               = false
Config option 'log_dir'                      = /var/log/onedrive
Config option 'disable_notifications'        = false
Config option 'skip_dir'                     = 
Config option 'skip_dir_strict_match'        = false
Config option 'skip_file'                    = ~*|.~*|*.tmp|*.swp|*.partial
Config option 'skip_dotfiles'                = false
Config option 'skip_symlinks'                = false
Config option 'monitor_interval'             = 300
Config option 'monitor_log_frequency'        = 12
Config option 'monitor_fullscan_frequency'   = 12
Config option 'read_only_auth_scope'         = false
Config option 'dry_run'                      = false
Config option 'upload_only'                  = false
Config option 'download_only'                = false
Config option 'local_first'                  = false
Config option 'check_nosync'                 = false
Config option 'check_nomount'                = false
Config option 'resync'                       = false
Config option 'resync_auth'                  = false
Config option 'cleanup_local_files'          = false
Config option 'classify_as_big_delete'       = 1000
Config option 'disable_upload_validation'    = false
Config option 'disable_download_validation'  = false
Config option 'bypass_data_preservation'     = false
Config option 'no_remote_delete'             = false
Config option 'remove_source_files'          = false
Config option 'sync_dir_permissions'         = 700
Config option 'sync_file_permissions'        = 600
Config option 'space_reservation'            = 52428800
Config option 'permanent_delete'             = false
Config option 'application_id'               = d50ca740-c83f-4d1b-b616-12c519384f0c
Config option 'azure_ad_endpoint'            = 
Config option 'azure_tenant_id'              = 
Config option 'user_agent'                   = ISV|abraunegg|OneDrive Client for Linux/v2.5.4-1+np1+1.1
Config option 'force_http_11'                = true
Config option 'debug_https'                  = false
Config option 'rate_limit'                   = 0
Config option 'operation_timeout'            = 3600
Config option 'dns_timeout'                  = 60
Config option 'connect_timeout'              = 10
Config option 'data_timeout'                 = 60
Config option 'ip_protocol_version'          = 1
Config option 'threads'                      = 8
Config option 'max_curl_idle'                = 120
Environment var 'XDG_RUNTIME_DIR'            = true
Environment var 'DBUS_SESSION_BUS_ADDRESS'   = true
Config option 'notify_file_actions'          = false

Selective sync 'sync_list' configured        = false

Config option 'sync_business_shared_items'   = false

Config option 'webhook_enabled'              = false

What is your 'curl' version

curl 8.5.0 (x86_64-pc-linux-gnu) libcurl/8.5.0 OpenSSL/3.0.13 zlib/1.3 brotli/1.1.0 zstd/1.5.5 libidn2/2.3.7 libpsl/0.21.2 (+libidn2/2.3.7) libssh/0.10.6/openssl/zlib nghttp2/1.59.0 librtmp/2.3 OpenLDAP/2.6.7
Release-Date: 2023-12-06, security patched: 8.5.0-2ubuntu10.6
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM PSL SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd

Where is your 'sync_dir' located

Local

What are all your system 'mount points'

sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,nosuid,relatime,size=16244816k,nr_inodes=4061204,mode=755,inode64)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,noexec,relatime,size=3256800k,mode=755,inode64)
/dev/mapper/ubuntu--vg-ubuntu--lv on / type ext4 (rw,relatime)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,inode64)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k,inode64)
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
efivarfs on /sys/firmware/efi/efivars type efivarfs (rw,nosuid,nodev,noexec,relatime)
bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=32,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5452)
debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,nosuid,nodev,relatime,pagesize=2M)
tracefs on /sys/kernel/tracing type tracefs (rw,nosuid,nodev,noexec,relatime)
configfs on /sys/kernel/config type configfs (rw,nosuid,nodev,noexec,relatime)
fusectl on /sys/fs/fuse/connections type fusectl (rw,nosuid,nodev,noexec,relatime)
/var/lib/snapd/snaps/bare_5.snap on /snap/bare/5 type squashfs (ro,nodev,relatime,errors=continue,threads=single,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/code_183.snap on /snap/code/183 type squashfs (ro,nodev,relatime,errors=continue,threads=single,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/code_184.snap on /snap/code/184 type squashfs (ro,nodev,relatime,errors=continue,threads=single,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/core20_2496.snap on /snap/core20/2496 type squashfs (ro,nodev,relatime,errors=continue,threads=single,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/core20_2434.snap on /snap/core20/2434 type squashfs (ro,nodev,relatime,errors=continue,threads=single,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/core22_1748.snap on /snap/core22/1748 type squashfs (ro,nodev,relatime,errors=continue,threads=single,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/core22_1564.snap on /snap/core22/1564 type squashfs (ro,nodev,relatime,errors=continue,threads=single,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/firefox_4793.snap on /snap/firefox/4793 type squashfs (ro,nodev,relatime,errors=continue,threads=single,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/firefox_5783.snap on /snap/firefox/5783 type squashfs (ro,nodev,relatime,errors=continue,threads=single,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/firmware-updater_127.snap on /snap/firmware-updater/127 type squashfs (ro,nodev,relatime,errors=continue,threads=single,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/firmware-updater_167.snap on /snap/firmware-updater/167 type squashfs (ro,nodev,relatime,errors=continue,threads=single,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/gnome-42-2204_176.snap on /snap/gnome-42-2204/176 type squashfs (ro,nodev,relatime,errors=continue,threads=single,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/gnome-42-2204_202.snap on /snap/gnome-42-2204/202 type squashfs (ro,nodev,relatime,errors=continue,threads=single,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/gtk-common-themes_1535.snap on /snap/gtk-common-themes/1535 type squashfs (ro,nodev,relatime,errors=continue,threads=single,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/obsidian_42.snap on /snap/obsidian/42 type squashfs (ro,nodev,relatime,errors=continue,threads=single,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/libreoffice_336.snap on /snap/libreoffice/336 type squashfs (ro,nodev,relatime,errors=continue,threads=single,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/obsidian_43.snap on /snap/obsidian/43 type squashfs (ro,nodev,relatime,errors=continue,threads=single,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/snap-store_1173.snap on /snap/snap-store/1173 type squashfs (ro,nodev,relatime,errors=continue,threads=single,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/snap-store_1248.snap on /snap/snap-store/1248 type squashfs (ro,nodev,relatime,errors=continue,threads=single,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/snapd_21759.snap on /snap/snapd/21759 type squashfs (ro,nodev,relatime,errors=continue,threads=single,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/snapd-desktop-integration_178.snap on /snap/snapd-desktop-integration/178 type squashfs (ro,nodev,relatime,errors=continue,threads=single,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/snapd_23545.snap on /snap/snapd/23545 type squashfs (ro,nodev,relatime,errors=continue,threads=single,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/snapd-desktop-integration_253.snap on /snap/snapd-desktop-integration/253 type squashfs (ro,nodev,relatime,errors=continue,threads=single,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/thunderbird_634.snap on /snap/thunderbird/634 type squashfs (ro,nodev,relatime,errors=continue,threads=single,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/thunderbird_644.snap on /snap/thunderbird/644 type squashfs (ro,nodev,relatime,errors=continue,threads=single,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/zoom-client_242.snap on /snap/zoom-client/242 type squashfs (ro,nodev,relatime,errors=continue,threads=single,x-gdu.hide,x-gvfs-hide)
/dev/nvme0n1p2 on /boot type ext4 (rw,relatime)
/dev/nvme0n1p1 on /boot/efi type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,nosuid,nodev,noexec,relatime)
tmpfs on /run/snapd/ns type tmpfs (rw,nosuid,nodev,noexec,relatime,size=3256800k,mode=755,inode64)
nsfs on /run/snapd/ns/snapd-desktop-integration.mnt type nsfs (rw)
tmpfs on /run/user/1000 type tmpfs (rw,nosuid,nodev,relatime,size=3256796k,nr_inodes=814199,mode=700,uid=1000,gid=1000,inode64)
portal on /run/user/1000/doc type fuse.portal (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)
gvfsd-fuse on /run/user/1000/gvfs type fuse.gvfsd-fuse (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)
nsfs on /run/snapd/ns/firefox.mnt type nsfs (rw)
nsfs on /run/snapd/ns/thunderbird.mnt type nsfs (rw)
nsfs on /run/snapd/ns/snap-store.mnt type nsfs (rw)
nsfs on /run/snapd/ns/firmware-updater.mnt type nsfs (rw)

What are all your local file system partition types

nvme0n1                                                                                                    
├─nvme0n1p1               vfat        FAT32          E1C3-53AC                                   1G     3% /boot/efi
├─nvme0n1p2               ext4        1.0            47e19f46-72cc-4edc-8f2b-a5eee65d6032      1,7G     5% /boot
└─nvme0n1p3               LVM2_member LVM2 001       nin9Tv-hsRs-2VO1-mg9B-Lx9O-a42D-vqvS1k                
  └─ubuntu--vg-ubuntu--lv ext4        1.0            b13a8459-6f82-4f71-bbec-39fe645df827    793,9G    10% /

How do you use 'onedrive'

I normally use the client as a service to sync my local folders with OneDrive between two Ubuntu installations on two different laptops. Both systems recently started to show the same "crash behaviour".

The same problem occurs if I start the client manually with onedrive -s -v

Steps to reproduce the behaviour

onedrive -s -v

Complete Verbose Log Output

$ onedrive -s -v
Reading configuration file: /home/***/.config/onedrive/config
Configuration file successfully loaded
Using 'user' configuration path for application config and state data: /home/***/.config/onedrive
D-Bus message bus daemon is available; GUI notifications are now enabled

WARNING: Your cURL/libcurl version (8.5.0) has known operational bugs that impact the use of this client.
         Please report this to your distribution, requesting an update to a newer cURL version, or consider upgrading it yourself for optimal stability.

Forcing client to use IPv4 connections only
Attempting to contact Microsoft OneDrive Login Service
Successfully reached Microsoft OneDrive Login Service
Checking Application Version ...
Attempting to initialise the OneDrive API ...
Configuring Global Azure AD Endpoints
The OneDrive API was initialised successfully
Opening the item database ...
Downgrading all HTTP operations to HTTP/1.1 due to user configuration
Application Version:  onedrive v2.5.4-1+np1+1.1
Account Type:         personal
Default Drive ID:     24d0df035216dbd7
Default Root ID:      24D0DF035216DBD7!108
Remaining Free Space: 978.50 GB (1050656374784 bytes)
Sync Engine Initialised with new Onedrive API instance
All application operations will be performed in the configured local 'sync_dir' directory: /home/***/OneDrive
Fetching /delta response from the OneDrive API for Drive ID: 24d0df035216dbd7
Processing API Response Bundle: 1 - Quantity of 'changes|items' in this bundle to process: 0
Finished processing /delta JSON response from the OneDrive API
No changes or items that can be applied were discovered while processing the data received from Microsoft OneDrive
Syncing this OneDrive Personal Shared Folder: ./***
Generating a /delta response from the OneDrive API for this Drive ID: 216f35c81ce6d8b9 and Item ID: 216F35C81CE6D8B9!106
Adding OneDrive folder details for processing
Adding 4 OneDrive items for processing from the OneDrive Shared Folder

//*** removed lines because of sensitive data ***//

Attempting to perform a database vacuum to optimise database
Database vacuum is complete
core.exception.AssertError@src/itemdb.d(892): Assertion failure
----------------
??:? _d_assert [0x7acdb176e942]
??:? [0x59360bbf9f01]
??:? [0x59360bc08d22]
??:? [0x59360bc1a76f]
??:? [0x59360bc13fc3]
??:? [0x59360bc07763]
??:? [0x59360bc0547a]
??:? [0x59360bcd0636]
??:? [0x59360bccd0ff]
??:? void rt.dmain2._d_run_main2(char[][], ulong, extern (C) int function(char[][])*).runAll() [0x7acdb17a5dcc]
??:? _d_run_main2 [0x7acdb17a5be6]
??:? _d_run_main [0x7acdb17a5a3c]
??:? [0x59360bcf9f61]
??:? [0x7acdb142a1c9]
??:? __libc_start_main [0x7acdb142a28a]
??:? [0x59360bbc8294]

Screenshots

No response

Other Log Information or Details

Additional context

No response

[abraunegg]

@htw-ikt-noelle
This is the same issue as:

• Bug: Application crash when calculating path due to 15 character driveId API bug #3115
• Bug: Can't sync OneDrive Personal Shared Folders due to inconsistent API response values #3136

[abraunegg]

@htw-ikt-noelle
Closing issue. Please contribute to one of those open issues , #3115 preferred

[abraunegg]

To those watching | impacted by this bug(s)

I was working on this issue heavily yesterday (1st March 2025) and the solution was working rather well with all testing.

This morning, when I was re validating the code, overnight there has been an account change to all of my OneDrive Personal accounts where the accounts now have the 'sea8cc6beffdb43d7976fbc7da445c639' Microsoft change.

As such, when I now query the API for all the folders online, the /delta query that this client heavily uses no longer is providing the Shared Folder JSON data that the /delta query was providing yesterday before the 'sea8cc6beffdb43d7976fbc7da445c639' account change.

I have raised a bug with Microsoft for this issue - but until this data quality issue from the Microsoft OneDrive API is fixed, a full resolution for this insideous series of bugs caused by this 'sea8cc6beffdb43d7976fbc7da445c639' change cannot be resolved.

The new API bug is listed here: OneDrive/onedrive-api-docs#1891

[abraunegg]

To those watching | impacted by this bug(s)

Caution

This maybe a 100% coding fix - however at the moment this needs a workaround to also be implemented by impacted users

Please can you test this updated PR and provide valuable feedback.

To test the PR, first install all the require platform dependencies to build the client on your respective platforms. Please read https://github.com/abraunegg/onedrive/blob/master/docs/install.md#building-from-source---high-level-requirements and then follow correctly for your platform.

Important

Currently there is an issue with installing the DMD or LDC compiler using the documentation instructions:
curl -fsS https://dlang.org/install.sh | bash -s dmd or curl -fsS https://dlang.org/install.sh | bash -s ldc

This is not something I can solve and I have emailed the appropriate folk and raised dlang/dmd#20830

Please augment the installation of DMD or LDC with your distribution version if it is available

Once this is done, to clone the PR to resolve your issue, you can use a script like the following:

#!/bin/bash

PR=3116

rm -rf ./onedrive-pr${PR}
git clone https://github.com/abraunegg/onedrive.git onedrive-pr${PR}
cd onedrive-pr${PR}
git fetch origin pull/${PR}/head:pr${PR}
git checkout pr${PR}

# Configure and Build
./configure --enable-debug --enable-notifications; make clean; make;
./onedrive --version

This script will create a local folder called onedrive-pr3116 with the PR version.

Important

Before running this PR please add to your configuration file force_children_scan = "true"

This will bypass the new OneDrive API /delta bug OneDrive/onedrive-api-docs#1891 where OneDrive Personal Folder details are no longer sent in the /delta response when your account has been migrated to the new Microsoft OneDrive platform. You can tell when you are impacted as when you run the application in verbose mode, it will look like this:

The OneDrive API was initialised successfully
Opening the item database ...
Forcing client to use /children API call rather than /delta API to retrieve objects from the OneDrive API
Application Version:   onedrive v2.5.4-29-g5d836ad
Account Type:          personal
Default Drive ID:      66D53BE8A5056ECA
Default Root ID:       66D53BE8A5056ECA!sea8cc6beffdb43d7976fbc7da445c639
Microsoft Data Centre: Australia Southeast

To run the PR, you need to run the client from the PR build directory:

./onedrive <any other options needed>

To install the PR, you will need to perform sudo make install to install the PR version to your system.

When running the PR, your version should be: onedrive v2.5.4-29-g5d836ad or greater.

If you still encounter an issue, please generate a new verbose debug log following this process: https://github.com/abraunegg/onedrive/wiki/Generate-debug-log-for-support

[abraunegg]

To those watching | impacted by this bug(s)

Please rebuild your PR client version to onedrive v2.5.4-30-g43ba76c

A further UPPERCASE / lowercase API inconsistency has been uncovered, that required remediation as part of this PR.

Please can you ensure you are running this version + apply the force_children_scan = "true" temporary work around to get your system back into an operational state.

Whilst your client will run slower due to manually building a /delta response, Microsoft need to fix their API to provide again the Shared Folder data in the /delta API query, that is now removed when your account is moved to their new backend platform.

Please read OneDrive/onedrive-api-docs#1891 for details.

[abraunegg]

To those watching | impacted by this bug(s)

Please rebuild your PR client version to onedrive v2.5.4-33-gd45ca81

It potentially appears that Microsoft has resolved the /delta API query bug that failed to send OneDrive Personal Shared Folder details, thus potentially you may be able to stop the use of the force_children_scan = "true" temporary workaround.

If you get this sort of error or similar without force_children_scan = "true" ..

ERROR: Microsoft OneDrive API returned an error with the following message:
  Error Message:    HTTP request returned status code 404 (Not Found)
  Error Reason:     Item not found
  Error Code:       itemNotFound
  Error Timestamp:  2025-03-07T04:17:05
  API Request ID:   65c928ec-97b5-4c63-9379-d905dedd9706
  Calling Function: syncEngine.performNewFileUpload()

You need to re-enable force_children_scan = "true"

When using this client version, please use --resync --resync-auth to ensure that your local database is cleaned up to remove any of the old database entries.

Please can you test this client version and advise.

[abraunegg]

Closing this issue as fixed, and is being tracked by #3115