Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Office files are corrupted/emptied after some time #3289

Open
6 of 8 tasks
cyberb opened this issue Nov 9, 2023 · 24 comments
Open
6 of 8 tasks

[Bug]: Office files are corrupted/emptied after some time #3289

cyberb opened this issue Nov 9, 2023 · 24 comments
Labels
0. Needs triage bug Something isn't working

Comments

@cyberb
Copy link

cyberb commented Nov 9, 2023

⚠️ This issue respects the following points: ⚠️

Bug description

I use Collabora Online server on the same box as Nextcloud to edit libre calc ods files and it works great I can edit a file on web and then on mobile phone.
But some times I come back to the file and it is empty, I check versions and they exist but they are all also empty no matter what revision I open.
This happened twice with different versions of Nextcloud, I lost hours of work :( I will have to make a copy manually and not open it using Nextcloud Office.

Steps to reproduce

  1. Create Calc file (ods)
  2. Edit on desktop browser
  3. Edit on mobile phone
  4. After some time open on desktop web and it is empty
  5. Open on mobile phone it is empty
  6. Check revision history all versions are empty (you can see it from their sizes)

Expected behavior

If some device corrupts the file it should at least not corrupt all the versions

Installation method

Other Community project

Nextcloud Server version

27

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.2

Web server

Nginx

Database engine version

PostgreSQL

Is this bug present after an update or on a fresh install?

None

Are you using the Nextcloud Server Encryption module?

None

What user-backends are you using?

  • Default user-backend (database)
  • LDAP/ Active Directory
  • SSO - SAML
  • Other

Configuration report

{
    "system": {
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "log_type": "syslog",
        "logfile": "\/var\/snap\/nextcloud\/common\/log\/nextcloud.log",
        "apps_paths": [
            {
                "path": "\/snap\/nextcloud\/current\/nextcloud\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/snap\/nextcloud\/current\/extra-apps",
                "url": "\/extra-apps",
                "writable": true
            }
        ],
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "localhost",
            "192.168.1.***",
            "nextcloud.***.syncloud.it"
        ],
        "dbtype": "pgsql",
        "version": "27.1.3.2",
        "overwrite.cli.url": "http:\/\/localhost",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "ldapIgnoreNamingRules": false,
        "ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "loglevel": "2",
        "mail_smtpmode": "smtp",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "maintenance": false,
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "default_phone_region": "GB",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_sendmailmode": "smtp",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauth": false,
        "check_data_directory_permissions": false
    }
}

List of activated Apps

Enabled:
  - activity: 2.19.0
  - circles: 27.0.1
  - cloud_federation_api: 1.10.0
  - comments: 1.17.0
  - contactsinteraction: 1.8.0
  - dashboard: 7.7.0
  - dav: 1.27.0
  - federatedfilesharing: 1.17.0
  - federation: 1.17.0
  - files: 1.22.0
  - files_pdfviewer: 2.8.0
  - files_reminders: 1.0.0
  - files_rightclick: 1.6.0
  - files_sharing: 1.19.0
  - files_trashbin: 1.17.0
  - files_versions: 1.20.0
  - firstrunwizard: 2.16.0
  - logreader: 2.12.0
  - lookup_server_connector: 1.15.0
  - music: 1.9.1
  - nextcloud_announcements: 1.16.0
  - notifications: 2.15.0
  - oauth2: 1.15.1
  - password_policy: 1.17.0
  - photos: 2.3.0
  - privacy: 1.11.0
  - provisioning_api: 1.17.0
  - recommendations: 1.6.0
  - related_resources: 1.2.0
  - richdocuments: 8.2.2
  - serverinfo: 1.17.0
  - settings: 1.9.0
  - sharebymail: 1.17.0
  - spreed: 17.1.2
  - support: 1.10.0
  - survey_client: 1.15.0
  - systemtags: 1.17.0
  - text: 3.8.0
  - theming: 2.2.0
  - twofactor_backupcodes: 1.16.0
  - user_ldap: 1.17.0
  - user_status: 1.7.0
  - viewer: 2.1.0
  - weather_status: 1.7.0
  - workflowengine: 2.9.0
Disabled:
  - admin_audit: 1.17.0
  - bruteforcesettings: 2.7.0
  - encryption: 2.15.0
  - files_external: 1.19.0
  - maps: 1.1.0 (installed 1.1.0)
  - suspicious_login: 5.0.0
  - twofactor_totp: 9.0.0

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

No response

Additional info

No response

@cyberb cyberb added 0. Needs triage bug Something isn't working labels Nov 9, 2023
@joshtrichards
Copy link
Member

Can you look in your Nextcloud server logs for entries mentioning one of these corrupted files? Both entries mentioning the file and immediately before it would be of interest.

Also, what external Collabora Online version/edition are you using?

Lastly, to clarify, you are able to edit in one place (i.e. web) the same file... Then go to your phone's Nextcloud client and edit the file successfully? But the. later the file disappears? Does it ever disappear prior to accessing from a second device?

@cyberb
Copy link
Author

cyberb commented Nov 11, 2023

Sorry I do not have logs now, but when I checked both nextcloud and collabora logs they had no exceptions and looked similar to other files.

Also, what external Collabora Online version/edition are you using?

I am actually extracting binaries from their docker image: collabora/code:21.11.4.2.1
And then I convert it to a snap package: https://github.com/syncloud/collabora/blob/master/app/Dockerfile

  1. I am able edit on the web and collabora web opens. Auto saves and I leave it running.
  2. Then continue adding more stuff on the mobile. Now as I opened the file again actually it is collabora mobile app
  3. Then opened on the web again and it was empty.

I can try to reproduce it to give more details, maybe it is related to user session exporation or something else.

@cyberb
Copy link
Author

cyberb commented Nov 11, 2023

Ok just tried to repro and it happened again!

  1. Create a new file on Nextcloud Android (test1.ods) embedded collabora calc opens (looks like embedded web one, not my collabora mobile app) add few test entries. Save. Not close.
  2. Open nextcloud we in firefox same mobile phone. File exists but empty.
  3. Go back to Nextcloud mobile app with open file, not it says taking too long to open (it was actually open, but I guess app switch causes to reload file). Click cancel, reopen file again and it is empty now.

Logs from Nextcloud server:

root@syncloud:~# journalctl -u snap.nextcloud.* | grep test1.ods
Nov 11 08:41:53 syncloud nginx[2915]: syncloud nginx: *** - boris [11/Nov/2023:08:41:53 +0000] "PROPFIND /remote.php/dav/files/boris/Temp/test1.ods HTTP/1.1" 207 1308 "-" "Mozilla/5.0 (Android) Nextcloud-android/3.26.0"
Nov 11 08:43:28 syncloud nginx[2914]: syncloud nginx: ***  - - [11/Nov/2023:08:43:28 +0000] "GET /apps/files/?dir=/Temp&openfile=1322457&scrollto=test1.ods HTTP/1.1" 200 16904 "-" "Mozilla/5.0 (Android 10; Mobile; rv:109.0) Gecko/119.0 Firefox/119.0"
Nov 11 08:43:33 syncloud nginx[2914]: syncloud nginx: ***  - - [11/Nov/2023:08:43:33 +0000] "GET /ocs/v2.php/search/providers?from=%2Fapps%2Ffiles%2F%3Fdir%3D%2FTemp%26openfile%3D1322457%26scrollto%3Dtest1.ods HTTP/1.1" 200 277 "-" "Mozilla/5.0 (Android 10; Mobile; rv:109.0) Gecko/119.0 Firefox/119.0"
Nov 11 08:43:38 syncloud nginx[2915]: syncloud nginx: ***  - - [11/Nov/2023:08:43:38 +0000] "PROPFIND /remote.php/dav/files/boris/Temp/test1.ods HTTP/1.1" 207 585 "-" "Mozilla/5.0 (Android 10; Mobile; rv:109.0) Gecko/119.0 Firefox/119.0"
Nov 11 08:44:22 syncloud nginx[2915]: syncloud nginx: ***  - boris [11/Nov/2023:08:44:22 +0000] "GET /index.php/apps/files/api/v1/thumbnail/256/256/Temp/test1.ods HTTP/1.1" 404 29 "-" "Mozilla/5.0 (Android) Nextcloud-android/3.26.0"

Collabora server logs:

journalctl -u snap.collabora.* | grep test1.ods
Nov 11 08:41:57 syncloud collabora.server[497]: wsd-01320-02476 2023-11-11 08:41:57.706792 +0000 [ websrv_poll ] INF  nextcloud/server#21: Client HTTP Request: POST /browser/3eff190/cool.html?WOPISrc=https%3A%2F%2Fnextcloud.***.syncloud.it%2Findex.php%2Fapps%2Frichdocuments%2Fwopi%2Ffiles%2F1322457_ocy0e4tzvl95&title=test1.ods&lang=en-GB&closebutton=1 HTTP/1.0 / Host: collabora.***.syncloud.it / Connection: close / Content-Length: 4246 / X-Forwarded-Proto: https / X-Forwarded-Host: collabora.***.syncloud.it / X-Real-IP: *** / X-Forwarded-For: *** / cache-control: max-age=0 / sec-ch-ua: "Chromium";v="118", "Android WebView";v="118", "Not=A?Brand";v="99" / sec-ch-ua-mobile: ?1 / sec-ch-ua-platform: "Android" / upgrade-insecure-requests: 1 / origin: null / content-type: application/x-www-form-urlencoded / user-agent: Mozilla/5.0 (Android) Nextcloud-android/3.26.0 / accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7 / x-requested-with: com.nextcloud.client / sec-fetch-site: same-site / sec-fetch-mode: navigate / sec-fetch-dest: iframe / accept-encoding: gzip, deflate, br / accept-language: en-GB,en-US;q=0.9,en;q=0.8| net/Socket.cpp:1071
Nov 11 08:41:57 syncloud nginx[1388]: syncloud nginx: *** - - [11/Nov/2023:08:41:57 +0000] "POST /browser/3eff190/cool.html?WOPISrc=https%3A%2F%2Fnextcloud.***.syncloud.it%2Findex.php%2Fapps%2Frichdocuments%2Fwopi%2Ffiles%2F1322457_ocy0e4tzvl95&title=test1.ods&lang=en-GB&closebutton=1 HTTP/1.1" 200 31706 "-" "Mozilla/5.0 (Android) Nextcloud-android/3.26.0"
Nov 11 08:41:58 syncloud collabora.server[497]: kit-13368-01946 2023-11-11 08:41:58.305061 +0000 [ kitbroker_00a ] INF  Saving the template document after loading to [file:///var/snap/collabora/current/child-roots/vxu5YoKVNakdaJVc//var/snap/collabora/current/child-roots/vxu5YoKVNakdaJVc/tmp/user/docs/vxu5YoKVNakdaJVc/test1.ods].| kit/ChildSession.cpp:741
Nov 11 08:41:58 syncloud collabora.server[497]: wsd-01320-01325 2023-11-11 08:41:58.607738 +0000 [ docbroker_00a ] ERR  Expected to renamed the document [/var/snap/collabora/current/child-roots/vxu5YoKVNakdaJVc/tmp/user/docs/vxu5YoKVNakdaJVc/test1.ods.upload] after template-loading to [/var/snap/collabora/current/child-roots/vxu5YoKVNakdaJVc/tmp/user/docs/vxu5YoKVNakdaJVc/test1.ods.uploading] (ENOENT: No such file or directory)| wsd/DocumentBroker.cpp:1379
Nov 11 08:41:59 syncloud collabora.server[497]: wsd-01320-01325 2023-11-11 08:41:59.338410 +0000 [ docbroker_00a ] ERR  Expected to renamed the document [/var/snap/collabora/current/child-roots/vxu5YoKVNakdaJVc/tmp/user/docs/vxu5YoKVNakdaJVc/test1.ods.upload] after template-loading to [/var/snap/collabora/current/child-roots/vxu5YoKVNakdaJVc/tmp/user/docs/vxu5YoKVNakdaJVc/test1.ods.uploading] (ENOENT: No such file or directory)| wsd/DocumentBroker.cpp:1379
Nov 11 08:43:38 syncloud collabora.server[497]: wsd-01320-02476 2023-11-11 08:43:38.756012 +0000 [ websrv_poll ] INF  nextcloud/server#21: Client HTTP Request: POST /browser/3eff190/cool.html?WOPISrc=https%3A%2F%2Fnextcloud.***.syncloud.it%2Findex.php%2Fapps%2Frichdocuments%2Fwopi%2Ffiles%2F1322457_ocy0e4tzvl95&title=%2FTemp%2Ftest1.ods&lang=en-GB&closebutton=1&revisionhistory=1 HTTP/1.0 / Host: collabora.***.syncloud.it / Connection: close / Content-Length: 4233 / X-Forwarded-Proto: https / X-Forwarded-Host: collabora.***.syncloud.it / X-Real-IP: *** / X-Forwarded-For: *** / user-agent: Mozilla/5.0 (Android 10; Mobile; rv:109.0) Gecko/119.0 Firefox/119.0 / accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8 / accept-language: en-GB / accept-encoding: gzip, deflate, br / content-type: application/x-www-form-urlencoded / origin: null / upgrade-insecure-requests: 1 / sec-fetch-dest: iframe / sec-fetch-mode: navigate / sec-fetch-site: same-site| net/Socket.cpp:1071
Nov 11 08:43:38 syncloud nginx[1391]: syncloud nginx: *** - - [11/Nov/2023:08:43:38 +0000] "POST /browser/3eff190/cool.html?WOPISrc=https%3A%2F%2Fnextcloud.***.syncloud.it%2Findex.php%2Fapps%2Frichdocuments%2Fwopi%2Ffiles%2F1322457_ocy0e4tzvl95&title=%2FTemp%2Ftest1.ods&lang=en-GB&closebutton=1&revisionhistory=1 HTTP/1.1" 200 31694 "-" "Mozilla/5.0 (Android 10; Mobile; rv:109.0) Gecko/119.0 Firefox/119.0"
Nov 11 08:44:28 syncloud collabora.server[497]: wsd-01320-02476 2023-11-11 08:44:28.980647 +0000 [ websrv_poll ] INF  nextcloud/server#21: Client HTTP Request: POST /browser/3eff190/cool.html?WOPISrc=https%3A%2F%2Fnextcloud.***.syncloud.it%2Findex.php%2Fapps%2Frichdocuments%2Fwopi%2Ffiles%2F1322457_ocy0e4tzvl95&title=test1.ods&lang=en-GB&closebutton=1 HTTP/1.0 / Host: collabora.***.syncloud.it / Connection: close / Content-Length: 4246 / X-Forwarded-Proto: https / X-Forwarded-Host: collabora.***.syncloud.it / X-Real-IP: *** / X-Forwarded-For: *** / cache-control: max-age=0 / sec-ch-ua: "Chromium";v="118", "Android WebView";v="118", "Not=A?Brand";v="99" / sec-ch-ua-mobile: ?1 / sec-ch-ua-platform: "Android" / upgrade-insecure-requests: 1 / origin: null / content-type: application/x-www-form-urlencoded / user-agent: Mozilla/5.0 (Android) Nextcloud-android/3.26.0 / accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7 / x-requested-with: com.nextcloud.client / sec-fetch-site: same-site / sec-fetch-mode: navigate / sec-fetch-dest: iframe / accept-encoding: gzip, deflate, br / accept-language: en-GB,en-US;q=0.9,en;q=0.8| net/Socket.cpp:1071
Nov 11 08:44:28 syncloud nginx[1391]: syncloud nginx: ***  - - [11/Nov/2023:08:44:28 +0000] "POST /browser/3eff190/cool.html?WOPISrc=https%3A%2F%2Fnextcloud.***.syncloud.it%2Findex.php%2Fapps%2Frichdocuments%2Fwopi%2Ffiles%2F1322457_ocy0e4tzvl95&title=test1.ods&lang=en-GB&closebutton=1 HTTP/1.1" 200 31706 "-" "Mozilla/5.0 (Android) Nextcloud-android/3.26.0"

@cyberb
Copy link
Author

cyberb commented Nov 11, 2023

All versions are empty now.
IMG_20231111_085622

@joshtrichards
Copy link
Member

joshtrichards commented Nov 11, 2023

Logs from Nextcloud server:

Those are all helpful, but these logs are from your web server (Nginx) not Nextcloud Server. Can you share those from the same time period? Thanks.

Oh also since some things (like errors) may not be logged with the user filename, can you share the Collabora logs for that time window rather than just grep'd by filename?

Also going to move this Issue over to the richdocuments repository for now.

@joshtrichards joshtrichards transferred this issue from nextcloud/server Nov 11, 2023
@cyberb
Copy link
Author

cyberb commented Nov 11, 2023

Just reproduced again using the same workflow, there is nothing from nextcloud itself, usually I have it in the same output from systemd as I run it using snap.nextcloud.phpfpm process.

Also I have this in the config:

 'log_type' => 'syslog',

@cyberb
Copy link
Author

cyberb commented Nov 11, 2023

I can try upgrading collabora to 23.05.5.3.1 to see if that helps, at least I can reproduce it now.

@cyberb
Copy link
Author

cyberb commented Nov 14, 2023

Ok, upgrade to 23.05.5.3.1 actually helped at least now I cannot reproduce the isssue, I will reopen if it happens again.

@cyberb cyberb closed this as completed Nov 14, 2023
@cyberb cyberb reopened this Nov 15, 2023
@cyberb
Copy link
Author

cyberb commented Nov 15, 2023

Sorry the problem is there still.
Yesterday I eas testing a file and was switching between mobile app and web collabora integrations, eberything was fine and changes instantly visible.
Today opened the same file on the web and it is empty and history shows empty changes.

Screenshot_2023-11-15-09-32-33-661_com nextcloud client

@cyberb
Copy link
Author

cyberb commented Nov 15, 2023

I will try to get more logs

@marcinkuk
Copy link

I heve exacltly the same issue with my nextclouds.
Have several nextcloud environements with different version and collabora or nextcloud sometimes eat my files.
I lost a lot of work.

@pehlert
Copy link

pehlert commented Jan 19, 2024

Same here, ods file completely empty after a while, showing the same size (683 B) for all versions.

Needless to say this is disastrous for a software who's main purpose is so safely store files.

@voggur
Copy link

voggur commented Feb 7, 2024

@artfulrobot
Copy link

This is a terrifying bug! Glad I found this report in time, I was going to recommend nextcloud office to a client, but I would be pilloried if this happened to their docs! As I understand, Nextcloud passes the file to CODE which then somehow passes it back after an update, which makes sense, but blanking all the previous versions?! I don't get that at all.

Not sure if it's related (and will check the issue queue) but I get similarly weird behaviours with Excalidraw - like content is just not there, or is old etc. Figured it could be related as it's another case of pass the doc out somewhere and store it when updated.

@zsQ8ymRSY8B4owpdCvTRdkz8rQUK

+1. Just had a client lose all of their work in an ods after they edited the file on their phone app. All versions became empty.

Has it been acknowledged if this is being worked on?

@estux
Copy link

estux commented Apr 17, 2024

It seems there's a link with mobile editing (with Collabora app) and it could be also a Collabora problem. Did someone opened an issue also on their side?

@estux
Copy link

estux commented Apr 20, 2024

It seems there's a link with mobile editing (with Collabora app) and it could be also a Collabora problem. Did someone opened an issue also on their side?

Replying to myself:
Linked to this bug CollaboraOnline/online#8242 in the Collabora Online repo.

@ServError
Copy link

Experienced this bug myself yesterday after installing built in CODE and migrating some financial documents. I actually couldn't find this issue at the time and stumbled upon only by accident today, so maybe it needs more tags. The data loss in previous versions was mortifying and made me look incompetent. It's disappointing to see that it was reported 7 months ago.

In my instance we were also playing with opening the document from the Nextcloud android app after creating and editing the file from the desktop browser.

@estux
Copy link

estux commented Jun 21, 2024

I'm just writing here to add that something about this seems it was fixed with the new release of Collabora CODE:
CollaboraOnline/richdocumentscode#238 (comment)
I hope at least this solves some of your problems.

@poiNt3D
Copy link

poiNt3D commented Sep 22, 2024

Hi.
I'm experiencing a similar issue, but only current version is overwritten with empty (0 kb).
Still can't get the exact stemps, but i'm just fiddeling around: scrolling up and down, adjusting my router settings, refreshing, etc. No real editing. But sometimes i get an empty file somehow.

@cyberb
Copy link
Author

cyberb commented Sep 23, 2024

What version of CODE are you on?

@bartart3d
Copy link

Using nextcloud office Version 8.4.7 on nextcloud 29.0.7

  • I have tried it to edit uploaded excel sheets, seemed to work very well.
  • Saving a copy in the online editor did result in a synced local file, but changes made in nextcloud office, in that new file, did not sync to my local file.
  • an update to 8.4.7 happened
  • opening the same online file after the update showed that all made changes were gone. The file was reverted to its non edited state.
  • reverting to previous versions did not return the edits. It's as if they never were made.

@cyberb
Copy link
Author

cyberb commented Oct 7, 2024

People say it is fixed in Collabora (CODE) 24.4.402.
What version do you have?
CODE is not nextcloud or nextcloud office app.

@bartart3d
Copy link

hello,
My installed version was 24.4.702, now about to upgrade to 24.4.802

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0. Needs triage bug Something isn't working
Projects
None yet
Development

No branches or pull requests