vCard v4 contact entries with photos vanish

[Bodenseematze]

Describe the bug

I synchronize my NC addressbooks with Thunderbird and the plugin CardBook.
With that plugin I'm able to change all vCard entries of an addressbook to v4.

After doing this, when I go to the NC contacts app and click to a v4 entry with integrated photo, a message pops up saying "The contact doesn't exist anymore at the server"

So I assume there is either something wrong with the v4 entry or the NC app interprets something different as the Thunderbird's CardBook plugin.

BTW, why are exported entries from NC contacts always vCard v3 and not v4 as they actually are?

To Reproduce

1. Create a new addressbook with contacts app
2. Create an contact entry with photo
3. Add the addressbook to Thunderbird/CardBook, using vCard 4.0 format
4. Synchronize it
5. Convert all entries in Thunderbird/CardBook to v4
   (in German: in menu "Adressbuch bearbeiten" / "vCard" / "Konvertieren")
6. Synchronize again
7. Reopen contacts app (or change to other app and back to contacts app) in NC and the created entry

Expected behavior
The entry will be shown.

Actual behavior
A message pops up "The contact doesn't exists anymore on the server" and the entry vanishes.

The mentioned Issue Template app is not available for Nextcloud 15...

Operating system:
Server: Linux (Debian, Shared Hoster)
Client: Windows 10

Web server: Apache

Database: MariaDB 5.7

PHP version: 7.2

Nextcloud version: 15.0

Contacts version: 3.0.1

Updated from an older Nextcloud or fresh install: Update from latest V14

List of activated apps:

• activity: 2.8.2
• admin_audit: 1.5.0
• announcementcenter: 3.4.0
• apporder: 0.6.0
• audioplayer: 2.5.1
• bookmarks: 0.15.1
• calendar: 1.6.4
• cloud_federation_api: 0.1.0
• comments: 1.5.0
• contacts: 3.0.1
• dav: 1.8.1
• external: 3.2.0
• federatedfilesharing: 1.5.0
• federation: 1.5.0
• files: 1.10.0
• files_external: 1.6.0
• files_pdfviewer: 1.4.0
• files_sharing: 1.7.0
• files_texteditor: 2.7.0
• files_trashbin: 1.5.0
• files_versions: 1.8.0
• files_videoplayer: 1.4.0
• firstrunwizard: 2.4.0
• flowupload: 0.0.9
• gallery: 18.2.0
• gpxpod: 3.0.1
• groupfolders: 2.0.2
• impersonate: 1.2.0
• logreader: 2.0.0
• lookup_server_connector: 1.3.0
• metadata: 0.8.0
• nextcloud_announcements: 1.4.0
• notifications: 2.3.0
• oauth2: 1.3.0
• polls: 0.9.5
• previewgenerator: 2.0.0
• provisioning_api: 1.5.0
• serverinfo: 1.5.0
• sharebymail: 1.5.0
• systemtags: 1.5.0
• tasks: 0.9.8
• theming: 1.6.0
• twofactor_backupcodes: 1.4.1
• updatenotification: 1.5.0
• user_usage_report: 1.1.1
• workflowengine: 1.5.0

Nextcloud configuration:
{ "system": { "instanceid": "***REMOVED SENSITIVE VALUE***", "passwordsalt": "***REMOVED SENSITIVE VALUE***", "trusted_domains": [ "***REMOVED SENSITIVE VALUE***", "***REMOVED SENSITIVE VALUE***" ], "activity_expire_days": "60", "datadirectory": "***REMOVED SENSITIVE VALUE***", "filelocking.debug": false, "filelocking.enabled": true, "filelocking.ttl": 1800, "overwrite.cli.url": "***REMOVED SENSITIVE VALUE***", "version": "15.0.2.0", "installed": true, "dbtype": "mysql", "dbname": "***REMOVED SENSITIVE VALUE***", "dbhost": "***REMOVED SENSITIVE VALUE***", "dbtableprefix": "oc_", "dbuser": "***REMOVED SENSITIVE VALUE***", "dbpassword": "***REMOVED SENSITIVE VALUE***", "default_language": "de", "defaultapp": "files", "integrity.check.disabled": true, "knowledgebaseenabled": true, "enable_avatars": true, "allow_user_to_change_display_name": true, "theme": "", "secret": "***REMOVED SENSITIVE VALUE***", "loglevel": 3, "logdateformat": "Y-m-d H:i:s", "logtimezone": "Europe\/Berlin", "log_rotate_size": 20971520, "skeletondirectory": "\/home\/www\/nextcloud_data\/skeleton", "updatechecker": true, "trashbin_retention_obligation": "auto", "versions_retention_obligation": "disabled", "appstoreenabled": true, "appstore.experimental.enabled": true, "singleuser": false, "maintenance": false, "htaccess.RewriteBase": "\/", "mail_smtpmode": "smtp", "updater.release.channel": "stable", "mail_smtpsecure": "ssl", "mail_smtpauthtype": "LOGIN", "mail_from_address": "***REMOVED SENSITIVE VALUE***", "mail_smtpauth": 1, "mail_domain": "***REMOVED SENSITIVE VALUE***", "mail_smtphost": "***REMOVED SENSITIVE VALUE***", "mail_smtpport": "465", "mail_smtpname": "***REMOVED SENSITIVE VALUE***", "mail_smtppassword": "***REMOVED SENSITIVE VALUE***", "mysql.utf8mb4": true, "updater.secret": "***REMOVED SENSITIVE VALUE***" } }

Client configuration

Browser: Firefox 64.0, 64-Bit

Operating system:

Windows 10 - Pro

CardDAV-clients:

Thunderbird 60.3.1 (32-Bit) / CardBook-Plugin v33.5

Logs

Browser log

Test vCards

I added the vCard entries - exported by Thunderbird/CardBook (v3 & v4) and by NC15/contacts app (only v3):

Test-vCards.zip

[nextcloud-bot]

GitMate.io thinks possibly related issues are #16 (Problems while import vcards), #324 (VCard version), #176 (problems with huge addressbooks), and #779 (Additional phone number field not found).

[skjnldsv]

Hello!
Your last vcard is invalid.
PHOTO;DATA:IMAGE/JPEG;BASE64,/9j/4AAQSkZJRgABAQEASABIAAD/4YgaRXhpZgAATU0AKg
Should be
PHOTO:DATA:IMAGE/JPEG;BASE64,/9j/4AAQSkZJRgABAQEASABIAAD/4YgaRXhpZgAATU0AKg

Therefore, your vcard can't be read :)

[Bodenseematze]

You're right; there are some situations where the CardBook plugin creates wrong vcard entries!
I contacted the CardBook author and he is currently searching for the bug!

Inbetween: would it be possible to fix the entries by hand? Where are the vCards stored? In the database?
Would it be possible to execute a SQL statement to change the malformed entries?
Or to export the complete addressbook including the wrong entries and to edit it with a text editor and then import it back?

[skjnldsv]

Yes, you can fix manually the entity and import it back :)
CardBook is known to create some issues, we have plenty of tickets of invalid data from it ;)

The vcard is already in your nextcloud database? You imported it before updating to contacts 3.0?

[Bodenseematze]

Yes, the vcards were already in my database.
I fixed them by first exporting the complete addressbook (with CardBook), then looking for invalid entries by searing PHOTO:DATA,
then doing single export for the invalid entries (also with CardBook), fixing them by hand (editor),
deleting the entries from CardBook
and then importing them again from NC-contacts.

Fortunately there were only a few entries to fix :-)

[skjnldsv]

Awesome! Thanks for your feedback :)