Skip to content
This repository has been archived by the owner on Jan 10, 2023. It is now read-only.

error creating Google Drive client: gob: encoder: message too big #98

Closed
Namke opened this issue Jan 5, 2016 · 4 comments
Closed

error creating Google Drive client: gob: encoder: message too big #98

Namke opened this issue Jan 5, 2016 · 4 comments

Comments

@Namke
Copy link

Namke commented Jan 5, 2016

Today I experienced this issues while skicka try updating metadata cache, at some point it is stop and just spawn this message "error creating Google Drive client: gob: encoder: message too big"
This issues happen both in Windows and Mac, with go 1.5.2 installed.
Google around it seem internal issues of go and should be work around it for now.

@mmp
Copy link
Contributor

mmp commented Jan 7, 2016

Interesting! Do you have a large number of files / folders in Drive? That error is occurring when skicka attempts to serialize the cache of metadata about the files stored in Drive and store it on disk.

I reported this as a bug to the Go folks (golang/go#13850); we'll see what they say.

It would be fairly straightforward to work around this in skicka, though it'd require changing the serialization format for the metadata, which would be nice to avoid if possible. So we'll see if a fix is imminent on the Go side, or if this is claimed to be working as intended (in which case we'll definitely fix it over here.)

@Namke
Copy link
Author

Namke commented Jan 7, 2016

I checked with Go beta version (1.6.x), both in Mac and Windows and it still happen, guess it is not fix in Go side yet.
My drive contain huge amount of files (not in term of size but in term of number of files), the cache file size is now over 1GB. And yes, it is happen at the end of updating metadata file so I guess it is happen due to serialize the metadata cache.

Would be helpful if there are option to just push file to drive without syncing (force upload perhaps), I experience the case when skicka cost more time on update/serialize metadata than actual uploading.

mmp pushed a commit that referenced this issue Jan 8, 2016
Serialize the file id to File struct map manually, rather than by passing it to gob/Encode() in its entirety, as the Encode() implementation recently changed to issue an error if the map is very large.

(I believe that this will fix skicka issue #98.)
@mmp
Copy link
Contributor

mmp commented Jan 8, 2016

The fix just pushed should work-around the gloang issue from skicka. (Please let me know if it works!)

I opened issue #99 to track the request to not track/update metadata. I'll start looking through the code to think about what this would entail.

Thanks!

@Namke
Copy link
Author

Namke commented Jan 9, 2016

Just checked, it is work like a champ on both MacOS and Windows

Thanks!

@Namke Namke closed this as completed Jan 9, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants