CodeChecker store fails to process analysis, crashes with Error -5 while decompressing data: incomplete or truncated stream', extraInfo=None

[kenroser]

Describe the bug
I can successfully analyze and parse my results. My problem begins when I try to use the server and store the results. The store process aborts with errors. See attached file

CodeChecker version

CodeChecker analyzer version:

Kind | Version

Base package version | 6.13.0
Package build date | 2020-07-08T17:44
Git commit ID (hash) | db517a9
Git tag information | 6.13

CodeChecker web version:

Kind | Version

Base package version | 6.13.0
Package build date | 2020-07-08T17:44
Git commit ID (hash) | db517a9
Git tag information | 6.13
Server supported API (Thrift) | 6.28
Client API (Thrift) | 6.28

To Reproduce
CodeChecker store ./reports --name uu --url http://ramapo.wi.direct.com:61511/Default

Expected behaviour
No crash

Desktop (please complete the following information)

• OS: Linux
• Browser: Firefox

Additional context
output.txt

[csordasmarton]

@kenroser In case of SQLite the maximum number of host parameters in a single SQL statement is set to 999 (SQLite version <= 3.32.0 /https://www.sqlite.org/limits.html/). In you case it is larger then 2000. It can be configured with the SQLITE_MAX_VARIABLE_NUMBER variable during compile time (https://stackoverflow.com/questions/15312590/what-is-the-limit-of-sql-variables-one-can-specify-in-a-single-execsql-query). So you have to compile your own SQLite.

For big projects it is recommended to use PostgreSQL database instead of SQLite.

We have docker images for the CodeChecker web server. So you can run a CodeChecker web server and a PostgreSQL server easily with docker. You can find more information and some examples in our user guide: https://github.com/Ericsson/codechecker/blob/master/docs/web/docker.md

[kenroser]

I now have two sqlite3 binaries. There's one in /usr/bin and another newer one (version 3.35) at /home/roser/sqlite/bin. This newer one has the parameter limit set to 32767. When I run store, it appears as if the sqlite3 in /usr/bin is always being used despite it not being the first sqlite3 in my PATH. How do I force CodeChecker to use my new version?

[csordasmarton]

@kenroser Try to set the LD_LIBRARY_PATH environment variable to the directory where your sqlite .so files can be found. In your case I think it will be /home/roser/sqlite/lib.

export LD_LIBRARY_PATH="/home/roser/sqlite/lib:$LD_LIBRARY_PATH"

[kenroser]

@csordasmarton Unfortunately that didn't fix it.

[kenroser]

python3 -c "import sqlite3; print(sqlite3.sqlite_version)" does show 3.35.4 when I set LD_LIBRARY_PATH and it shows 3.7.17 when I don't

[kenroser]

I modified my new version of sqlite3 to change the string too many SQL variables to TOO many SQL variables so I could tell where the error is coming from. The error I keep getting when I run CodeChecker store is too many SQL variables, indicating to me the old sqlite3 libraries are being used.

[csordasmarton]

@kenroser I tried it out and it works for me.

I downloaded the sqlite source code from here: https://sqlite.org/2021/sqlite-autoconf-3350400.tar.gz.

I compiled it on the following way: ./configure CFLAGS=-DSQLITE_MAX_VARIABLE_NUMBER=10 --prefix=$(pwd)/install

On my system the SQLITE_MAX_VARIABLE_NUMBER by default is set to 999 and it doesn't give me an error on the server side. For this reason to test whether my compiled version is used I set this variable at compile time to 10 to make sure it will throw an exception. In you case you have to set it to a higher value.

I added the lib directory to the LD_LIBRARY_PATH environment variable:
export LD_LIBRARY_PATH="/home/username/packages/sqlite-autoconf-3350400/install/lib/:$LD_LIBRARY_PATH"

I started a CodeChecker server in the same terminal where I exported this environment variable: CodeChecker server

It threw me the following exception: Failed to upgrade severity levels: (sqlite3.OperationalError) too many SQL variables. It means that my compiled sqlite version is used where the limit is set to 10.

[kenroser]

Thanks so much for your help.  I got it to work. The confusion was that my original trouble report indicated a failure when running "CodeChecker store".  I had been doing all my testing by setting LD_LIBRARY_PATH and then reinvoking "CodeChecker store".  Finally now with your example below, I noticed you were running the server.  I thought it was the "store" code that needed the new sqlite3 when in fact, it was the "server" that did. So I restarted my server with LD_LIBRARY_PATH set, then ran the "store" successfully.

…

On 4/8/21 3:23 AM, Márton Csordás wrote: @kenroser <https://github.com/kenroser> I tried it out and it works for me. I downloaded the sqlite source code from here: https://sqlite.org/2021/sqlite-autoconf-3350400.tar.gz <https://sqlite.org/2021/sqlite-autoconf-3350400.tar.gz>. I compiled it on the following way: |./configure CFLAGS=-DSQLITE_MAX_VARIABLE_NUMBER=10 --prefix=$(pwd)/install| On my system the SQLITE_MAX_VARIABLE_NUMBER by default is set to 999 and it doesn't give me an error on the server side. For this reason to test whether my compiled version is used I set this variable at compile time to 10 to make sure it will throw an exception. In you case you have to set it to a higher value. I added the lib directory to the |LD_LIBRARY_PATH| environment variable: |export LD_LIBRARY_PATH="/home/username/packages/sqlite-autoconf-3350400/install/lib/:$LD_LIBRARY_PATH"| I started a CodeChecker server in the same terminal where I exported this environment variable: |CodeChecker server| It threw me the following exception: |Failed to upgrade severity levels: (sqlite3.OperationalError) too many SQL variables|. It means that my compiled sqlite version is used where the limit is set to 10. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2826 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AELWOX2FTX3TAOUMSNF3463THVKXLANCNFSM42REUD2A>.