tojsonl: Failed to infer field types

[datatraveller1]

First of all, thank you very much to jqnatividad for the great tool!

I want to convert CSV to JSON.
command:
qsv tojsonl file.csv -o file.json

e.g. file.csv with content:

col1,col2,col3
a,b,c
d,e,f

However, with this (and any other) file I get:
Failed to infer field types: qsv stats exited with code: 2
Am I doing something wrong?

Expected output file.json:

[
    {
        "col1": "a",
        "col2": "b",
        "col3": "c"
    },
    {
        "col1": "d",
        "col2": "e",
        "col3": "f"
    }
]

[jqnatividad]

First off, I'm glad you find it useful @datatraveller1 .

As to your problem, I'm afraid I cannot reproduce it...

$ qsv tojsonl file.csv -o file.jsonl
Enum list generated for field 'col1' (2 value/s)
Enum list generated for field 'col2' (2 value/s)
Enum list generated for field 'col3' (2 value/s)

$ echo file.jsonl
{"col1":"a","col2":"b","col3":"c"}
{"col1":"d","col2":"e","col3":"f"}

Also, the tojsonl qsv command produces JSONL, not JSON.

BTW, what version of qsv are you using? Can you run qsv --version?

[ondohotola]

can’t reproduce on

qsv 0.137.0-mimalloc-apply;fetch;foreach;geocode;Luau 0.640;to;polars-0.43.1-py-1.10.0;self_update-12-12;25.60 GiB-0 B-15.84 GiB-32.00 GiB (aarch64-apple-darwin compiled with Rust 1.82) prebuilt

Am running qsv -U now :-)-O

[datatraveller1]

"Can you run qsv --version?" -> I'm on MS Windows 11 and the result of this command is:
qsv 0.138.0-mimalloc-apply;fetch;foreach;geocode;Luau 0.650;prompt;to;polars-0.44.2-047e578;self_update-8-8;4.75 GiB-1.31 GiB-2.02 GiB-5.94 GiB (x86_64-pc-windows-msvc compiled with Rust 1.82) prebuilt

[datatraveller1]

I successfully use the JSON to CSV conversion which works great:
qsv json file.json -o file.csv

That's why I thought it would also be possible the other way round.

[datatraveller1]

Hi @jqnatividad,
I have made an interesting observation. I use the MS Windows command line (cmd.exe).
The command does work if there are no spaces in the directory name, e,g. C:\test\qsvtest

C:\test\qsvtest>type file.csv
col1,col2,col3
a,b,c
d,e,f
C:\test\qsvtest>qsv tojsonl file.csv -o file.jsonl
Enum list generated for field 'col1' (2 value/s)
Enum list generated for field 'col2' (2 value/s)
Enum list generated for field 'col3' (2 value/s)

C:\test\qsvtest>type file.jsonl
{"col1":"a","col2":"b","col3":"c"}
{"col1":"d","col2":"e","col3":"f"}

However, it does not work if there are spaces in the directory name, e.g. C:\test\qsv test

C:\test\qsv test>type file.csv
col1,col2,col3
a,b,c
d,e,f
C:\test\qsv test>qsv tojsonl file.csv -o file.jsonl
Failed to infer field types: qsv stats exited with code: 2

So this is an argument parsing issue, at least on MS Windows. Does it work on Linux?

[datatraveller1]

To be more precise, I open the command prompt (or Powershell), change to the directory "C:\test\qsv test" (with a space in the directory name) and call inside this directory the qsv command :

cd "c:\test\qsv test"
qsv tojsonl file.csv -o file.jsonl
Failed to infer field types: qsv stats exited with code: 2

It doesn't work either if the directory name does not contain spaces but the file name:

cd "c:\test\qsvtest"
qsv tojsonl "fil e.csv" -o file.jsonl
Failed to infer field types: qsv stats exited with code: 2

If it works correctly, e.g. for the file c:\test\qsvtest\file.csv, the generated file file.stats.csv.json shows:
"arg_input": "Some(\"\\\\\\\\?\\\\C:\\\\test\\\\qsvtest\\\\file.csv\")",
Maybe this part is wrong for a path name with spaces, but I can't see it because if it fails, the stats.csv.json file isn't generated.

[jqnatividad]

Thanks again @datatraveller1! This has been fixed in #2294

[datatraveller1]

Great thank you very much, @jqnatividad