xlsx

[Konfekt]

  "custom_adapters": [
        {
            "name": "xlsx2csv",
            "version": 1,
            "description": "Uses adaption of xlsx2csv (the Python package) to convert XLSX from STDIN to CSV files",
            "extensions": ["xlsx"],
            "mimetypes": ["application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"],
            "binary": "xlsx2csv.sh",
            "args": [ "--all", "-"],
            "disabled_by_default": false,
            "match_only_by_mime": false
        }
  ]

where xlsx2csv.sh relies on either Dilshod's xlsx2csv or Spreadsheet-Read's xlsx2csv and reads

#!/usr/bin/env bash

# exit on error or use of undeclared variable or pipe error:
set -o errtrace -o errexit -o nounset -o pipefail
# optionally debug output by supplying TRACE=1
[[ "${TRACE:-0}" == "1" ]] && set -o xtrace

shopt -s inherit_errexit
IFS=$'\n\t'
PS4='+\t '

error_handler() { echo "Error: In ${BASH_SOURCE[0]} Line ${1} exited with Status ${2}"; }
trap 'error_handler ${LINENO} $?' ERR

# Get the last argument
for arg; do :; done
# Check if the last argument is a single hyphen
if [ "$arg" = "-" ]; then
  temp_file="$(mktemp "${TMPDIR:-/tmp}/tempXXXXXXXXXX.xlsx")"
  clean_up() {
    rm "$temp_file"
  }
  trap clean_up EXIT
  cat > "$temp_file"

  if [ $# -gt 1 ]; then
    # Store all arguments in an array
    args=("$@")
    # Remove the last element from the array
    unset "args[${#args[@]}-1]"
    xlsx2csv ${args[*]} "$temp_file"
  else
    xlsx2csv "$temp_file"
  fi
else
  xlsx2csv "$@"
fi

[Konfekt]

Once dilshod/xlsx2csv#264 is released, this wrapper script is no longer necessary for https://github.com/dilshod/xlsx2csv/

[Konfekt]

With today's release of xlsx2csv 0.82, the wrapper script is dispensable using https://github.com/dilshod/xlsx2csv/.

[AnabasisXu]

Hi Konfekt,

Could you kindly check why I could not copy your adapeter in Cygwin? Debug logs bellow:

Administrator in HOME\.config\rga
❯ xlsx2csv 0.xlsx
test,0
John,111

Administrator in HOME\.config\rga
❯ rga 11 0.xlsx

Administrator in HOME\.config\rga
❯ rga -V
ripgrep-all 0.10.6

Administrator in HOME\.config\rga
❯ rga 11 0.xlsx

Administrator in HOME\.config\rga
❯ xlsx2csv 0.xlsx
test,0

Administrator in HOME\.config\rga
❯ rga --debug --rga-config-file='D:\HOME\.config\rga\config.jsonc' "11" 0.xlsx | clip
[2024-11-02T05:08:46Z DEBUG ripgrep_all::config] rga (our) args: ["C:\\Users\\Administrator\\scoop\\apps\\rga\\current\\rga.exe", "--rga-co
nfig-file=D:\\HOME\\.config\\rga\\config.jsonc"]
[2024-11-02T05:08:46Z DEBUG ripgrep_all::config] Configs:
    D:\HOME\.config\rga\config.jsonc: {
      "custom_adapters": [
        {
          "args": [
            "--all",
            "-"
          ],
          "binary": "xlsx2csv.sh",
          "description": "Uses adaption of xlsx2csv (the Python package) to convert XLSX from STDIN to CSV files",
          "disabled_by_default": false,
          "extensions": [
            "xlsx"
          ],
          "match_only_by_mime": false,
          "mimetypes": [
            "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
          ],
          "name": "xlsx2csv",
          "version": 1
        }
      ]
    }
    RGA_CONFIG: {}
    Args: {}
    Merged: {
      "custom_adapters": [
        {
          "args": [
            "--all",
            "-"
          ],
          "binary": "xlsx2csv.sh",
          "description": "Uses adaption of xlsx2csv (the Python package) to convert XLSX from STDIN to CSV files",
          "disabled_by_default": false,
          "extensions": [
            "xlsx"
          ],
          "match_only_by_mime": false,
          "mimetypes": [
            "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
          ],
          "name": "xlsx2csv",
          "version": 1
        }
      ]
    }
[2024-11-02T05:08:46Z DEBUG ripgrep_all::config] rga (passthrough) args: ["--debug", "11", "0.xlsx"]
[2024-11-02T05:08:46Z DEBUG ripgrep_all::adapters] Chosen available adapters: xlsx2csv,pandoc,poppler,postprocpagebreaks,ffmpeg,zip,decompr
ess,tar,sqlite
[2024-11-02T05:08:46Z DEBUG rga] rg command to run: "rg" "--no-line-number" "--smart-case" "--pre" "C:\\Users\\Administrator\\scoop\\apps\\
rga\\current\\rga-preproc" "--pre-glob" "*.{xlsx,XLSX,epub,EPUB,odt,ODT,docx,DOCX,fb2,FB2,ipynb,IPYNB,html,HTML,htm,HTM,pdf,PDF,asciipagebr
eaks,ASCIIPAGEBREAKS,mkv,MKV,mp4,MP4,avi,AVI,mp3,MP3,ogg,OGG,flac,FLAC,webm,WEBM,zip,ZIP,jar,JAR,als,ALS,bz2,BZ2,gz,GZ,tbz,TBZ,tbz2,TBZ2,tg
z,TGZ,xz,XZ,zst,ZST,tar,TAR,db,DB,db3,DB3,sqlite,SQLITE,sqlite3,SQLITE3}" "--debug" "11" "0.xlsx"
rg: DEBUG|rg::flags::parse|crates/core\flags\parse.rs:97: no extra arguments found from configuration file
rg: DEBUG|rg::flags::hiargs|crates/core\flags\hiargs.rs:1083: number of paths given to search: 1
rg: DEBUG|rg::flags::hiargs|crates/core\flags\hiargs.rs:1094: is_one_file? true
rg: DEBUG|rg::flags::hiargs|crates/core\flags\hiargs.rs:1269: found hostname for hyperlink configuration: Anabasis
rg: DEBUG|rg::flags::hiargs|crates/core\flags\hiargs.rs:1279: hyperlink format: ""
rg: DEBUG|globset|crates\globset\src\lib.rs:448: glob converted to regex: Glob { glob: "**/*.{xlsx,XLSX,epub,EPUB,odt,ODT,docx,DOCX,fb2,FB2
,ipynb,IPYNB,html,HTML,htm,HTM,pdf,PDF,asciipagebreaks,ASCIIPAGEBREAKS,mkv,MKV,mp4,MP4,avi,AVI,mp3,MP3,ogg,OGG,flac,FLAC,webm,WEBM,zip,ZIP,
jar,JAR,als,ALS,bz2,BZ2,gz,GZ,tbz,TBZ,tbz2,TBZ2,tgz,TGZ,xz,XZ,zst,ZST,tar,TAR,db,DB,db3,DB3,sqlite,SQLITE,sqlite3,SQLITE3}", re: "(?-u)^(?:
/?|.*/)[^/]*\\.(?:SQLITE3|sqlite3|SQLITE|sqlite|DB3|db3|DB|db|TAR|tar|ZST|zst|XZ|xz|TGZ|tgz|TBZ2|tbz2|TBZ|tbz|GZ|gz|BZ2|bz2|ALS|als|JAR|jar
|ZIP|zip|WEBM|webm|FLAC|flac|OGG|ogg|MP3|mp3|AVI|avi|MP4|mp4|MKV|mkv|ASCIIPAGEBREAKS|asciipagebreaks|PDF|pdf|HTM|htm|HTML|html|IPYNB|ipynb|
FB2|fb2|DOCX|docx|ODT|odt|EPUB|epub|XLSX|xlsx)$", opts: GlobOptions { case_insensitive: false, literal_separator: true, backslash_escape: t
rue, empty_alternates: false }, tokens: Tokens([RecursivePrefix, ZeroOrMore, Literal('.'), Alternates([Tokens([Literal('S'), Literal('Q'),
Literal('L'), Literal('I'), Literal('T'), Literal('E'), Literal('3')]), Tokens([Literal('s'), Literal('q'), Literal('l'), Literal('i'), Lit
eral('t'), Literal('e'), Literal('3')]), Tokens([Literal('S'), Literal('Q'), Literal('L'), Literal('I'), Literal('T'), Literal('E')]), Toke
ns([Literal('s'), Literal('q'), Literal('l'), Literal('i'), Literal('t'), Literal('e')]), Tokens([Literal('D'), Literal('B'), Literal('3')]
), Tokens([Literal('d'), Literal('b'), Literal('3')]), Tokens([Literal('D'), Literal('B')]), Tokens([Literal('d'), Literal('b')]), Tokens([
Literal('T'), Literal('A'), Literal('R')]), Tokens([Literal('t'), Literal('a'), Literal('r')]), Tokens([Literal('Z'), Literal('S'), Literal
('T')]), Tokens([Literal('z'), Literal('s'), Literal('t')]), Tokens([Literal('X'), Literal('Z')]), Tokens([Literal('x'), Literal('z')]), To
kens([Literal('T'), Literal('G'), Literal('Z')]), Tokens([Literal('t'), Literal('g'), Literal('z')]), Tokens([Literal('T'), Literal('B'), L
iteral('Z'), Literal('2')]), Tokens([Literal('t'), Literal('b'), Literal('z'), Literal('2')]), Tokens([Literal('T'), Literal('B'), Literal(
'Z')]), Tokens([Literal('t'), Literal('b'), Literal('z')]), Tokens([Literal('G'), Literal('Z')]), Tokens([Literal('g'), Literal('z')]), Tok
ens([Literal('B'), Literal('Z'), Literal('2')]), Tokens([Literal('b'), Literal('z'), Literal('2')]), Tokens([Literal('A'), Literal('L'), Li
teral('S')]), Tokens([Literal('a'), Literal('l'), Literal('s')]), Tokens([Literal('J'), Literal('A'), Literal('R')]), Tokens([Literal('j'),
 Literal('a'), Literal('r')]), Tokens([Literal('Z'), Literal('I'), Literal('P')]), Tokens([Literal('z'), Literal('i'), Literal('p')]), Toke
ns([Literal('W'), Literal('E'), Literal('B'), Literal('M')]), Tokens([Literal('w'), Literal('e'), Literal('b'), Literal('m')]), Tokens([Lit
eral('F'), Literal('L'), Literal('A'), Literal('C')]), Tokens([Literal('f'), Literal('l'), Literal('a'), Literal('c')]), Tokens([Literal('O
'), Literal('G'), Literal('G')]), Tokens([Literal('o'), Literal('g'), Literal('g')]), Tokens([Literal('M'), Literal('P'), Literal('3')]), T
okens([Literal('m'), Literal('p'), Literal('3')]), Tokens([Literal('A'), Literal('V'), Literal('I')]), Tokens([Literal('a'), Literal('v'),
Literal('i')]), Tokens([Literal('M'), Literal('P'), Literal('4')]), Tokens([Literal('m'), Literal('p'), Literal('4')]), Tokens([Literal('M'
), Literal('K'), Literal('V')]), Tokens([Literal('m'), Literal('k'), Literal('v')]), Tokens([Literal('A'), Literal('S'), Literal('C'), Lite
ral('I'), Literal('I'), Literal('P'), Literal('A'), Literal('G'), Literal('E'), Literal('B'), Literal('R'), Literal('E'), Literal('A'), Lit
eral('K'), Literal('S')]), Tokens([Literal('a'), Literal('s'), Literal('c'), Literal('i'), Literal('i'), Literal('p'), Literal('a'), Litera
l('g'), Literal('e'), Literal('b'), Literal('r'), Literal('e'), Literal('a'), Literal('k'), Literal('s')]), Tokens([Literal('P'), Literal('
D'), Literal('F')]), Tokens([Literal('p'), Literal('d'), Literal('f')]), Tokens([Literal('H'), Literal('T'), Literal('M')]), Tokens([Litera
l('h'), Literal('t'), Literal('m')]), Tokens([Literal('H'), Literal('T'), Literal('M'), Literal('L')]), Tokens([Literal('h'), Literal('t'),
 Literal('m'), Literal('l')]), Tokens([Literal('I'), Literal('P'), Literal('Y'), Literal('N'), Literal('B')]), Tokens([Literal('i'), Litera
l('p'), Literal('y'), Literal('n'), Literal('b')]), Tokens([Literal('F'), Literal('B'), Literal('2')]), Tokens([Literal('f'), Literal('b'),
 Literal('2')]), Tokens([Literal('D'), Literal('O'), Literal('C'), Literal('X')]), Tokens([Literal('d'), Literal('o'), Literal('c'), Litera
l('x')]), Tokens([Literal('O'), Literal('D'), Literal('T')]), Tokens([Literal('o'), Literal('d'), Literal('t')]), Tokens([Literal('E'), Lit
eral('P'), Literal('U'), Literal('B')]), Tokens([Literal('e'), Literal('p'), Literal('u'), Literal('b')]), Tokens([Literal('X'), Literal('L
'), Literal('S'), Literal('X')]), Tokens([Literal('x'), Literal('l'), Literal('s'), Literal('x')])])]) }
rg: DEBUG|globset|crates\globset\src\lib.rs:453: built glob set; 0 literals, 0 basenames, 0 extensions, 0 prefixes, 0 suffixes, 0 required
extensions, 1 regexes
rg: DEBUG|rg::flags::hiargs|crates/core\flags\hiargs.rs:174: using 1 thread(s)
rg: DEBUG|globset|crates\globset\src\lib.rs:453: built glob set; 0 literals, 0 basenames, 12 extensions, 0 prefixes, 0 suffixes, 0 required
 extensions, 0 regexes
[2024-11-02T05:08:47Z DEBUG rga] running rg took 0.243s

[AnabasisXu]

In fact, I am trying to implement my own adapter which allow searching for .mdx files using https://github.com/liuyug/mdict-utils. Basically, I will use mdict -x 1.mdx to convert 1.mdx to 1.mdx.txt (annoyingly it does not allow customizing the output). But I just could not get the adapter to work. So I tried your adapter, which unfortunately did not work out either...

[Konfekt]

I only know MSYS2 a bit, but Cygwin not so much.

This log shows that one file was found, but not that Rga even tried to invoke the adapter.
How does this output compare to those for adapters that work fine?
I suspect that Rga does not work as expected here.

Other than that, with recent the (Python) Xlsx2csv, the shell script Xlsx2csv.sh is no longer needed.

However, for Mdict, you might want to write a similar bash script piping the converted output, similar maybe to Pptxmd here

[AnabasisXu]

Hi @Konfekt , Thanks for your kind reply.

How does this output compare to those for adapters that work fine?
not sure where to look for the difference. I attach both normal and error log.
error.log
normal.log

Other than that, with recent the (Python) Xlsx2csv, the shell script Xlsx2csv.sh is no longer needed.
rga cannot search .xlsx files directly, even with Xlsx2csv installed:

❯ rga 11 0.xlsx

for Mdict, you might want to write a similar bash script piping the converted output

Do you mean this line?
application/vnd.ms-powerpoint; ppthtml %s | w3m -dump -T text/html; copiousoutput

[Konfekt]

Hello,

I meant that one

pptx2md --disable-image --disable-wmf --disable-color --disable-escaping %s --output $HOME/.cache/mutt/presentation.md 2>/dev/null && cat --squeeze-blank $HOME/.cache/mutt/presentation.md

so that the shell script would do something like mdict --db-txt dict.db && cat dict.txt

The normal log is too normal, as it doesn't show any conversion of any file (only a CSV file was given).
So something using a built-in (PDF?) adapter would be more helpful for comparison.
I fear that Rga's debug is not very helpful in any case and maybe an issue would be more suitable, but let's see