Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use shellcheck to check/lint shell scripts #579

Closed
nichtich opened this issue Jan 31, 2025 · 1 comment
Closed

Use shellcheck to check/lint shell scripts #579

nichtich opened this issue Jan 31, 2025 · 1 comment

Comments

@nichtich
Copy link
Collaborator

nichtich commented Jan 31, 2025

qa-catalogue includes 63 shell scripts. Several of these can be replaced by config files (#489).

wc -l $(find . -type f -exec head -1 {} /dev/null \;  | awk '/bash/ { print file } { file = $0 } ' | sed 's/[=<> ]//g' )

The rest should be checked with automatic code analysis, e.g. with shellcheck. It's these files with current lines of code:

   23 ./scala/build.sh
  229 ./index
    9 ./setdir.sh.template
   25 ./network-export.sh
  723 ./common-script
   17 ./postprocess-solr
   49 ./docker/qa-catalogue
  114 ./export-network.sh
  130 ./qa-catalogue
   41 ./prepare-solr
  203 ./solr-functions
   12 ./src/main/resources/pica/update-avram-k10plus-title.sh
   11 ./src/main/resources/pica/update-avram-k10plus-subjects.sh
   15 ./avram-schemas/validate-schemas
   56 ./scripts/sqlite/index-issue-details.sh
   63 ./scripts/sqlite/completeness.sqlite.sh
   77 ./scripts/sqlite/calculate-aggregated-numbers.grouped.sh
  148 ./scripts/timeline/timeline.sh
   61 ./scripts/scripts-for-2019-paper/run-all.sh
nichtich added a commit that referenced this issue Jan 31, 2025
nichtich added a commit that referenced this issue Feb 6, 2025
Down to 50 warnings (#579).

Shel script errors are detected via GitHub webhook.
@nichtich
Copy link
Collaborator Author

nichtich commented Feb 6, 2025

Warnings are down to 50, this should be ok. Errors are detected via GitHub action. We have 2000 lines of shell scripts, excluding the individual catalogue config files (which should be replaced by config files IMHO, see #489). The handling of PARAMS could also be changed to use config files instead (see #500).

$ for f in $(git ls-files); do [ -x "$f" ] && grep -q bash "$f" || continue; echo $f; done | grep -v catalogues/ | xargs wc -l | sort -n
   11 src/main/resources/pica/update-avram-k10plus-subjects.sh
   12 src/main/resources/pica/update-avram-k10plus-title.sh
   15 avram-schemas/validate-schemas
   17 postprocess-solr
   22 scala/build.sh
   25 network-export.sh
   40 prepare-solr
   49 docker/qa-catalogue
   55 scripts/sqlite/index-issue-details.sh
   61 scripts/scripts-for-2019-paper/run-all.sh
   63 scripts/sqlite/completeness.sqlite.sh
   76 scripts/sqlite/calculate-aggregated-numbers.grouped.sh
  114 export-network.sh
  130 qa-catalogue
  142 scripts/timeline/timeline.sh
  198 solr-functions
  230 index
  739 common-script
 1999 insgesamt

@nichtich nichtich closed this as completed Feb 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant