Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve build-scripts #3168

Merged
merged 65 commits into from
Feb 6, 2025
Merged

Conversation

Blargian
Copy link
Member

@Blargian Blargian commented Jan 29, 2025

Summary

  • replaces prep-from-local, 'prep-from-master' and copyClickhouseRepoDocs.sh with a single script scripts/copy-clickhouse-repo-docs.sh which performs old prep-from-master by default or can perform prep-from-local if you provide it arg -l.
  • copy-clickhouse-repo-docs.sh additionally copies files needed for any kind of autogeneration (changelog and settings generation) to settings/tmp. New files can be added to package.json "autogen_needed_files" to extend.
  • copy-clickhouse-repo-docs.sh uses rsync, falls back to cp if it cannot find rsync
  • Moves logic for generating changelog to own script.

To do:

  • get rid of "new-build" and have just build
  • add linter for python and bash from the main repo

Checklist

@Blargian
Copy link
Member Author

Blargian commented Feb 1, 2025

Okay, i'm not crazy: https://stackoverflow.com/questions/68636358/file-not-found-in-alpine-container-despite-existing https://stackoverflow.com/questions/66963068/docker-alpine-executable-binary-not-found-even-if-in-path

Seems to be an underlying reason why no matter what I try I cannot run the clickhouse binary in the docs build (we just never picked this up because there was zero error handling in the original autogenerate-settings script, so it just always printed "Successfully generated settings" LOL

#75397

@Blargian
Copy link
Member Author

Blargian commented Feb 4, 2025

@gingerwizard ready for review on this. I think I know why we have "new-build".

here we pass arg "/output_path/build":

https://github.com/ClickHouse/ClickHouse/blob/0bfda2c21d37fdc82a9bff9ccadade7fe66e4d63/tests/ci/docs_check.py#L123

This is passed as an argument to yarn build which is just docusaurus build in package.json:

"build": "docusaurus build",

If I try change yarn build to new-build which calls build-swagger at the end of it then the docs build fails because /output_path/build is getting passed as an argument to build-swagger which then fails because it is not expecting this argument.

When we remove htmltest we'll be able to put everything in new-build into build. For now build needs to stay separate.

@Blargian
Copy link
Member Author

Blargian commented Feb 4, 2025

Need to fix broken links in settings first: ClickHouse/ClickHouse#75524

and URLs: ClickHouse/ClickHouse#75549

and broken hashes: ClickHouse/ClickHouse#75637

@Blargian Blargian merged commit bb038ac into ClickHouse:main Feb 6, 2025
3 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants