Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve & simplify valgrind testing #278

Open
wants to merge 10 commits into
base: master
Choose a base branch
from
Open

Conversation

onurctirtir
Copy link
Member

@onurctirtir onurctirtir commented Jan 21, 2025

  • Reduce the dependency on the other constructs of this repo, e.g., creation & deallocation of Azure VMs via the existing tooling doesn't seem so much stable. So, introduce a simple docker image that can allow us running multiple valgrind tests in a single vm in parallel and anywhere. Optionally, create_cluster.sh and its friends can still be used to create an Azure VM but they are not a must now. See the changes in README.md for more details.
  • Get rid of valgrind code from fab infra because using this infra for valgrind testing was both unnecessary and this was heavily complicating the things. Instead, a simpler infra that we can also use in our local machines seems much more useful and easier to maintain and understand.
  • Also remove fabfile/pg_report_vg_query.patch since the Postgres patch that we depend on to report the query string that causes the memory error has been merged into Postgres as of PG16.
  • Automatically collect stack traces from valgrind core files that are generated in case of a process crash.

Note to reviewer - although the PR touches nine files, this PR mainly introduces following three files and updates the README on the usage of valgrind testing infra. The other 5 files are only modified to remove the old infra.

  • valgrind/Docker/Dockerfile
    The dockerfile to build a simple docker image that can be used to run valgrind tests.
    This has nothing but Postgres & Citus built with the flags that are needed to properly run valgrind tests and valgrind itself.
  • valgrind/Docker/run_valgrind_test.sh
    The script that we run within the container to run the valgrind test for the specified test schedule.
    We could set the entrypoint of the container to this script but we keep it as a separate script for simplicity.
  • run.sh
    The host side script that we use to build the container and run the valgrind tests through the container.

I strongly recommend reading the relevant section of the README from this PR to better understand the changes.

* Get rid of valgrind code from fab infra because using this infra
  for valgrind testing was both unnecessary and this was heavily
  complicatings the things. Instead, a simpler infra that we can
  also use in our local machines seems much more usuful and easier
  to maintain & understand.

* Also remove fabfile/pg_report_vg_query.patch since the Postgres
  patch that we depend on to report the query string that causes
  the memory error has been merged into Postgres as of PG16.
@onurctirtir onurctirtir changed the title Fix & modernize valgrind testing Fix & simplify valgrind testing Jan 21, 2025
@onurctirtir onurctirtir requested review from hanefi and naisila January 21, 2025 16:34
@onurctirtir onurctirtir changed the title Fix & simplify valgrind testing Improve & simplify valgrind testing Jan 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant