Skip to content

Commit

Permalink
fix: truncate .err files to 1GiB
Browse files Browse the repository at this point in the history
  • Loading branch information
librarian committed Feb 3, 2025
1 parent 846e316 commit 6eeb171
Show file tree
Hide file tree
Showing 4 changed files with 20 additions and 4 deletions.
18 changes: 16 additions & 2 deletions .github/actions/test/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,10 @@ inputs:
required: false
default: 'no'
description: 'Use nebius'
truncate_enabled:
required: false
default: 'yes'
description: 'Truncate err files'

runs:
using: composite
Expand Down Expand Up @@ -283,8 +287,18 @@ runs:
find "$TESTS_DATA_DIR" -type f -print0 | xargs -0 -n 10 file -i | grep "application/x-executable" | awk -F: '{print $1}' | xargs rm
echo "::endgroup::"
echo "::group::remove-images-from-tests-data-dir"
find "$TESTS_DATA_DIR" -name generated_raw_image -o -name generated_vmdk_image -o -name invalid_qcow2_image -o -name qcow2_fuzzing_image
find "$TESTS_DATA_DIR" \( -name generated_raw_image -o -name generated_vmdk_image -o -name invalid_qcow2_image -o -name qcow2_fuzzing_image \) -delete
find "$TESTS_DATA_DIR" -name generated_raw_image -o -name generated_vmdk_image -o -name invalid_qcow2_image -o -name qcow2_fuzzing_image -name NVMENBS01
find "$TESTS_DATA_DIR" \( -name generated_raw_image -o -name generated_vmdk_image -o -name invalid_qcow2_image -o -name qcow2_fuzzing_image -name NVMENBS01 \) -delete
echo "::endgroup::"
echo "::group::truncate-err-files::"
find "$TESTS_DATA_DIR" -type f -name "*.err" -size +1G -print0 | while IFS= read -r -d '' file; do
orig_size=$(du -h "$file" | cut -f1)
echo "$file - $orig_size"
if [ "${{ inputs.truncate_enabled }}" == "yes" ]; then
truncate -s 1G "$file"
echo "... truncated (original size was $orig_size) ..." >> "$file"
fi
done
echo "::endgroup::"
echo "::group::s3-sync"
if [ "$SYNC_TO_S3" = "true" ];
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/build_and_test_ya.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,7 @@ jobs:
clean_ya_dir: ${{ inputs.run_build && 'no' || inputs.clean_ya_dir }}
use_network_cache: ${{ inputs.use_network_cache }}
nebius: ${{ inputs.nebius }}
truncate_enabled: ${{ contains(github.event.pull_request.labels.*.name, 'disable_truncate') && 'no' || 'yes' }}

- id: failure
name: set sleep_after_tests in case of failure
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/create_and_delete_vm.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -163,8 +163,8 @@ jobs:
runner_label: ${{ needs.provide-runner.outputs.label }}
runner_instance_id: ${{ needs.provide-runner.outputs.instance-id }}
runner_ipv4: ${{ needs.provide-runner.outputs.runner_ipv4 }}
build_target: ${{ needs.provide-runner.outputs.large_tests == 'yes' && 'cloud/blockstore/apps/,cloud/filestore/apps/,cloud/disk_manager/,cloud/tasks/' || 'cloud/tasks' }}
test_target: ${{ needs.provide-runner.outputs.large_tests == 'yes' && 'cloud/blockstore/,cloud/filestore/,cloud/disk_manager/,cloud/tasks/' || 'cloud/tasks' }}
build_target: 'cloud/filestore/apps/'
test_target: 'cloud/filestore/tests/fio_index/qemu-kikimr-nemesis-test/'
build_preset: "relwithdebinfo"
run_build: true
run_tests: true
Expand Down
1 change: 1 addition & 0 deletions GITHUB.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ There is also a list of labels that slightly alters how and which tests are run:
4. `asan` or `tsan` to add address sanitizer or thread sanitizer builds on top of the regular build.
5. `recheck` trigger checks without commit. Removed automatically after launch.
6. `allow-downgrade` allows to downgrade VM preset dynamically in case of problems with resources
7. `disable_truncate`, by default, .err files are truncated to 1GiB, this disables that for this PR

Also, you can launch [ya make](https://github.com/ydb-platform/nbs/actions/workflows/build_and_test_on_demand.yaml) or [cmake](https://github.com/ydb-platform/nbs/actions/workflows/build_and_test_on_demand_cmake.yaml) builds on your branch with any timeout you want (but please do not do more than 12 hours, VMs are expensive). You can find the IP of the VM inside of the jobs. The first occurrence happens in the `Prepare runner` job in the `Configure NCP` step, later IP is set in the header of jobs. You must use your GitHub key for it. User `github`. Feature not available for non-members.

Expand Down

0 comments on commit 6eeb171

Please sign in to comment.