Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(ci): fixing the lint errors [on hold] #1066

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 10 additions & 8 deletions .github/workflows/compiler_benchmark.yml
Original file line number Diff line number Diff line change
Expand Up @@ -68,15 +68,17 @@ jobs:
- name: Export specific variables (GPU)
if: ${{ startswith(inputs.instance_type, 'p3.') }}
run: |
echo "CUDA_SUPPORT=ON" >> "${GITHUB_ENV}"
echo "BENCHMARK_TARGET=run-gpu-benchmarks" >> "${GITHUB_ENV}"
echo "CUDA_PATH=$CUDA_PATH" >> "${GITHUB_ENV}"
{ echo "CUDA_SUPPORT=ON"; \
echo "BENCHMARK_TARGET=run-gpu-benchmarks"; \
echo "CUDA_PATH=$CUDA_PATH"; \
echo "LD_LIBRARY_PATH=$CUDA_PATH/lib:$LD_LIBRARY_PATH"; \
echo "CC=/usr/bin/gcc-${{ env.GCC_VERSION }}"; \
echo "CXX=/usr/bin/g++-${{ env.GCC_VERSION }}"; \
echo "CUDAHOSTCXX=/usr/bin/g++-${{ env.GCC_VERSION }}"; \
echo "CUDACXX=$CUDA_PATH/bin/nvcc"; } >> "${GITHUB_ENV}"

# FIXME: expected to be in another file?
echo "$CUDA_PATH/bin" >> "${GITHUB_PATH}"
echo "LD_LIBRARY_PATH=$CUDA_PATH/lib:$LD_LIBRARY_PATH" >> "${GITHUB_ENV}"
echo "CC=/usr/bin/gcc-${{ env.GCC_VERSION }}" >> "${GITHUB_ENV}"
echo "CXX=/usr/bin/g++-${{ env.GCC_VERSION }}" >> "${GITHUB_ENV}"
echo "CUDAHOSTCXX=/usr/bin/g++-${{ env.GCC_VERSION }}" >> "${GITHUB_ENV}"
echo "CUDACXX=$CUDA_PATH/bin/nvcc" >> "${GITHUB_ENV}"

- name: Setup rust toolchain for concrete-cpu
uses: ./.github/workflows/setup_rust_toolchain_for_concrete_cpu
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/compiler_build_and_test_cpu.yml
Original file line number Diff line number Diff line change
Expand Up @@ -114,11 +114,11 @@ jobs:

- name: Enable complete tests on push to main
if: github.ref == 'refs/heads/main'
run: echo "MINIMAL_TESTS=OFF" >> $GITHUB_ENV
run: echo "MINIMAL_TESTS=OFF" >> "${GITHUB_ENV}"

- name: Enable minimal tests otherwise
if: github.ref != 'refs/heads/main'
run: echo "MINIMAL_TESTS=ON" >> $GITHUB_ENV
run: echo "MINIMAL_TESTS=ON" >> "${GITHUB_ENV}"

- name: Test compiler
uses: addnab/docker-run-action@4f65fabd2431ebc8d299f8e5a018d79a769ae185 # v3
Expand Down
8 changes: 8 additions & 0 deletions frontends/concrete-python/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,14 @@ check-links:
linkchecker ../../docs --check-extern \
--no-warnings

actionlint:
./scripts/actionlint/actionlint.sh

shelllint:
@# grep -v "^\./\." is to avoid files in .hidden_directories
find . -type f -name "*.sh" | grep -v "^\./\." | \
xargs shellcheck

pcc: check-format check-sanitize-notebooks mypy pydocstyle pylint ruff check-links

# ============
Expand Down
28 changes: 28 additions & 0 deletions frontends/concrete-python/scripts/actionlint/actionlint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
#!/usr/bin/env bash

set -e

LOG_FILE=$(mktemp /tmp/actionlint.script.XXXXXX)
SUMMARY_LOG_FILE=$(mktemp /tmp/actionlint.script.XXXXXX)

# Get actionlint errors
actionlint | cat > "$LOG_FILE"

# Get only where the errors are, not their type
grep -v .yml "$LOG_FILE" | grep -v ^" |" | cat > "$SUMMARY_LOG_FILE"

# Check errors which are not whitelisted
if python3 scripts/actionlint/actionlint_check_with_whitelists.py < "$SUMMARY_LOG_FILE";
then
echo "Successful end"
exit 0
else
echo "Full log file: "
cat "$LOG_FILE"

echo
echo "Summary log file:"
cat "$SUMMARY_LOG_FILE"
exit 255
fi

Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
""" Check an actionlint log against some whitelists """

import sys
from typing import Set

# Exact lines which are whitelisted
whitelisted_lines: Set[str] = set()

# Pattern which are whitelisted
whitelisted_pattern: Set[str] = {
}


def main():
"""Do the test

Raises:
ValueError: if non whitelisted error occurred
"""
status = 0
bad_lines = []
for line in sys.stdin:
if line in whitelisted_lines:
continue

is_bad_line = True

for pattern in whitelisted_pattern:
if pattern in line:
is_bad_line = False
break

if is_bad_line:
print("->", line)
status = 1
bad_lines.append(line)

if status:
errors = "\n------\n".join(bad_lines)
raise ValueError("Some non whitelisted errors, look at full log file:" f"{errors}")


if __name__ == "__main__":
main()
Loading