Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: ✨ Multi-language support #32

Merged
merged 70 commits into from
Apr 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
ffe1e86
Refactoring towards multi-language support
robvanderleek Mar 3, 2024
a68ffa7
Move languages to init
robvanderleek Mar 5, 2024
aecbe09
Add live table and version command
robvanderleek Mar 7, 2024
75805f5
Add version to report
robvanderleek Mar 7, 2024
11b86fc
Introducing scan result table
robvanderleek Mar 8, 2024
f0fc3bf
Run black in GHA
robvanderleek Mar 8, 2024
0794168
Run black with GHA
robvanderleek Mar 8, 2024
e18beb0
Add Java support
robvanderleek Mar 9, 2024
0a9df12
Move Ruff to GHA
robvanderleek Mar 9, 2024
ef15f4e
Add report command
robvanderleek Mar 10, 2024
c72f8a5
Remove textual
robvanderleek Mar 10, 2024
2f8585c
Fix ruff error
robvanderleek Mar 10, 2024
5c47edb
Fix workflow
robvanderleek Mar 10, 2024
ad1063e
Fix workflow
robvanderleek Mar 10, 2024
4ede70e
Code style fixes
robvanderleek Mar 10, 2024
f380ba2
Read ignores from top-level .gitignore file
robvanderleek Mar 11, 2024
0ed915b
Code style fixes
robvanderleek Mar 11, 2024
b60fc63
Add verbose flag
robvanderleek Mar 11, 2024
c74d28f
Code style fixes
robvanderleek Mar 11, 2024
c6de546
Refactoring tests
robvanderleek Mar 12, 2024
5e451f5
Code style fixes
robvanderleek Mar 12, 2024
a2444ee
Refactoring tests...
robvanderleek Mar 14, 2024
ad2961a
Done refactoring test cases
robvanderleek Mar 15, 2024
88134e2
Code style fixes
robvanderleek Mar 15, 2024
ea4aea4
Working on C++ support
robvanderleek Mar 16, 2024
7cda9a1
Code style fixes
robvanderleek Mar 16, 2024
301b3fe
Basic C++ support
robvanderleek Mar 17, 2024
a9b9442
Fix test
robvanderleek Mar 17, 2024
c2baa55
Code style fixes
robvanderleek Mar 17, 2024
0247f3c
Scan result header and footer
robvanderleek Mar 19, 2024
36b20f3
Code style fixes
robvanderleek Mar 19, 2024
611516a
Fix tests
robvanderleek Mar 19, 2024
7b6608b
Improve report output
robvanderleek Mar 20, 2024
5500f98
Code style fixes
robvanderleek Mar 20, 2024
1e9df48
Get version from file
robvanderleek Mar 20, 2024
ec02470
Code style fixes
robvanderleek Mar 20, 2024
3023d17
Add regression test workflow
robvanderleek Mar 21, 2024
93e1a7f
Naming
robvanderleek Mar 21, 2024
909733b
Move MyPy to GHA
robvanderleek Mar 21, 2024
1aa04b9
Fix MyPy errors
robvanderleek Mar 21, 2024
5dc34bf
Fixed nested scopes for JavaScript
robvanderleek Mar 23, 2024
2c3ca6f
Fix MyPy
robvanderleek Mar 23, 2024
b6aa239
Code style fixes
robvanderleek Mar 23, 2024
dff0f10
Fixed nested scopes for TypeScript
robvanderleek Mar 23, 2024
0a17c3a
Merge branch 'issue-31-Multi-language_support' of github.com:getcodel…
robvanderleek Mar 23, 2024
c453d7a
Fix MyPy
robvanderleek Mar 23, 2024
f46defd
Code style fixes
robvanderleek Mar 23, 2024
3564117
Tweaked output a bit
robvanderleek Mar 23, 2024
66201c4
Code style fixes
robvanderleek Mar 23, 2024
19f339a
Updating documentation
robvanderleek Mar 24, 2024
c3bdf6c
Code style fixes
robvanderleek Mar 24, 2024
f141f19
Fix build
robvanderleek Mar 24, 2024
04dddff
Sort scan results
robvanderleek Mar 24, 2024
0919d54
Add failing tests
robvanderleek Mar 27, 2024
cb3534c
Hello GSM :wave:
robvanderleek Mar 27, 2024
99b6a3a
Hello GSM :wave:
robvanderleek Mar 27, 2024
9e698b8
Merge branch 'issue-31-Multi-language_support' of github.com:getcodel…
robvanderleek Mar 27, 2024
ff80c98
Going from NFA to DFA...
robvanderleek Mar 29, 2024
b7bca46
Refactoring
robvanderleek Mar 29, 2024
ecd0d5f
Completed NFA to DFA conversion
robvanderleek Mar 30, 2024
d09b85f
First implementation of find_all()
robvanderleek Mar 31, 2024
0941c12
Finalizing GSM...
robvanderleek Apr 2, 2024
401241c
Fix ruff
robvanderleek Apr 2, 2024
e658484
Code style fixes
robvanderleek Apr 2, 2024
e6acfc9
Migrating from TokenMatcher to GSM
robvanderleek Apr 3, 2024
3093854
Fix NFA to DFA conversion
robvanderleek Apr 4, 2024
077a938
Replace TokenMatcher by GSM
robvanderleek Apr 6, 2024
79452bf
Updated documentation
robvanderleek Apr 6, 2024
a8a4b03
Fix ruff issue
robvanderleek Apr 6, 2024
8bf23c9
Code style fixes
robvanderleek Apr 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions .github/workflows/analyze.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: 'analyze'
on:
push:
branches:
- main
pull_request:
branches:
- main
jobs:
analyze:
runs-on: self-hosted
steps:
- name: 'Checkout sources'
uses: actions/checkout@v2
- name: 'Set up Python'
uses: actions/setup-python@v2
with:
python-version: 3.12
- name: 'Set up Poetry'
uses: snok/install-poetry@v1
- name: 'Install dependencies'
run: poetry install --no-interaction --no-root
- name: 'Checkout pl-snakebattle'
run: git clone https://github.com/getcodelimit/pl-battlesnake.git
- name: 'Run codelimit'
run: poetry run codelimit scan pl-battlesnake
28 changes: 28 additions & 0 deletions .github/workflows/code-style.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
name: 'code-style'

on:
pull_request:
branches:
- main

jobs:
code_style:
runs-on: ubuntu-latest
steps:
- name: 'Checkout sources'
uses: actions/checkout@v2
- name: 'Lint code with Ruff'
run: |
pip install ruff
ruff check --fix .
- name: 'Type cheking with MyPy'
run: |
pip install mypy
mypy --ignore-missing-imports codelimit/
- name: 'Format code with black'
run: |
pip install black
black .
- uses: stefanzweifel/git-auto-commit-action@v4
with:
commit_message: 'Code style fixes'
2 changes: 1 addition & 1 deletion .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ jobs:
ci:
runs-on: ubuntu-latest
steps:
- name: 'Checkout'
- name: 'Checkout sources'
uses: actions/checkout@v2
- name: 'Set up Python'
uses: actions/setup-python@v2
Expand Down
17 changes: 0 additions & 17 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,23 +2,6 @@ default_language_version:
python: python3.11

repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
hooks:
- id: trailing-whitespace
- repo: https://github.com/psf/black
rev: 23.7.0
hooks:
- id: black
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.0.283
hooks:
- id: ruff
args: [ --fix, --exit-non-zero-on-fix ]
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.4.1
hooks:
- id: mypy
- repo: https://github.com/getcodelimit/codelimit
rev: v0.7.0
hooks:
Expand Down
628 changes: 616 additions & 12 deletions LICENSE

Large diffs are not rendered by default.

149 changes: 46 additions & 103 deletions codelimit/__main__.py
Original file line number Diff line number Diff line change
@@ -1,55 +1,36 @@
import os
import sys
from pathlib import Path
from typing import List, Annotated, Optional

import typer
from click import Context
from rich import print
from typer.core import TyperGroup

from codelimit.commands import github
from codelimit.common.CheckResult import CheckResult
from codelimit.commands import app
from codelimit.commands.check import check_command
from codelimit.commands.report import report_command
from codelimit.commands.scan import scan_command
from codelimit.common.Configuration import Configuration
from codelimit.common.Scanner import scan_codebase, is_excluded
from codelimit.common.report.Report import Report
from codelimit.common.report.ReportReader import ReportReader
from codelimit.common.report.ReportWriter import ReportWriter
from codelimit.github_auth import get_github_token
from codelimit.tui.CodeLimitApp import CodeLimitApp
from codelimit.utils import upload_report, check_file, read_cached_report
from codelimit.version import version

cli = typer.Typer(no_args_is_help=True, add_completion=False)
cli.add_typer(github.app, name="github", help="GitHub commands")

class OrderCommands(TyperGroup):
def list_commands(self, ctx: Context):
return list(self.commands)


cli = typer.Typer(cls=OrderCommands, no_args_is_help=True, add_completion=False)
cli.add_typer(app.app, name="app", help="Code Limit GitHub App commands")


@cli.command(help="Check file(s)")
def check(
paths: Annotated[List[Path], typer.Argument(exists=True)],
quiet: Annotated[
bool, typer.Option("--quiet", help="Not output when successful")
bool, typer.Option("--quiet", help="No output when successful")
] = False,
):
check_result = CheckResult()
for path in paths:
if path.is_file():
check_file(path, check_result)
elif path.is_dir():
for root, dirs, files in os.walk(path.absolute()):
files = [f for f in files if not f[0] == "."]
dirs[:] = [d for d in dirs if not d[0] == "."]
for file in files:
abs_path = Path(os.path.join(root, file))
rel_path = abs_path.relative_to(path.absolute())
if is_excluded(rel_path):
continue
check_file(abs_path, check_result)
exit_code = 1 if check_result.unmaintainable > 0 else 0
if (
not quiet
or check_result.hard_to_maintain > 0
or check_result.unmaintainable > 0
):
check_result.report()
raise typer.Exit(code=exit_code)
check_command(paths, quiet)


@cli.command(help="Scan a codebase")
Expand All @@ -58,83 +39,45 @@ def scan(
Path, typer.Argument(exists=True, file_okay=False, help="Codebase root")
]
):
cache_dir = path.joinpath(".codelimit_cache").resolve()
report_path = cache_dir.joinpath("codelimit.json").resolve()
if report_path.exists():
cached_report = ReportReader.from_json(report_path.read_text())
else:
cached_report = None
codebase = scan_codebase(path, cached_report)
codebase.aggregate()
report = Report(codebase)
if not cache_dir.exists():
cache_dir.mkdir()
cache_dir_tag = cache_dir.joinpath("CACHEDIR.TAG").resolve()
cache_dir_tag.write_text("Signature: 8a477f597d28d172789f06886806bc55")
cache_dir_gitignore = cache_dir.joinpath(".gitignore").resolve()
cache_dir_gitignore.write_text("# Created by codelimit automatically.\n*\n")
report_path.write_text(ReportWriter(report).to_json())
if sys.stdout.isatty():
app = CodeLimitApp(report)
app.run()


@cli.command(help="Upload report to Code Limit")
def upload(
repository: Annotated[
str,
typer.Argument(
envvar="GITHUB_REPOSITORY", show_default=False, help="GitHub repository"
),
],
branch: Annotated[
str,
typer.Argument(envvar="GITHUB_REF", show_default=False, help="GitHub branch"),
scan_command(path)


@cli.command(help="Show report for codebase")
def report(
path: Annotated[
Path, typer.Argument(exists=True, file_okay=False, help="Codebase root")
],
report_file: Path = typer.Option(
None,
"--report",
exists=True,
dir_okay=False,
file_okay=True,
help="JSON report file",
),
token: str = typer.Option(None, "--token", help="GitHub access token"),
url: str = typer.Option(
"https://codelimit.vercel.app/api/upload",
"--url",
help="Upload JSON report to this URL.",
),
full: Annotated[bool, typer.Option("--full", help="Show full report")] = False,
):
if report_file:
report = ReportReader.from_json(report_file.read_text())
else:
cached_report = read_cached_report(Path("."))
if not cached_report:
print("[red]No cached report found in current folder[/red]")
raise typer.Exit(code=1)
else:
report = cached_report
if not token:
token = get_github_token()
if not token:
print("[red]Invalid or no credentials, please login or supply token[/red]")
raise typer.Exit(code=1)
try:
upload_report(report, repository, branch, url, token)
raise typer.Exit(code=0)
except FileNotFoundError as error:
typer.secho(f"File not found: {error}", fg="red")
raise typer.Exit(code=1) from None
report_command(path, full)


def _version_callback(show: bool):
if show:
print(f"Code Limit version: {version}")
raise typer.Exit()


@cli.callback()
def main(
verbose: Annotated[
Optional[bool], typer.Option("--verbose", "-v", help="Verbose output")
] = False,
exclude: Annotated[
Optional[list[str]], typer.Option(help="Glob patterns for exclusion")
] = None
] = None,
version: Annotated[
Optional[bool],
typer.Option(
"--version", "-V", help="Show version", callback=_version_callback
),
] = None,
):
"""CodeLimit: Your refactoring alarm."""
if verbose:
Configuration.verbose = True
if version:
raise typer.Exit()
if exclude:
Configuration.excludes.extend(exclude)

Expand Down
56 changes: 56 additions & 0 deletions codelimit/commands/app.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
from pathlib import Path
from typing import Annotated

import typer
from rich import print

from codelimit.commands.upload import upload_command
from codelimit.github_auth import device_flow_logout, device_flow_login

app = typer.Typer(no_args_is_help=True)


@app.command(help="Login to GitHub App")
def login():
if device_flow_login():
print("[green]Logged in successfully[/green]")
typer.Exit(code=0)
else:
print("[red]Login failed[/red]")
typer.Exit(code=1)


@app.command(help="Logout from GitHub App")
def logout():
device_flow_logout()
print("Logged out")


@app.command(help="Upload report to Code Limit GitHub App")
def upload(
repository: Annotated[
str,
typer.Argument(
envvar="GITHUB_REPOSITORY", show_default=False, help="GitHub repository"
),
],
branch: Annotated[
str,
typer.Argument(envvar="GITHUB_REF", show_default=False, help="GitHub branch"),
],
report_file: Path = typer.Option(
None,
"--report",
exists=True,
dir_okay=False,
file_okay=True,
help="JSON report file",
),
token: str = typer.Option(None, "--token", help="GitHub access token"),
url: str = typer.Option(
"https://codelimit.vercel.app/api/upload",
"--url",
help="Upload JSON report to this URL.",
),
):
upload_command(repository, branch, report_file, token, url)
54 changes: 54 additions & 0 deletions codelimit/commands/check.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
import os
from pathlib import Path

import typer
from pygments.lexers import get_lexer_for_filename

from codelimit.common.CheckResult import CheckResult
from codelimit.common.Scanner import is_excluded, scan_file
from codelimit.common.lexer_utils import lex
from codelimit.common.utils import load_language_by_name
from codelimit.languages import LanguageName


def check_command(paths: list[Path], quiet: bool):
check_result = CheckResult()
for path in paths:
if path.is_file():
check_file(path, check_result)
elif path.is_dir():
for root, dirs, files in os.walk(path.absolute()):
files = [f for f in files if not f[0] == "."]
dirs[:] = [d for d in dirs if not d[0] == "."]
for file in files:
abs_path = Path(os.path.join(root, file))
rel_path = abs_path.relative_to(path.absolute())
if is_excluded(rel_path):
continue
check_file(abs_path, check_result)
exit_code = 1 if check_result.unmaintainable > 0 else 0
if (
not quiet
or check_result.hard_to_maintain > 0
or check_result.unmaintainable > 0
):
check_result.report()
raise typer.Exit(code=exit_code)


def check_file(path: Path, check_result: CheckResult):
lexer = get_lexer_for_filename(path)
language = lexer.__class__.name
if language in LanguageName:
with open(path) as f:
code = f.read()
tokens = lex(lexer, code, False)
language = load_language_by_name(lexer.__class__.name)
if language:
measurements = scan_file(tokens, language)
risks = sorted(
[m for m in measurements if m.value > 30],
key=lambda measurement: measurement.value,
reverse=True,
)
check_result.add(path, risks)
Loading