Skip to content

Commit

Permalink
Merge pull request #69 from kusumotolab/update_deps
Browse files Browse the repository at this point in the history
Update ANTLR
  • Loading branch information
T45K authored Jan 12, 2025
2 parents d43a91e + b0fc889 commit 967bb98
Show file tree
Hide file tree
Showing 75 changed files with 94,207 additions and 65,763 deletions.
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,8 @@ build
.idea

.DS_Store

# Execution results
clone_pairs
code_blocks
result_*.csv
56 changes: 32 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,47 +4,55 @@ FSE '21 paper implementation ([preprint](./camera-ready.pdf))<br>
DOI of submitted version executable file of NIL is `10.5281/zenodo.4492665`.

# NIL

NIL is a clone detector using N-gram, Inverted index, and LCS.
NIL provides scalable large-variance clone detection.

## Requirements

- JDK 21+

## Install & Usage

- Clone this repository (`git clone https://github.com/kusumotolab/NIL`)
- Move into NIL's directory (`cd NIL`) and build NIL (`./gradlew ShadowJar`)
- Run NIL (`java -jar ./build/libs/NIL-all.jar [options]`)
- Check the result file
- If you didn't specify `-bce` option, the format is `/path/to/file_A,start_line_A,end_line_A,/path/to/file_B,start_line_B,end_line_B`.
- If you specified `-bce` option, the format is `dir_A,file_A,start_line_A,end_line_A,dir_B,file_B,start_line_B,end_line_B`
- If you didn't specify `-bce` option, the format is
`/path/to/file_A,start_line_A,end_line_A,/path/to/file_B,start_line_B,end_line_B`.
- If you specified `-bce` option, the format is
`dir_A,file_A,start_line_A,end_line_A,dir_B,file_B,start_line_B,end_line_B`

## Options
|Name|Description|Default|
|:--:|:--|:--:|
|`-s`,`--src`|Input source directory. You must specify the target dir.|None|
|`-mil`,`--min-line`|Minimum number of lines that a code fragment must be to be treated as a clone.|`6`|
|`-mit`,`--min-token`|Minimum number of tokens that a code fragment must be to be treated as a clone.|`50`|
|`-n`,`--n-gram`|N for N-gram.|`5`|
|`-p`,`--partition-size`|The number of partitions.|`10`|
|`-f`,`--filtration-threshold`|Threshold used in the filtration phase (%).|`10`|
|`-v`,`--verification-threshold`|Threshold used in the verificatioin phase (%).|`70`|
|`-o`,`--output`|Output file name.|`result_{n}_{f}_{v}.csv`|
|`-t`,`--threads`|The number of threads used for parallel execution (both the *Preprocess* and *Clone detection* phases)|all threads|
|`-l`,`--language`|[Target language](#Languages)|`java`|
|`-bce`,`--bigcloneeval`|If you specify `-bce` option, NIL outputs result file feasible to BigCloneEval.|not specified|
|`-mif`,`--mutationinjectionframework`|If you specify `-mif` option, NIL outputs nothing except for the output file name as standard output.|not specified|

| Name | Description | Default |
|:-------------------------------------:|:-------------------------------------------------------------------------------------------------------|:------------------------:|
| `-s`,`--src` | Input source directory. You must specify the target dir. | None |
| `-mil`,`--min-line` | Minimum number of lines that a code fragment must be to be treated as a clone. | `6` |
| `-mit`,`--min-token` | Minimum number of tokens that a code fragment must be to be treated as a clone. | `50` |
| `-n`,`--n-gram` | N for N-gram. | `5` |
| `-p`,`--partition-size` | The number of partitions. | `10` |
| `-f`,`--filtration-threshold` | Threshold used in the filtration phase (%). | `10` |
| `-v`,`--verification-threshold` | Threshold used in the verificatioin phase (%). | `70` |
| `-o`,`--output` | Output file name. | `result_{n}_{f}_{v}.csv` |
| `-t`,`--threads` | The number of threads used for parallel execution (both the *Preprocess* and *Clone detection* phases) | all threads |
| `-l`,`--language` | [Target language](#Languages) | `java` |
| `-bce`,`--bigcloneeval` | If you specify `-bce` option, NIL outputs result file feasible to BigCloneEval. | not specified |
| `-mif`,`--mutationinjectionframework` | If you specify `-mif` option, NIL outputs nothing except for the output file name as standard output. | not specified |

## Languages
|Name|Option|Extension|
|:--:|:--:|:--:|
|Java|`java`|`.java`|
|C|`c`|`.c`,`.h`|
|C++|`cpp`|`.cpp`,`.hpp`|
|C#|`cs`,`csharp`|`.cs`|
|Python|`py`,`python`|`.py`|
|Kotlin|`kt`,`kotlin`|`.kt`|

| Name | Option | Extension |
|:------:|:-------------:|:-------------:|
| Java | `java` | `.java` |
| C | `c` | `.c`,`.h` |
| C++ | `cpp` | `.cpp`,`.hpp` |
| C# | `cs`,`csharp` | `.cs` |
| Python | `py`,`python` | `.py` |
| Kotlin | `kt`,`kotlin` | `.kt` |

If you execute NIL on 250-MLOC codebase, we recommend `-p` option to 135.

## Experiments, datasets, and baseline tools

- Please refer to [EXPERIMENTS.md](./EXPERIMENTS.md)
12 changes: 9 additions & 3 deletions build.gradle.kts
Original file line number Diff line number Diff line change
@@ -1,12 +1,16 @@
plugins {
kotlin("jvm") version "2.0.0"
id("com.github.johnrengelman.shadow") version "8.1.1"
id("antlr")

antlr

application
}

java.sourceCompatibility = JavaVersion.VERSION_21
java {
sourceCompatibility = JavaVersion.VERSION_21
targetCompatibility = JavaVersion.VERSION_21
}

repositories {
mavenCentral()
Expand All @@ -29,15 +33,17 @@ dependencies {
testImplementation("org.jetbrains.kotlin:kotlin-test-junit")

// Use ANTLR
antlr("org.antlr:antlr4:4.9")
antlr("org.antlr:antlr4:4.13.2")
}

application {
mainClass.set("jp.ac.osaka_u.sdl.nil.NILMainKt")
}

tasks.generateGrammarSource {
// Comment out when generating lexers and parsers from g4 file
enabled = false
outputDirectory = file("$projectDir/src/main/java")
}

tasks.generateTestGrammarSource {
Expand Down
2 changes: 1 addition & 1 deletion gradle/wrapper/gradle-wrapper.properties
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-8.8-bin.zip
distributionUrl=https\://services.gradle.org/distributions/gradle-8.12-bin.zip
networkTimeout=10000
validateDistributionUrl=true
zipStoreBase=GRADLE_USER_HOME
Expand Down
Loading

0 comments on commit 967bb98

Please sign in to comment.