Skip to content

Commit

Permalink
update readme.md
Browse files Browse the repository at this point in the history
  • Loading branch information
xmlyqing00 committed Jan 8, 2020
1 parent 2afd7f3 commit e77e969
Show file tree
Hide file tree
Showing 6 changed files with 230 additions and 39 deletions.
118 changes: 108 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
# StripeReassembly

This repository is a C++ implementation for
```
Reassembling Shredded Document Stripes Using Word-path Metric and Greedy Composition Optimal Matching Solver
```
This repository is a C++ implementation for TMM 19 paper
> Liang, Yongqing, and Xin Li. "Reassembling Shredded Document Stripes Using Word-path Metric and Greedy Composition Optimal Matching Solver." IEEE Transactions on Multimedia (2019).
If you use these codes in your research, please cite the paper.

## Environment
**Paper correction:** The Equation (5) should be
![](eq_correct.png)

## 1. Environment

We build and evaluate our codes under Ubuntu 18.04 and Mac OS X 10.14.5. The following packages are used in this repository:
1. OpenCV: 3.2.0
Expand All @@ -15,7 +17,7 @@ We build and evaluate our codes under Ubuntu 18.04 and Mac OS X 10.14.5. The fol
4. g++: 7.4.0
5. Python: 3.6.8

## DocDataset description
## 2. DocDataset description

Click [here](http://t.lyq.me?d=DocDataset) to download the `DocDataset`. Unzip the package and copy the `gt` and `stripes` into the `/data/` folder of the repository.

Expand All @@ -24,10 +26,106 @@ Click [here](http://t.lyq.me?d=DocDataset) to download the `DocDataset`. Unzip t
2. 3 physically shredded document puzzles. They are named as `real*_*`.
3. 1 randomly oriented puzzle named `doc3_36`.

The comparison performance results are reported in the paper.
The comparison performance results are reported in the paper Table I, Table II, and Table III.

## 3. Usage

Download this repository, the source code can be compiled into `debug` and `release` executable files.

### 3.1 Compile

To generate the executable file in the `debug` mode:
```
./autogen debug
```
To generate the executable file in the `release` mode:
```
./autogen release
```

### 3.2 Reassemble a stripe puzzle

## Usage
A quick example to reassemble the synthesized stripe puzzle
```
./bin/release/solver --text doc0 --num 40 --comp 2 --metric 2 --samples 300
```
Another example to reassemble the real-word stripe puzzle:
```
./bin/release/solver -t real1 -n 27 -c 2 -m 2 -s 10000 -r --word_conf_thres 70 --lambda0 0.5 --lambda1 0.7 --u_a 1 --filter_rate 0.2 --candidate_factor 5
```

### Compile
The detailed document can be found in
```
./bin/release/solver --help
```

### 3.3 Run benchmark

`CMakeLists.txt`
We also provide an option to run the whole dataset instead of running each test case individually.
```
./benchmark.sh doc [gen]
```

When you run the benchmark or add `--benchmark` option to `./bin/release/solver`, the results are saved in `data/scores`.


#### 3.3.1 Generate stripe puzzles
In most cases, we recommend you to use the provided dataset for fair comparison.

The `gen` provides you an alternative option when running the benchmark. It which will run the `./bin/release/generator` to generate the stripe puzzles from the groundtruth.

Details about randomly generating stripes from groundtruths can be found in `src/generator/generate_puzzle.cpp`.

#### 3.3.2 Recommend parameters
For synthetic data, default parameters are good enough.
```
const double word_conf_thres = 70;
const double lambda0 = 0.3;
const double lambda1 = 0.5;
const double U_a = 2;
const double filter_rate = 0.7;
const int candidate_factor = 4;
```
We recommend the `--samples` at least 150, 300, 1000, 8000 for 20-, 30-, 40-, and 60-stripe puzzles.

For real-word data, `real1`, `real2`, and `real3`. We report our results in the following parameters.

```
const double word_conf_thres {70}; // or 60
const double lambda0 = 0.5;
const double lambda1 = 0.7;
const double U_a = 1;
// For Real Case 1
const double filter_rate = 0.2;
const int candidate_factor {5};
// For Real Case 2
const double filter_rate = 0.5;
const int candidate_factor {3};
// For Real Case 3
const double filter_rate = 0.6;
const int candidate_factor {5};
```
We recommend the `--samples` at least larger than 8000.

### 3.4 Clean
```
./autoclean debug
```
or
```
./autoclean release
```

## 4. Reference
```
@article{liang2019reassembling,
title={Reassembling Shredded Document Stripes Using Word-path Metric and Greedy Composition Optimal Matching Solver},
author={Liang, Yongqing and Li, Xin},
journal={IEEE Transactions on Multimedia},
year={2019},
publisher={IEEE}
}
```
Binary file added eq_correct.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions include/solve_puzzle.h
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
#define SOLVE_PUZZLE

#include <unistd.h>
#include <getopt.h>
#include <string>
#include <fstream>
#include <ctime>
Expand Down
6 changes: 3 additions & 3 deletions include/stripes_solver.h
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ class StripesSolver {
// Path
PathManager path_manager;

StripesSolver(const string & _puzzle_foler, int _stripes_n, int _samples_n, bool _real_flag);
StripesSolver(const string & _puzzle_folder, int _stripes_n, int _samples_n, bool _real_flag, double _word_conf_thres, double _lambda0, double _lambda1, double _U_a, double _filter_rate, int _candidate_factor);
~StripesSolver();

void m_metric();
Expand Down Expand Up @@ -91,12 +91,12 @@ class StripesSolver {
const string tesseract_model_path {"data/tesseract_model/"};

// -- For synthetic cases
const double word_conf_thres {70};
const double word_conf_thres = 70;
const double lambda0 = 0.3;
const double lambda1 = 0.5;
const double U_a = 2;
const double filter_rate = 0.7;
const int candidate_factor {4};
const int candidate_factor = 4;
// ---------------------


Expand Down
134 changes: 110 additions & 24 deletions src/solver/solve_puzzle.cpp
Original file line number Diff line number Diff line change
@@ -1,15 +1,26 @@
#include <solve_puzzle.h>

void solve_stripes( const string & stripes_folder,
const string & case_name,
int vertical_n,
int samples_n,
StripesSolver::Metric metric_mode,
StripesSolver::Composition composition_mode,
bool benchmark_flag,
bool real_flag) {
// Default parameters
string case_name = "doc0";
PuzzleType puzzle_type = PuzzleType::STRIPES;
int vertical_n = 4;
int samples_n = 10;
StripesSolver::Composition composition_mode = StripesSolver::GREEDY;
StripesSolver::Metric metric_mode = StripesSolver::PIXEL;
bool benchmark_flag = false;
bool real_flag = false;

// -- Default params for synthetic cases
double word_conf_thres = 70;
double lambda0 = 0.3;
double lambda1 = 0.5;
double U_a = 2;
double filter_rate = 0.7;
int candidate_factor = 4;

void solve_stripes( const string & stripes_folder) {

StripesSolver stripes_solver(stripes_folder, vertical_n, samples_n, real_flag);
StripesSolver stripes_solver(stripes_folder, vertical_n, samples_n, real_flag, word_conf_thres, lambda0, lambda1, U_a, filter_rate, candidate_factor);
stripes_solver.reassemble(metric_mode, composition_mode, case_name, benchmark_flag);

#ifdef DEBUG
Expand All @@ -26,24 +37,71 @@ void solve_stripes( const string & stripes_folder,
#endif
}

int main(int argc, char ** argv) {
void print_help() {
cout <<
"--test <str> The test case to evaluate.\n"
"--num <int> The number of stripes.\n"
"--comp <int> The composition type: 0 Greedy, 1 GCOM,\n"
" 2 Greedy and GCOM, 3 Groundtruth,\n"
" 4 user defined.\n"
"--metric <int> The similarity metric type: 0 pixel-level,\n"
" 1 character-level, 2 word-level.\n"
"--samples <int> When GCOM is set, it defines how many\n"
" sequences are sampled for word-level OCR. Note\n"
" that high reassembly score requires enough samples.\n"
" For 20 stripes, we recommend 150 samples.\n"
" For 30 stripes, we recommend 300 samples.\n"
" For 40 stripes, we recommend 1000 samples.\n"
" For 60 stripes, we recommend 8000 samples.\n"
"--benchmark Flag. Whether to write results in files.\n"
"--real Flag. Whether the test case is in real-world.\n";
" "
"--word_conf_thres <float> OCR score threshold [0-100]. Default 70.\n"
"--lambda0 <float> Balance char-level and pixel-level metrics in Eq. 5.\n"
" [0-1]. Default 0.3.\n"
"--lambda1 <float> Balance word-level and low-level metrics in Eq. 13.\n"
" [0-1]. Default: 0.5.\n"
"--u_a <float> Scale factor in Eq. 9. [0-100]. Default: 2.\n"
"--filter_rate <float> Filter out stripe pairs that have low-level scores. \n"
" [0-1]. Default: 0.7.\n"
"--candidate_factor <int> Seq len = Stripe num / candidate_factor. Default 4.\n";
}

// Default parameters
string case_name = "doc0";
PuzzleType puzzle_type = PuzzleType::STRIPES;
int vertical_n = 4;
int samples_n = 10;
StripesSolver::Composition composition_mode = StripesSolver::GREEDY;
StripesSolver::Metric metric_mode = StripesSolver::PIXEL;
bool benchmark_flag = false;
bool real_flag = false;
int main(int argc, char ** argv) {

// Parse command line parameters
const string opt_str = "t:n:c:m:s:br";
int opt = getopt(argc, argv, opt_str.c_str());
const string short_opts = "t:n:c:m:s:brh1:2:3:4:5:6:";
const option long_opts[] = {
{"test", required_argument, nullptr, 't'},
{"num", required_argument, nullptr, 'n'},
{"comp", required_argument, nullptr, 'c'},
{"metric", required_argument, nullptr, 'm'},
{"samples", required_argument, nullptr, 's'},
{"benchmark", no_argument, nullptr, 'b'},
{"real", no_argument, nullptr, 'r'},
{"help", no_argument, nullptr, 'h'},
{"word_conf_thres", required_argument, nullptr, 1},
{"lambda0", required_argument, nullptr, 2},
{"lambda1", required_argument, nullptr, 3},
{"u_a", required_argument, nullptr, 4},
{"filter_rate", required_argument, nullptr, 5},
{"candidate_factor", required_argument, nullptr, 6},
{nullptr, no_argument, nullptr, 0}
};
int opt_id;

while (true) {
const auto opt = getopt_long(argc, argv, short_opts.c_str(), long_opts, &opt_id);

if (opt == -1) break;

while (opt != -1) {
switch (opt) {

case 0:
printf("option %s", long_opts[opt_id].name);
if (optarg) printf(" with arg %s", optarg);
printf("\n");
break;
case 't':
case_name = string(optarg);
break;
Expand All @@ -65,12 +123,34 @@ int main(int argc, char ** argv) {
case 'r':
real_flag = true;
break;
case 'h':
print_help();
exit(0);

case 1:
word_conf_thres = atof(optarg);
break;
case 2:
lambda0 = atof(optarg);
break;
case 3:
lambda1 = atof(optarg);
break;
case 4:
U_a = atof(optarg);
break;
case 5:
filter_rate = atof(optarg);
break;
case 6:
candidate_factor = atoi(optarg);
break;
default:
print_help();
cerr << "[ ERR] Unknon options " << opt << endl;
exit(-1);
}

opt = getopt(argc, argv, opt_str.c_str());
}

const string metric_mode_str =
Expand All @@ -94,12 +174,18 @@ int main(int argc, char ** argv) {
if (composition_mode == StripesSolver::Composition::GCOM || composition_mode == StripesSolver::Composition::GREEDY_GCOM) {
cout << "Samples times: \t" << samples_n << endl;
}
cout << "word_conf_thres: \t" << word_conf_thres << endl;
cout << "lambda0: \t" << lambda0 << endl;
cout << "lambda1: \t" << lambda1 << endl;
cout << "u_a: \t" << U_a << endl;
cout << "filter_rate: \t" << filter_rate << endl;
cout << "candidate_factor: \t" << candidate_factor << endl;

// Import stripes
if (puzzle_type == PuzzleType::STRIPES) {

const string puzzle_folder = "data/stripes/" + case_name + "_" + to_string(vertical_n) + "/";
solve_stripes(puzzle_folder, case_name, vertical_n, samples_n, metric_mode, composition_mode, benchmark_flag, real_flag);
solve_stripes(puzzle_folder);

}

Expand Down
10 changes: 8 additions & 2 deletions src/solver/stripes_solver.cpp
Original file line number Diff line number Diff line change
@@ -1,11 +1,17 @@
#include <stripes_solver.h>

StripesSolver::StripesSolver(const string & _puzzle_folder, int _stripes_n, int _samples_n, bool _real_flag) :
StripesSolver::StripesSolver(const string & _puzzle_folder, int _stripes_n, int _samples_n, bool _real_flag, double _word_conf_thres, double _lambda0, double _lambda1, double _U_a, double _filter_rate, int _candidate_factor) :
puzzle_folder(_puzzle_folder),
stripes_n(_stripes_n),
candidate_seqs_n(_samples_n),
path_manager(_stripes_n, _samples_n),
real_flag(_real_flag) {
real_flag(_real_flag),
word_conf_thres(_word_conf_thres),
lambda0(_lambda0),
lambda1(_lambda1),
U_a(_U_a),
filter_rate(_filter_rate),
candidate_factor(_candidate_factor) {

// Timestamp array
ts_arr.clear();
Expand Down

0 comments on commit e77e969

Please sign in to comment.