diff --git a/metrics/low_scale_vLLM/Readme.md b/metrics/low_scale_vLLM/Readme.md index 8dc4ad6..980f39d 100644 --- a/metrics/low_scale_vLLM/Readme.md +++ b/metrics/low_scale_vLLM/Readme.md @@ -3,7 +3,7 @@ ## Overview This repository contains Jupyter notebooks that demonstrate how to use key-value and table extraction services provided by various cloud providers. Specifically, we use Amazon Textract, Google Document AI, and Azure Form Recognizer to extract information from images. Additionally, there is a notebook for extracting key-value and table information as JSON files from in-house labeled ground truth. - +We provide 2 ground-truth files and the results of our in-house models. The cloud providers results can be extracted with the ressources below. ## File Structure - `credentials.json`: Configuration file containing the necessary credentials for accessing the cloud services. @@ -17,21 +17,21 @@ This repository contains Jupyter notebooks that demonstrate how to use key-value - `Generic_Report_GT_KV_Table_Extraction.ipynb`: Notebook for extracting key-value pairs and tables from in-house labeled ground truth. - `Evaluate_KV_Results.ipynb`: Notebook for evaluating key-value results. -- `data/`: datasets to reproduce metrics. - - +- `data/`: datasets to reproduce metrics. Images folder contains all the images used, and the rest of the zip files contains the inference results in json format. + - `images/`: This folder contain the images used + - `FUNSD_images.zip`: Funsd images + - `GenericReport01_images.zip`: GenericReport data images - `FUNSD_GT.zip`: FUNSD Ground-Truth - `GenericReports01_GT.zip`: Generic Report Ground-Truth - `vLLM_v1_GenericReports01_fixoutput.zip`: Result of vLLM_v1 in Generic_Report01 - `vLLM_v1_funsd_fixoutput.zip`: Result of vLLM_v1 in FUNSD - - `vLLM_v2_GenericReports01_fixoutput.zip`: Result of vLLM_v in Generic_Report01 + - `vLLM_v2_GenericReports01_fixoutput.zip`: Result of vLLM_v2 in Generic_Report01 - `vLLM_v2_funsd_fixoutput.zip`: Result of vLLM_v2 in FUNSD - `vLLM_v3_GenericReports01_fixoutput.zip`: Result of vLLM_v3 in Generic_Report01 - `vLLM_v3_funsd_fixoutput.zip`: Result of vLLM_v3 in FUNSD ## Prerequisites -- Make sure images are available for processing. - Add your credentials to the `credentials.json` file, before attempting to run any of the notebooks. - Python 3.7+ - Jupyter Notebook diff --git a/metrics/low_scale_vLLM/credentials.json b/metrics/low_scale_vLLM/credentials.json index 714e1d1..2814275 100644 --- a/metrics/low_scale_vLLM/credentials.json +++ b/metrics/low_scale_vLLM/credentials.json @@ -1,9 +1,10 @@ { + "_comment": "Here below you pout your AWS credentials", "access_id" : "", "secret_key" : "", "region" : "", "mfa" : "", - "role_arn" : "", + "role_mfa" : "", "azure_vision_key" : "", "azure_vision_endpoint" : "", "google_document_location" : "", diff --git a/metrics/low_scale_vLLM/data/images/FUNSD_images.zip b/metrics/low_scale_vLLM/data/images/FUNSD_images.zip new file mode 100644 index 0000000..4bebc7e Binary files /dev/null and b/metrics/low_scale_vLLM/data/images/FUNSD_images.zip differ diff --git a/metrics/low_scale_vLLM/data/images/GenericReport01_images.zip b/metrics/low_scale_vLLM/data/images/GenericReport01_images.zip new file mode 100644 index 0000000..55b6605 Binary files /dev/null and b/metrics/low_scale_vLLM/data/images/GenericReport01_images.zip differ