Skip to content

Latest commit

 

History

History
22 lines (15 loc) · 943 Bytes

README.md

File metadata and controls

22 lines (15 loc) · 943 Bytes
Raw

FTIIBench

(ARXIV24) This is the official code repository for "FTII-Bench: A Comprehensive Multimodal Benchmark for Flow Text with Image Insertion."

Dataset

The text of FTII-Bench could be download from Google Drive

The images of FTII-Bench could be download from Google Drive

Note that the data is only used for research purposes!

Evaluation

  1. Set the appropriate paths in the run_eval_fi and run_eval_sc scripts. 
    bash run_eval_fi.sh # for flow insertion tasks
bash run_eval_sc.sh # for single choice tasks
  2. For evaluating with BGE models You can run ./mllm_eval/bge_eval.ipynb in the Jupyter environment.

Acknowledgement

Thanks to the open-source code from Mantis