Error with Phase 2: 'NoneType' object is not subscriptable

[aquilaziale]

Hello,

I'm novice in the creation of dataset, and I would like to thank you for this tool.

I'm doing a test on my computer with a small document using Ollama.

'Phase 0' and 'Phase 1' seems correctly processed. It generate 4 questions in 'raw_qatuples_saved' directory.
But 'Phase 2' fails.
It creates json file in 'qatuples_filtered' containing only the word 'failed'.

Here are the log:

025-01-20 20:38:08,468 - INFO - HTTP Request: POST http://localhost:11434/v1/chat/completions "HTTP/1.1 200 OK"
Output written to ./output/check_question_generations\fde77876-b64c-49f9-a181-db47c0886cdd--subquestion--8e1bd16e-9086-4e8e-b121-794f87e85f13--check--d8067d27-7b38-4a9c-93bc-323fed01e94a.yaml
!!ERROR!!
'NoneType' object is not subscriptable
Traceback (most recent call last):
File "C:\IA-Projet\augmentoolkit\original\steps.py", line 749, in vet_question_loop
if not judgement[0]: # if not relevant
~~~~~~~~~^^^
TypeError: 'NoneType' object is not subscriptable
25%|████████████████████▊ | 1/4 [02:04<06:13, 124.63s/it]2025-01-20 20:40:11,996 - INFO - HTTP Request: POST http://localhost:11434/v1/chat/completions "HTTP/1.1 200 OK"
Output written to ./output/check_question_generations\fde77876-b64c-49f9-a181-db47c0886cdd--subquestion--eeb90408-9c26-4d94-9632-12a12a6a48da--check--6ee09efc-14b5-41df-b8d3-16fc7031f8ef.yaml
!!ERROR!!
'NoneType' object is not subscriptable
Traceback (most recent call last):
File "C:\IA-Projet\augmentoolkit\original\steps.py", line 749, in vet_question_loop
if not judgement[0]: # if not relevant
~~~~~~~~~^^^
TypeError: 'NoneType' object is not subscriptable
50%|█████████████████████████████████████████▌ | 2/4 [04:09<04:09, 124.69s/it]2025-01-20 20:42:15,859 - INFO - HTTP Request: POST http://localhost:11434/v1/chat/completions "HTTP/1.1 200 OK"
Output written to ./output/check_question_generations\fde77876-b64c-49f9-a181-db47c0886cdd--subquestion--7fbee023-71b7-47af-8eb9-01f3440457c9--check--e8b3c45d-68dd-4f6d-9341-727e41f320b1.yaml
!!ERROR!!
'NoneType' object is not subscriptable
Traceback (most recent call last):
File "C:\IA-Projet\augmentoolkit\original\steps.py", line 749, in vet_question_loop
if not judgement[0]: # if not relevant
~~~~~~~~~^^^
TypeError: 'NoneType' object is not subscriptable
75%|██████████████████████████████████████████████████████████████▎ | 3/4 [06:16<02:05, 125.88s/it]2025-01-20 20:44:23,892 - INFO - HTTP Request: POST http://localhost:11434/v1/chat/completions "HTTP/1.1 200 OK"
Output written to ./output/check_question_generations\fde77876-b64c-49f9-a181-db47c0886cdd--subquestion--cb7de81a-edb3-4786-963e-06fab0702ff7--check--39056e19-5238-4673-85f9-0d51ec756e2a.yaml
!!ERROR!!
'NoneType' object is not subscriptable
Traceback (most recent call last):
File "C:\IA-Projet\augmentoolkit\original\steps.py", line 749, in vet_question_loop
if not judgement[0]: # if not relevant
~~~~~~~~~^^^
TypeError: 'NoneType' object is not subscriptable
100%|███████████████████████████████████████████████████████████████████████████████████| 4/4 [08:26<00:00, 126.51s/it]

My ollama-config-normal.yaml:

API:
LARGE_API_KEY: key-here
LARGE_MODEL: llama3.1:8b-instruct-q8_0
LARGE_BASE_URL: http://localhost:11434/v1
LARGE_MODE: api
SMALL_MODEL: llama3.1:8b-instruct-q8_0
SMALL_BASE_URL: http://localhost:11434/v1
SMALL_API_KEY: key-here
SMALL_MODE: api
HUGGINGFACE:
HUB_PATH: yourusername/your-path-here
PRIVATE: false
PUSH_TO_HUB: false
PATH:
DEFAULT_PROMPTS: ./prompts
INPUT: ./input
OUTPUT: ./output
PROMPTS: ./prompts
PHASE:
PHASE_INDEX: 2
WORK_IN_PHASES: true
SKIP:
ANSWER_RELEVANCY_CHECK: false
REPAIR_QA_TUPLES: false
FILTER_CHUNKS: false
QUESTION_CHECK: false
CONVERSATION_GENERATION: false
SYSTEM:
CHUNK_SIZE: 1900
COMPLETION_MODE: false
CONCURRENCY_LIMIT: 50
CONVERSATION_INSTRUCTIONS: Pour cette conversation, tu génères une discussion entre un assistant IA et un humain. En français.
DOUBLE_CHECK_COUNTER: 1
DO_NOT_USE_SYSTEM_PROMPTS: true
FINAL_ASSISTANT_PROMPTS_NO_RAG:

• Tu es un assistant IA très utile.

• 'Tu es une intelligence artificielle avec des connaissance experte dans beaucoup de domaine.

  Use tes connaissances pour répondre aux questions.'

• u are ai asstant plz answr questions
  FINAL_ASSISTANT_PROMPTS_RAG:

• 'Tu es un assistant très utile. Voici des connaissances :

  {data}'

• '{data}

  Tu es un expert dans le domaine. Réponds aux questions'

• 'Tu est une intelligence artificielle avec beaucoup de connaissance. Voici un context potentiellement utile:

  {data}

  Réponds aux questions grâce à tes connaissances.'
  STOP: true
  SUBSET_SIZE: 20
  USE_FILENAMES: false
  USE_SUBSET: false
  RAG_FAILURE_PERCENTAGE: 0.1
  SCRAPING:
  USE_GUTENBERG: false
  START_URL: https://www.gutenberg.org/ebooks/bookshelf/57
  MAX_BOOKS: 5
  MAX_FAILURES: 5

As I'm testing on my local computer, I'm using small models.
I've tested with:
llama3.1:8b-instruct-q8_0
mistral
llama3.2-vision
qwen2.5

I also tried with others document but I always have the same issue.

Could you help me?

Thank you very much.

[knyeoh]

I also encounter the same issue, and I have trace it back to this file: output/judge_paragraph_generations/intermediate_generations/4.yaml

content should not be blank, if you send blank content to LLM, it will create error. I am not sure how to fix this. Anyone have idea what may have cause this?