Skip to content

Working with Sessions (preprocessed prompt)

John edited this page Jul 7, 2023 · 1 revision

If you are working with large prompts or if plan to run the same initial prompt with many changing questions you will want to use sessions.

Basic usage:

  • --prompt-cache <path_to_session_file> to create or read from a session file
  • --prompt-cache-ro this will make the file readonly
  • -f <path_to_prompt_file> to read a entire file as prompt
  • -p "your prompt" this prompt would be prepended to the file, if you want to add a custom prefix to it
  • -enc this will enclose your entire prompt with a fine-tune specific syntax (OpenAssistant, Instruct, Wizard, Generic)

Save the processed prompt to a session file

falcon_main.exe -m Q:\models\OpenAssist\falcon-7b-sft-mix-2000\q4_1.ggcc  -c 512 -n 128 -p "If my jacket is red and my eyes are blue, what would be a good color for my shoes?" --prompt-cache q:\caches\example.cache -enc --color
<|prompter|>If my jacket is red and my eyes are blue, what would be a good color for my shoes?<|endoftext|><|assistant|>
That depends on your personal style and preferences. Some popular colors that could work well with a red jacket and blue eyes include:

1. Black or dark brown - These classic colors will add contrast to the red jacket and bring out the blue

Use previous session to skip the prompt processing except for the last part

falcon_main.exe -m Q:\models\OpenAssist\falcon-7b-sft-mix-2000\q4_1.ggcc  -c 512 -n 128 -p "If my jacket is red and my eyes are blue, what would be a good color for my shirt?" --prompt-cache q:\caches\example.cache -enc --color --prompt-cache-ro
<|prompter|>If my jacket is red and my eyes are blue, what would be a good color for my shirt?<|endoftext|><|assistant|>
I would say that a light blue or dark blue shirt would go well with the red jacket and blue eyes. Both colors have a calming effect and create a sense of balance. If you want to add an accent color, you could also try a light