forked from ggerganov/llama.cpp
-
Notifications
You must be signed in to change notification settings - Fork 21
Working with Sessions (preprocessed prompt)
John edited this page Jul 7, 2023
·
1 revision
If you are working with large prompts or if plan to run the same initial prompt with many changing questions you will want to use sessions.
Basic usage:
-
--prompt-cache <path_to_session_file>
to create or read from a session file -
--prompt-cache-ro
this will make the file readonly -
-f <path_to_prompt_file>
to read a entire file as prompt -
-p "your prompt"
this prompt would be prepended to the file, if you want to add a custom prefix to it -
-enc
this will enclose your entire prompt with a fine-tune specific syntax (OpenAssistant, Instruct, Wizard, Generic)
Save the processed prompt to a session file
falcon_main.exe -m Q:\models\OpenAssist\falcon-7b-sft-mix-2000\q4_1.ggcc -c 512 -n 128 -p "If my jacket is red and my eyes are blue, what would be a good color for my shoes?" --prompt-cache q:\caches\example.cache -enc --color
<|prompter|>If my jacket is red and my eyes are blue, what would be a good color for my shoes?<|endoftext|><|assistant|>
That depends on your personal style and preferences. Some popular colors that could work well with a red jacket and blue eyes include:
1. Black or dark brown - These classic colors will add contrast to the red jacket and bring out the blue
Use previous session to skip the prompt processing except for the last part
falcon_main.exe -m Q:\models\OpenAssist\falcon-7b-sft-mix-2000\q4_1.ggcc -c 512 -n 128 -p "If my jacket is red and my eyes are blue, what would be a good color for my shirt?" --prompt-cache q:\caches\example.cache -enc --color --prompt-cache-ro
<|prompter|>If my jacket is red and my eyes are blue, what would be a good color for my shirt?<|endoftext|><|assistant|>
I would say that a light blue or dark blue shirt would go well with the red jacket and blue eyes. Both colors have a calming effect and create a sense of balance. If you want to add an accent color, you could also try a light