squash! finetune: rename feed-forward tensors (w1/w2/w3)

This commit updates the README.md file to reflect the new names of the feed-forward tensors. Signed-off-by: Daniel Bevenius <[email protected]>
danbev · Jan 18, 2024 · 77cfcb4 · 77cfcb4
1 parent 039f1ce
commit 77cfcb4
Showing 1 changed file with 3 additions and 3 deletions.
diff --git a/examples/finetune/README.md b/examples/finetune/README.md
@@ -80,9 +80,9 @@ The LORA rank can be configured for each model tensor type separately with these
   --rank-wk N                LORA rank for wk tensor (default 4)
   --rank-wv N                LORA rank for wv tensor (default 4)
   --rank-wo N                LORA rank for wo tensor (default 4)
-  --rank-w1 N                LORA rank for w1 tensor (default 4)
-  --rank-w2 N                LORA rank for w2 tensor (default 4)
-  --rank-w3 N                LORA rank for w3 tensor (default 4)
+  --rank-ffn_gate N          LORA rank for ffn_gate tensor (default 4)
+  --rank-ffn_down N          LORA rank for ffn_down tensor (default 4)
+  --rank-ffn_up N            LORA rank for ffn_up tensor (default 4)
 ```
 
 The LORA rank of 'norm' tensors should always be 1.