Small improvements in RM docs (#196)

Signed-off-by: Terry Kong <[email protected]>
NVIDIA · Jun 10, 2024 · eaa2738 · eaa2738
1 parent 717e2fb
commit eaa2738
Showing 1 changed file with 4 additions and 6 deletions.
diff --git a/docs/user-guide/RLHF.rst b/docs/user-guide/RLHF.rst
@@ -40,10 +40,10 @@ where || denotes string concatenation and *prompt1* and *prompt2* are different
 
 An example JSONL file can look like the following::
 
-  {"text": User: When did Virgin Australia start operating?\nAssistant: 31 August 2000}
-  {"text": User: When did Virgin Australia start operating?\nAssistant: I refuse to answer this question.}
-  {"text": User: What is 6*10?\nAssistant: 60}
-  {"text": User: What is 6*10?\nAssistant: 90}
+  {"text": "User: When did Virgin Australia start operating?\nAssistant: 31 August 2000"}
+  {"text": "User: When did Virgin Australia start operating?\nAssistant: I refuse to answer this question."}
+  {"text": "User: What is 6*10?\nAssistant: 60"}
+  {"text": "User: What is 6*10?\nAssistant: 90"}
   ...
 
 
@@ -62,8 +62,6 @@ To launch reward model training, you must start with a pretrained or SFT trained
             TRAIN_DATA_PATH="/path/to/train_comparisons.jsonl"
             VALID_DATA_PATH="/path/to/test_comparisons.jsonl"
 
-            GPFS="/path/to/nemo-aligner-repo"
-
             python -u ${GPFS}/examples/nlp/gpt/train_reward_model.py \
                trainer.num_nodes=1 \
                trainer.devices=8 \