From ac5bd970829a139b1c161b74eb6d66f20a46af68 Mon Sep 17 00:00:00 2001 From: Shishir Patil Date: Tue, 28 Nov 2023 17:58:46 +0530 Subject: [PATCH] Fixing link to RAT, and minor mods to OpenFunctions --- blog.html | 2 +- blogs/4_open_functions.html | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/blog.html b/blog.html index d79f0c4f7..ad3e7a1e9 100644 --- a/blog.html +++ b/blog.html @@ -285,7 +285,7 @@

Reality Bytes: How to Measure Hallucinations in LLMs

- +

Retriever-Aware Training (RAT): Are LLMs memorizing or understanding?

@@ -214,7 +214,7 @@

OpenFunctions Performance Benchmarking


When calling to OpenAI's function calling models, if the required parameters are not supplied within the instruction, this leads the function calling models to output "follow-up" questions requesting the required parameters. This results in "incomplete" status as displayed in above graph. - We treat "incomplete" execution as "success" during our accuracy calculation because the model recognized the missing parameters successfully. + We treat "incomplete" execution as "success" during our accuracy calculation because the model recognized the missing parameters successfully. Note that this is consistent across all evaluations. Our OpenFunctions model, as well as regular GPT-4, due to its chat completion nature, will fill up the required parameters with placeholders or default values, allowing undisturbed generation.