Release 1.4.2 (#112)

* Pagination bug * Bug fix * chore: add docker cmd * Compatibility fixes for SDK version 2.0.0 (#69) * Pagination bug * Bug fix * Fix for schema changes * Render tool calling * Support for Langgraph, Qdrant & Groq (#73) * Pagination bug * Bug fix * Add langgraph support * QDrant support * Add Groq support * update README * update README * feat: optimise docker image for self host setup * adding api access to traces endpoint * clean up * refactor * feat: add clickhouse db create on app start (#79) * docs: add railway deploy, fix sdk badges (#81) * untrack .env * Revert "untrack .env" This reverts commit 4551d7e. * Playground and Prompt Management (#83) * Pagination bug * Bug fix * Playground - basic implementation * Playground - streaming and nonstreaming * Move playground inside project * API key flow * Api key * Playground refactor * Add chat hookup * anthropic streaming support * Bug fixes to openai playground * Anthropic bugfixes * Anthropic bugfix * Cohere first iteration * Cohere role fixes * Cohere api fix * Parallel running * Playground cost calculation non streaming * playground - streaming token calculation * latency and cost * Support for Groq * Add model name * Prompt management views * Remove current promptset flow * Prompt management API hooks * Prompt registry final * Playground bugfixes * Bug fix playground * Rearrange project nav * Fix playground * Fix prompts * Bugfixes * Minor fix * Prompt versioning bugfix * Bugfix * fix: clickhouse table find queries (#82) * Fix to surface multiple LLM requests inside LLM View (#84) * Pagination bug * Bug fix * Fix for surfacing multiple LLM requests in LLMView * Minor bugfixes (#86) * Pagination bug * Bug fix * Bugfixes * api to fetch promptset with prompt filters * bug fixes * fix invalid redirect * fix invalid status code * Project Switcher (#90) * Pagination bug * Bug fix * Project switcher * Feat: dataset download (#60) * API: download dataset * API: Download dataset * updated download-dataset api * Updated: download_dataset api * Updated download dataset API * Updated Download API: changed Response to Next Response, add a condition to ensure max page size is 500 * updated the download-dataset API: fixed the format and removed redundant lines of code * Updated download_daatset API: file name and removed 'id' param * Added the Download dataset button. * Merged developemnt into my branch * Updated button size * Fixes --------- Co-authored-by: Karthik Kalyanaraman <[email protected]> * Update prompt registry with instructions to fetch prompts (#91) * Pagination bug * Bug fix * Update prompt registry * Minor bugfix (#94) * Pagination bug * Bug fix * Minor bugfix * chore: update github repo badges * optimizing token count function * Add GPT4-O Pricing and Playground (#98) * Pagination bug * Bug fix * Add GPT4-O support * Add GPT4-O support * Update cost * Dylan/s3en 2234 add perplexity support to playground (#89) * adding perplexity to playground types * adding ui stuff:' * adding perplexity chat api * fixing perplexity model dropdown --------- Co-authored-by: Karthik Kalyanaraman <[email protected]> * api changes * add api access to get api and fix all bugs * bug fix * bug fix * updating descriptions to optional * prio python * cleanup and fixes * more bug fixes * more fixes * remove console log * updating trace_service functions * add migration * add format function, updating from day to hour * adding dropwdown menu * updating query key * updating query keys v2 * clean up * fix bug * Minor bugfix (#102) * Pagination bug * Bug fix * Minor bugfix * fixes * Bug fixes to Evaluations (#111) * Pagination bug * Bug fix * Bug fixes to evals v2 * Minor fix * upgrading tiktoken, adding omni to pricing (#110) Co-authored-by: Karthik Kalyanaraman <[email protected]> * Update package --------- Co-authored-by: Darshit Suratwala <[email protected]> Co-authored-by: darshit-s3 <[email protected]> Co-authored-by: dylan <[email protected]> Co-authored-by: dylanzuber-scale3 <[email protected]> Co-authored-by: Rohit Kadhe <[email protected]> Co-authored-by: Rohit Kadhe <[email protected]> Co-authored-by: MayuriS24 <[email protected]>
Scale3-Labs · May 21, 2024 · eefb47a · eefb47a
1 parent 86f259e
commit eefb47a
Show file tree

Hide file tree

Showing 28 changed files with 14,998 additions and 334 deletions.
diff --git a/...roject_id]/evaluations/[test_id]/page.tsx → .../[project_id]/evaluate/[test_id]/page.tsx b/...roject_id]/evaluations/[test_id]/page.tsx → .../[project_id]/evaluate/[test_id]/page.tsx
@@ -1,7 +1,7 @@
 "use client";
 
-import { ScaleType } from "@/components/evaluations/eval-scale-picker";
-import { RangeScale } from "@/components/evaluations/range-scale";
+import { ScaleType } from "@/components/evaluate/eval-scale-picker";
+import { RangeScale } from "@/components/evaluate/range-scale";
 import UserLogo from "@/components/shared/user-logo";
 import { VendorLogo } from "@/components/shared/vendor-metadata";
 import { Button } from "@/components/ui/button";
@@ -44,7 +44,7 @@ export default function Page() {
         toast.error("Failed to fetch the test", {
           description: error?.message || "Failed to fetch test",
         });
-        router.push(`/project/${projectId}/evaluations`);
+        router.push(`/project/${projectId}/evaluate`);
         return;
       }
       const result = await response.json();
@@ -71,16 +71,6 @@ export default function Page() {
           operation: "NOT_EQUALS",
           value: "",
         },
-        // Accuracy is the default test. So no need to
-        // send the testId with the spans when using the SDK.
-        {
-          key: "langtrace.testId",
-          operation: "EQUALS",
-          value:
-            testData?.test?.name.toLowerCase() !== "factual accuracy"
-              ? testData?.test?.id
-              : "",
-        },
       ];
 
       // convert filterserviceType to a string
@@ -106,7 +96,7 @@ export default function Page() {
         toast.error("Failed to fetch the span data", {
           description: error?.message || "Failed to fetch the span data",
         });
-        router.push(`/project/${projectId}/evaluations`);
+        router.push(`/project/${projectId}/evaluate`);
         return;
       }
 
@@ -123,7 +113,7 @@ export default function Page() {
         page <= 0 ||
         page > parseInt(metadata?.total_pages)
       ) {
-        router.push(`/project/${projectId}/evaluations`);
+        router.push(`/project/${projectId}/evaluate`);
       }
 
       // Update the total pages and current page number
@@ -150,17 +140,15 @@ export default function Page() {
     evaluate();
     if (page < totalPages) {
       const nextPage = page + 1;
-      router.push(
-        `/project/${projectId}/evaluations/${testId}?page=${nextPage}`
-      );
+      router.push(`/project/${projectId}/evaluate/${testId}?page=${nextPage}`);
     }
   };
 
   const previous = () => {
     if (page > 1) {
       const previousPage = page - 1;
       router.push(
-        `/project/${projectId}/evaluations/${testId}?page=${previousPage}`
+        `/project/${projectId}/evaluate/${testId}?page=${previousPage}`
       );
     }
   };
@@ -174,12 +162,14 @@ export default function Page() {
         toast.error("Failed to fetch the evaluation data", {
           description: error?.message || "Failed to fetch the evaluation data",
         });
-        router.push(`/project/${projectId}/evaluations`);
+        router.push(`/project/${projectId}/evaluate`);
         return;
       }
       const result = await response.json();
       const sc =
-        result.evaluations.length > 0 ? result.evaluations[0].ltUserScore ?? -1 : -1;
+        result.evaluations.length > 0
+          ? result.evaluations[0].ltUserScore ?? -1
+          : -1;
       onScoreSelected(sc);
       return result;
     },
@@ -196,7 +186,7 @@ export default function Page() {
       }
 
       if (event.key === "Escape") {
-        router.push(`/project/${projectId}/evaluations`);
+        router.push(`/project/${projectId}/evaluate`);
       }
     };
 
@@ -233,7 +223,7 @@ export default function Page() {
           body: JSON.stringify({
             id: evaluationsData.evaluations[0].id,
             ltUserScore: score,
-            testId
+            testId,
           }),
         });
         queryClient.invalidateQueries({
@@ -361,7 +351,7 @@ export default function Page() {
               ))}
             </div>
           ) : (
-            < RangeScale
+            <RangeScale
               variant="large"
               type={ScaleType.Range}
               min={testData?.test?.min}
@@ -432,7 +422,7 @@ export default function Page() {
         <div className="absolute bottom-5 right-5 flex gap-2 items-center">
           <Button
             variant={"outline"}
-            onClick={() => router.push(`/project/${projectId}/evaluations`)}
+            onClick={() => router.push(`/project/${projectId}/evaluate`)}
             disabled={isEvaluationLoading}
           >
             Exit
@@ -478,8 +468,8 @@ function ConversationView({ span }: { span: any }) {
           const content = prompt?.content
             ? safeStringify(prompt?.content)
             : prompt?.function_call
-              ? safeStringify(prompt?.function_call)
-              : "No input found";
+            ? safeStringify(prompt?.function_call)
+            : "No input found";
           return (
             <div key={i} className="flex flex-col gap-2">
               <div className="flex gap-2 items-center">

diff --git a/.../[project_id]/evaluations/page-client.tsx → ...ect/[project_id]/evaluate/page-client.tsx b/.../[project_id]/evaluations/page-client.tsx → ...ect/[project_id]/evaluate/page-client.tsx
@@ -2,11 +2,11 @@
 
 import { EvalChart } from "@/components/charts/eval-chart";
 import LargeChartSkeleton from "@/components/charts/large-chart-skeleton";
-import { CreateTest } from "@/components/evaluations/create-test";
-import { EditTest } from "@/components/evaluations/edit-test";
+import { CreateTest } from "@/components/evaluate/create-test";
+import { EditTest } from "@/components/evaluate/edit-test";
 import EvaluationTable, {
   EvaluationTableSkeleton,
-} from "@/components/evaluations/evaluation-table";
+} from "@/components/evaluate/evaluation-table";
 import { AddtoDataset } from "@/components/shared/add-to-dataset";
 import { Button } from "@/components/ui/button";
 import { Separator } from "@/components/ui/separator";
@@ -106,17 +106,17 @@ export default function PageClient({ email }: { email: string }) {
   return (
     <div className="w-full flex flex-col">
       <div className="md:px-24 px-12 py-12 flex justify-between bg-muted">
-        <h1 className="text-3xl font-semibold">Evaluations</h1>
+        <h1 className="text-3xl font-semibold">Manual Evaluations</h1>
         <div className="flex gap-2">
           {selectedTest && (
             <Link
-              href={`/project/${projectId}/evaluations/${selectedTest?.id}?page=1`}
+              href={`/project/${projectId}/evaluate/${selectedTest?.id}?page=1`}
             >
               <Button
                 className="bg-gradient-to-r from-pink-500 via-red-500 to-yellow-500 background-animate"
                 variant="default"
               >
-                Start Testing <ChevronsRight className="ml-2" />{" "}
+                Start Evaluating <ChevronsRight className="ml-2" />{" "}
               </Button>
             </Link>
           )}

diff --git a/...project/[project_id]/evaluations/page.tsx → ...d)/project/[project_id]/evaluate/page.tsx b/...project/[project_id]/evaluations/page.tsx → ...d)/project/[project_id]/evaluate/page.tsx