Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add script to clean up e2e #366

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

add script to clean up e2e #366

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
106 changes: 106 additions & 0 deletions e2e/clean/clean.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
import {
client,
PipelinesService,
ProjectsService,
} from "@llamaindex/cloud/api";
import { DEFAULT_BASE_URL } from "@llamaindex/core/global";

function initService(apiKey?: string) {
client.setConfig({
baseUrl: DEFAULT_BASE_URL,
throwOnError: true,
});
const token = apiKey ?? process.env.LLAMA_CLOUD_API_KEY;
client.interceptors.request.use((request: any) => {
request.headers.set("Authorization", `Bearer ${token}`);
return request;
});
if (!token) {
throw new Error(
"API Key is required for LlamaCloudIndex. Please set the LLAMA_CLOUD_API_KEY environment variable",
);
}
}
Comment on lines +13 to +23
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Move token validation before setting up the interceptor

Currently, the interceptor is set up before verifying that token is defined, which could lead to setting an Authorization header with an undefined token, causing unintended behavior. To prevent this, validate that token is defined before setting up the interceptor.

Apply this diff to fix the issue:

 function initService(apiKey?: string) {
   client.setConfig({
     baseUrl: DEFAULT_BASE_URL,
     throwOnError: true,
   });
   const token = apiKey ?? process.env.LLAMA_CLOUD_API_KEY;
+  if (!token) {
+    throw new Error(
+      "API Key is required for LlamaCloudIndex. Please set the LLAMA_CLOUD_API_KEY environment variable",
+    );
+  }
   client.interceptors.request.use((request: any) => {
     request.headers.set("Authorization", `Bearer ${token}`);
     return request;
   });
-  if (!token) {
-    throw new Error(
-      "API Key is required for LlamaCloudIndex. Please set the LLAMA_CLOUD_API_KEY environment variable",
-    );
-  }
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const token = apiKey ?? process.env.LLAMA_CLOUD_API_KEY;
client.interceptors.request.use((request: any) => {
request.headers.set("Authorization", `Bearer ${token}`);
return request;
});
if (!token) {
throw new Error(
"API Key is required for LlamaCloudIndex. Please set the LLAMA_CLOUD_API_KEY environment variable",
);
}
}
const token = apiKey ?? process.env.LLAMA_CLOUD_API_KEY;
if (!token) {
throw new Error(
"API Key is required for LlamaCloudIndex. Please set the LLAMA_CLOUD_API_KEY environment variable",
);
}
client.interceptors.request.use((request: any) => {
request.headers.set("Authorization", `Bearer ${token}`);
return request;
});
}


async function getProjectId(projectName: string): Promise<string> {
const { data: projects } = await ProjectsService.listProjectsApiV1ProjectsGet(
{
query: {
project_name: projectName,
},
throwOnError: true,
},
);

if (projects.length === 0) {
throw new Error(
`Unknown project name ${projectName}. Please confirm a managed project with this name exists.`,
);
} else if (projects.length > 1) {
throw new Error(
`Multiple projects found with name ${projectName}. Please specify organization_id.`,
);
}

const project = projects[0]!;

if (!project.id) {
throw new Error(`No project found with name ${projectName}`);
}

return project.id;
}

async function deletePipelines(projectName: string) {
try {
initService();

const projectId = await getProjectId(projectName);

const { data: pipelines } =
await PipelinesService.searchPipelinesApiV1PipelinesGet({
query: { project_id: projectId },
throwOnError: true,
});

console.log(`Deleting pipelines for project "${projectName}":`);

for (const pipeline of pipelines) {
if (pipeline.id) {
try {
await PipelinesService.deletePipelineApiV1PipelinesPipelineIdDelete({
path: { pipeline_id: pipeline.id },
throwOnError: true,
});
console.log(
`✅ Deleted pipeline: ${pipeline.name} (ID: ${pipeline.id})`,
);
} catch (error) {
console.error(
`❌ Failed to delete pipeline: ${pipeline.name} (ID: ${pipeline.id})`,
);
console.error(
` Error: ${error instanceof Error ? error.message : String(error)}`,
);
}
} else {
console.warn(`⚠️ Skipping pipeline with no ID: ${pipeline.name}`);
}
}

console.log(`\nDeletion process completed for project "${projectName}".`);
console.log(`Total pipelines processed: ${pipelines.length}`);
} catch (error) {
console.error("Error during pipeline deletion process:", error);
}
Comment on lines +93 to +95
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Exit with a non-zero exit code when an error occurs

When an error occurs during the pipeline deletion process, the script logs the error but exits with a zero exit code. To indicate failure to the calling process or CI tools, consider exiting with a non-zero exit code.

Apply this diff to fix the issue:

 } catch (error) {
   console.error("Error during pipeline deletion process:", error);
+  process.exit(1);
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
} catch (error) {
console.error("Error during pipeline deletion process:", error);
}
} catch (error) {
console.error("Error during pipeline deletion process:", error);
process.exit(1);
}

}

// Get the project name from command line arguments
const projectName = process.argv[2];

if (!projectName) {
console.error("Please provide a project name as an argument.");
process.exit(1);
}

Comment on lines +100 to +105
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Consider using a command-line argument parser for scalability

Directly accessing process.argv works for simple scripts, but as the script evolves or requires more complex command-line options, using a library like yargs or commander can enhance scalability and maintainability.

For example, you could modify your script to use yargs:

+import yargs from 'yargs';
+
+const argv = yargs(process.argv.slice(2))
+  .usage('Usage: $0 --project <projectName>')
+  .option('project', {
+    alias: 'p',
+    describe: 'Name of the project',
+    type: 'string',
+    demandOption: true,
+  })
+  .help()
+  .argv;
+
+const projectName = argv.project;
-
-const projectName = process.argv[2];
-
-if (!projectName) {
-  console.error("Please provide a project name as an argument.");
-  process.exit(1);
-}
-
 deletePipelines(projectName);

Remember to install yargs by adding it to your dependencies.

Committable suggestion was skipped due to low confidence.

deletePipelines(projectName);
18 changes: 18 additions & 0 deletions e2e/clean/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{
"name": "@create-llama/e2e-clean",
"version": "0.1.0",
"private": true,
"type": "module",
"scripts": {
"clean": "tsx clean.ts create-llama"
},
"devDependencies": {
"@types/node": "^20.0.0",
"tsx": "^4.19.1"
},
"dependencies": {
"@llamaindex/cloud": "^0.2.14",
"@llamaindex/core": "^0.2.12",
"tiktoken": "^1.0.17"
}
}
Loading
Loading