diff --git a/docs/code/execute/execute_python.md b/docs/code/execute/execute_python.md index 280695b..09d263d 100644 --- a/docs/code/execute/execute_python.md +++ b/docs/code/execute/execute_python.md @@ -2,7 +2,7 @@ ## 1. Overview -The `/v1/code/execute/python` endpoint allows users to execute Python code on the server. This endpoint is part of the `v1` API and is registered in the `app.py` file. It is designed to provide a secure and controlled environment for executing Python code, with features like input validation, timeout handling, and output capturing. +The `/v1/code/execute/python` endpoint allows users to execute Python code on the server. This endpoint is part of the version 1.0 API structure defined in `app.py`. It is designed to provide a secure and controlled environment for executing Python code, with features like input validation, output capturing, and timeout handling. ## 2. Endpoint @@ -24,7 +24,7 @@ The request body must be a JSON object with the following properties: - `webhook_url` (string, optional): The URL to receive the execution result via a webhook. - `id` (string, optional): A unique identifier for the request. -The `validate_payload` decorator in the routes file enforces the following JSON schema for the request body: +The `validate_payload` directive in the routes file enforces the following JSON schema for the request body: ```json { @@ -42,6 +42,8 @@ The `validate_payload` decorator in the routes file enforces the following JSON ### Example Request +**Request Payload:** + ```json { "code": "print('Hello, World!')", @@ -51,6 +53,8 @@ The `validate_payload` decorator in the routes file enforces the following JSON } ``` +**cURL Command:** + ```bash curl -X POST \ -H "x-api-key: YOUR_API_KEY" \ @@ -67,6 +71,7 @@ The success response follows the general response format defined in `app.py`. He ```json { + "endpoint": "/v1/code/execute/python", "code": 200, "id": "unique-request-id", "job_id": "generated-job-id", @@ -77,8 +82,8 @@ The success response follows the general response format defined in `app.py`. He "exit_code": 0 }, "message": "success", - "pid": 1234, - "queue_id": 12345678, + "pid": 12345, + "queue_id": 1234567890, "run_time": 0.123, "queue_time": 0.0, "total_time": 0.123, @@ -89,45 +94,81 @@ The success response follows the general response format defined in `app.py`. He ### Error Responses -- **400 Bad Request**: - - Example: `{"error": "Execution failed", "stdout": "", "exit_code": 1}` - - Reason: The Python code execution failed, or there was an error in the user's code. +#### Missing or Invalid Parameters + +**Status Code:** 400 Bad Request + +```json +{ + "error": "Missing or invalid parameters", + "stdout": "", + "exit_code": 400 +} +``` + +#### Execution Error -- **408 Request Timeout**: - - Example: `{"error": "Execution timed out after 10 seconds"}` - - Reason: The Python code execution exceeded the specified timeout. +**Status Code:** 400 Bad Request + +```json +{ + "error": "Error message from the executed code", + "stdout": "Output from the executed code", + "exit_code": 400 +} +``` -- **500 Internal Server Error**: - - Example: `{"error": "Failed to parse execution result"}` - - Reason: An internal error occurred while parsing the execution result. +#### Execution Timeout + +**Status Code:** 408 Request Timeout + +```json +{ + "error": "Execution timed out after 10 seconds" +} +``` + +#### Internal Server Error + +**Status Code:** 500 Internal Server Error + +```json +{ + "error": "An internal server error occurred", + "stdout": "", + "stderr": "", + "exit_code": 500 +} +``` ## 5. Error Handling -The endpoint handles the following common errors: +The endpoint handles various types of errors, including: -- **Missing or invalid parameters**: If the request body is missing or contains invalid parameters, a `400 Bad Request` error is returned. -- **Execution timeout**: If the Python code execution exceeds the specified timeout, a `408 Request Timeout` error is returned. -- **Internal errors**: If an internal error occurs during code execution or result parsing, a `500 Internal Server Error` is returned. +- Missing or invalid parameters (400 Bad Request) +- Execution errors, such as syntax errors or exceptions (400 Bad Request) +- Execution timeout (408 Request Timeout) +- Internal server errors (500 Internal Server Error) -Additionally, the main application context (`app.py`) includes error handling for queue overload. If the maximum queue length is reached, a `429 Too Many Requests` error is returned. +The main application context (`app.py`) also includes error handling for queue overload (429 Too Many Requests) and other general errors. ## 6. Usage Notes -- The Python code is executed in a sandboxed environment, with limited access to system resources and libraries. -- The execution output (stdout and stderr) is captured and included in the response. -- The return value of the executed code is included in the response if the execution is successful. -- The `webhook_url` parameter is optional and can be used to receive the execution result via a webhook. +- The executed code runs in a sandboxed environment, with limited access to system resources. +- The code execution is limited to a maximum of 300 seconds (5 minutes) by default, but this can be adjusted using the `timeout` parameter. +- The execution result, including stdout, stderr, and the return value, is captured and returned in the response. +- If a `webhook_url` is provided, the execution result will also be sent to the specified webhook. ## 7. Common Issues -- **Syntax errors or exceptions in the user's code**: These will result in a `400 Bad Request` error, with the error message and traceback included in the response. -- **Infinite loops or long-running code**: The execution will be terminated after the specified timeout, resulting in a `408 Request Timeout` error. -- **Excessive resource usage**: The sandboxed environment may impose limits on memory, CPU, or other resources, which could cause the execution to fail. +- Attempting to execute code that accesses restricted resources or performs disallowed operations may result in an execution error. +- Long-running or resource-intensive code may trigger the execution timeout. +- Providing an invalid `webhook_url` will prevent the execution result from being delivered to the specified webhook. ## 8. Best Practices -- **Validate user input**: Always validate and sanitize user input to prevent code injection or other security vulnerabilities. -- **Set appropriate timeouts**: Choose a reasonable timeout value based on the expected complexity of the code to be executed. -- **Monitor resource usage**: Keep an eye on the resource usage of the execution environment to prevent excessive consumption or denial of service attacks. -- **Implement rate limiting**: Consider implementing rate limiting to prevent abuse or overload of the endpoint. -- **Log and monitor errors**: Ensure that errors are properly logged and monitored for debugging and security purposes. \ No newline at end of file +- Always validate and sanitize user input to prevent code injection attacks. +- Set an appropriate timeout value based on the expected execution time of the code. +- Monitor the execution logs for any errors or unexpected behavior. +- Implement rate limiting or queue management to prevent abuse or overload of the endpoint. +- Consider implementing additional security measures, such as code sandboxing or whitelisting/blacklisting certain operations or modules. \ No newline at end of file diff --git a/docs/ffmpeg/ffmpeg_compose.md b/docs/ffmpeg/ffmpeg_compose.md index e82e90e..bd9e786 100644 --- a/docs/ffmpeg/ffmpeg_compose.md +++ b/docs/ffmpeg/ffmpeg_compose.md @@ -2,12 +2,12 @@ ## 1. Overview -The `/v1/ffmpeg/compose` endpoint is a part of the Flask API application and is designed to provide a flexible and powerful way to compose and manipulate media files using FFmpeg. This endpoint allows users to specify input files, filters, and output options, enabling a wide range of media processing tasks such as transcoding, concatenation, and video editing. +The `/v1/ffmpeg/compose` endpoint is a flexible and powerful API that allows users to compose complex FFmpeg commands by providing input files, filters, and output options. This endpoint is part of the version 1.0 API structure, as shown in the `app.py` file. It is designed to handle various media processing tasks, such as video and audio manipulation, transcoding, and more. ## 2. Endpoint -- **URL Path**: `/v1/ffmpeg/compose` -- **HTTP Method**: `POST` +**URL Path:** `/v1/ffmpeg/compose` +**HTTP Method:** `POST` ## 3. Request @@ -21,17 +21,17 @@ The request body should be a JSON object with the following properties: - `inputs` (required, array): An array of input file objects, each containing: - `file_url` (required, string): The URL of the input file. - - `options` (optional, array): An array of input file options, each containing: - - `option` (required, string): The FFmpeg option to apply. + - `options` (optional, array): An array of option objects, each containing: + - `option` (required, string): The FFmpeg option. - `argument` (optional, string, number, or null): The argument for the option. - `filters` (optional, array): An array of filter objects, each containing: - - `filter` (required, string): The FFmpeg filter to apply. -- `outputs` (required, array): An array of output file objects, each containing: - - `options` (required, array): An array of output file options, each containing: - - `option` (required, string): The FFmpeg option to apply. + - `filter` (required, string): The FFmpeg filter. +- `outputs` (required, array): An array of output option objects, each containing: + - `options` (required, array): An array of option objects, each containing: + - `option` (required, string): The FFmpeg option. - `argument` (optional, string, number, or null): The argument for the option. - `global_options` (optional, array): An array of global option objects, each containing: - - `option` (required, string): The FFmpeg global option to apply. + - `option` (required, string): The FFmpeg global option. - `argument` (optional, string, number, or null): The argument for the option. - `metadata` (optional, object): An object specifying which metadata to include in the response, with the following properties: - `thumbnail` (optional, boolean): Whether to include a thumbnail for the output file. @@ -39,7 +39,7 @@ The request body should be a JSON object with the following properties: - `duration` (optional, boolean): Whether to include the duration of the output file. - `bitrate` (optional, boolean): Whether to include the bitrate of the output file. - `encoder` (optional, boolean): Whether to include the encoder used for the output file. -- `webhook_url` (required, string): The URL to send the response to. +- `webhook_url` (required, string): The URL to send the response webhook. - `id` (required, string): A unique identifier for the request. ### Example Request @@ -66,7 +66,7 @@ The request body should be a JSON object with the following properties: ], "filters": [ { - "filter": "scale=640:480" + "filter": "hflip" } ], "outputs": [ @@ -126,7 +126,7 @@ curl -X POST \ ], "filters": [ { - "filter": "scale=640:480" + "filter": "hflip" } ], "outputs": [ @@ -166,23 +166,23 @@ curl -X POST \ The response will be sent to the specified `webhook_url` as a JSON object with the following properties: -- `endpoint` (string): The endpoint that processed the request (`/v1/ffmpeg/compose`). +- `endpoint` (string): The endpoint URL (`/v1/ffmpeg/compose`). - `code` (number): The HTTP status code (200 for success). - `id` (string): The unique identifier for the request. - `job_id` (string): The unique job ID assigned to the request. - `response` (array): An array of output file objects, each containing: - `file_url` (string): The URL of the uploaded output file. - `thumbnail_url` (string, optional): The URL of the uploaded thumbnail, if requested. - - `filesize` (number, optional): The size of the output file in bytes, if requested. - - `duration` (number, optional): The duration of the output file in seconds, if requested. - - `bitrate` (number, optional): The bitrate of the output file in bits per second, if requested. + - `filesize` (number, optional): The file size of the output file, if requested. + - `duration` (number, optional): The duration of the output file, if requested. + - `bitrate` (number, optional): The bitrate of the output file, if requested. - `encoder` (string, optional): The encoder used for the output file, if requested. -- `message` (string): A success message (`"success"`). +- `message` (string): The success message ("success"). - `pid` (number): The process ID of the worker that processed the request. - `queue_id` (number): The ID of the queue used for processing the request. -- `run_time` (number): The time taken to process the request, in seconds. -- `queue_time` (number): The time the request spent in the queue, in seconds. -- `total_time` (number): The total time taken to process the request, including queue time, in seconds. +- `run_time` (number): The time taken to process the request (in seconds). +- `queue_time` (number): The time the request spent in the queue (in seconds). +- `total_time` (number): The total time taken to process the request, including queue time (in seconds). - `queue_length` (number): The current length of the processing queue. - `build_number` (string): The build number of the application. @@ -193,12 +193,18 @@ The response will be sent to the specified `webhook_url` as a JSON object with t - **429 Too Many Requests**: The maximum queue length has been reached. - **500 Internal Server Error**: An unexpected error occurred while processing the request. -Example error response (400 Bad Request): +Example error response: ```json { "code": 400, - "message": "Invalid request payload: 'inputs' is a required property" + "id": "unique-request-id", + "job_id": "job-id", + "message": "Invalid request payload: 'inputs' is a required property", + "pid": 123, + "queue_id": 456, + "queue_length": 0, + "build_number": "1.0.0" } ``` @@ -209,32 +215,31 @@ The API handles various types of errors, including: - **Missing or invalid parameters**: If the request payload is missing required parameters or contains invalid data types, a 400 Bad Request error will be returned. - **Authentication failure**: If the provided API key is invalid or missing, a 401 Unauthorized error will be returned. - **Queue limit reached**: If the maximum queue length is reached, a 429 Too Many Requests error will be returned. -- **Unexpected errors**: If an unexpected error occurs during request processing, a 500 Internal Server Error will be returned, along with an error message. +- **Unexpected errors**: If an unexpected error occurs during request processing, a 500 Internal Server Error will be returned. The main application context (`app.py`) includes error handling for the processing queue. If the maximum queue length is set and the queue size reaches that limit, new requests will be rejected with a 429 Too Many Requests error. ## 6. Usage Notes -- The `inputs` array can contain multiple input files, allowing for operations like concatenation or merging. -- The `filters` array allows applying FFmpeg filters to the input files. -- The `outputs` array specifies the output file options, such as codec, bitrate, and resolution. -- The `global_options` array allows setting global FFmpeg options that apply to the entire operation. -- The `metadata` object allows requesting specific metadata to be included in the response, such as thumbnail, file size, duration, bitrate, and encoder information. -- The `webhook_url` parameter is required, and the response will be sent to this URL as a POST request. -- The `id` parameter is a unique identifier for the request, which can be used for tracking and logging purposes. +- The `inputs` array must contain at least one input file object. +- The `outputs` array must contain at least one output option object. +- The `filters` array is optional and can be used to apply FFmpeg filters to the input files. +- The `global_options` array is optional and can be used to specify global FFmpeg options. +- The `metadata` object is optional and can be used to request specific metadata for the output files. +- The `webhook_url` parameter is required and specifies the URL where the response should be sent. +- The `id` parameter is required and should be a unique identifier for the request. ## 7. Common Issues - Providing invalid or malformed input file URLs. - Specifying invalid or unsupported FFmpeg options or filters. -- Attempting to process large or high-resolution files, which may result in long processing times or queue delays. -- Reaching the maximum queue length, which will cause new requests to be rejected. +- Reaching the maximum queue length, resulting in a 429 Too Many Requests error. +- Network or connectivity issues that prevent the response webhook from being delivered. ## 8. Best Practices -- Validate input file URLs and ensure they are accessible before submitting the request. -- Test FFmpeg options and filters with sample files before using them in production. -- Consider transcoding or downscaling large or high-resolution files to reduce processing time and queue delays. -- Monitor the queue length and adjust the maximum queue length as needed to prevent excessive queuing or request rejections. -- Implement retry mechanisms for failed requests or requests that encounter queue limits. -- Use the `id` parameter to track and correlate requests with responses for better logging and debugging. \ No newline at end of file +- Validate input file URLs and ensure they are accessible before sending the request. +- Test your FFmpeg command locally before using the API to ensure it works as expected. +- Monitor the queue length and adjust the maximum queue length as needed to prevent overloading the system. +- Implement retry mechanisms for handling failed webhook deliveries or other transient errors. +- Use unique and descriptive `id` values for each request to aid in troubleshooting and monitoring. \ No newline at end of file diff --git a/docs/image/transform/image_to_video.md b/docs/image/transform/image_to_video.md index 45b1a16..add311a 100644 --- a/docs/image/transform/image_to_video.md +++ b/docs/image/transform/image_to_video.md @@ -1,4 +1,4 @@ -# Image to Video Endpoint Documentation +# Image to Video Endpoint ## 1. Overview @@ -17,18 +17,18 @@ The `/v1/image/transform/video` endpoint is part of the Flask API application an ### Body Parameters -The request body must be a JSON object with the following properties: +The request body must be in JSON format and should include the following parameters: | Parameter | Type | Required | Description | |-------------|--------|----------|--------------------------------------------------------------| | `image_url` | string | Yes | The URL of the image to be converted into a video. | -| `length` | number | No | The desired length of the video in seconds (default: 5). | +| `length` | number | No | The desired length of the video in seconds (default: 5). | | `frame_rate`| integer| No | The frame rate of the output video (default: 30). | | `zoom_speed`| number | No | The speed of the zoom effect (0-100, default: 3). | | `webhook_url`| string| No | The URL to receive a webhook notification upon completion. | -| `id` | string | No | An optional ID to associate with the request. | +| `id` | string | No | An optional identifier for the request. | -The `validate_payload` decorator in the `routes` file enforces the following JSON schema for the request body: +The `validate_payload` decorator in the `routes.v1.image.transform.image_to_video` module enforces the following JSON schema for the request body: ```json { @@ -64,14 +64,14 @@ curl -X POST \ -H "x-api-key: YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{"image_url": "https://example.com/image.jpg", "length": 10, "frame_rate": 24, "zoom_speed": 5, "webhook_url": "https://example.com/webhook", "id": "request-123"}' \ - http://your-api-url/v1/image/transform/video + http://your-api-endpoint/v1/image/transform/video ``` ## 4. Response ### Success Response -Upon successful conversion, the endpoint returns a JSON response with the following structure: +Upon successful processing, the endpoint returns a JSON response with the following structure: ```json { @@ -80,9 +80,9 @@ Upon successful conversion, the endpoint returns a JSON response with the follow "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", "response": "https://cloud-storage.example.com/converted-video.mp4", "message": "success", - "run_time": 5.123, - "queue_time": 0.456, - "total_time": 5.579, + "run_time": 2.345, + "queue_time": 0.123, + "total_time": 2.468, "pid": 12345, "queue_id": 1234567890, "queue_length": 0, @@ -94,59 +94,33 @@ The `response` field contains the URL of the converted video file uploaded to cl ### Error Responses -- **400 Bad Request** +#### 429 Too Many Requests -```json -{ - "code": 400, - "id": "request-123", - "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", - "message": "Invalid request payload", - "pid": 12345, - "queue_id": 1234567890, - "queue_length": 0, - "build_number": "1.0.0" -} -``` - -- **401 Unauthorized** - -```json -{ - "code": 401, - "id": "request-123", - "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", - "message": "Invalid API key", - "pid": 12345, - "queue_id": 1234567890, - "queue_length": 0, - "build_number": "1.0.0" -} -``` - -- **429 Too Many Requests** +If the maximum queue length is reached, the endpoint returns a 429 Too Many Requests response: ```json { "code": 429, "id": "request-123", "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", - "message": "MAX_QUEUE_LENGTH (100) reached", + "message": "MAX_QUEUE_LENGTH (10) reached", "pid": 12345, "queue_id": 1234567890, - "queue_length": 100, + "queue_length": 10, "build_number": "1.0.0" } ``` -- **500 Internal Server Error** +#### 500 Internal Server Error + +If an exception occurs during the image-to-video conversion process, the endpoint returns a 500 Internal Server Error response: ```json { "code": 500, "id": "request-123", "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", - "message": "Error processing image to video: ", + "message": "Error message describing the exception", "pid": 12345, "queue_id": 1234567890, "queue_length": 0, @@ -156,32 +130,30 @@ The `response` field contains the URL of the converted video file uploaded to cl ## 5. Error Handling -- **Missing or invalid parameters**: If the request payload is missing required parameters or contains invalid values, the endpoint returns a 400 Bad Request error. -- **Authentication error**: If the provided `x-api-key` header is missing or invalid, the endpoint returns a 401 Unauthorized error. -- **Queue limit reached**: If the maximum queue length (`MAX_QUEUE_LENGTH`) is set and the queue size reaches that limit, the endpoint returns a 429 Too Many Requests error. -- **Other errors**: Any other exceptions raised during the image-to-video conversion process will result in a 500 Internal Server Error response, with the error message included in the response body. +The endpoint handles the following types of errors: + +- **Missing or invalid parameters**: If the request body is missing required parameters or contains invalid parameter values, the `validate_payload` decorator will return a 400 Bad Request response with a descriptive error message. +- **Queue length exceeded**: If the maximum queue length is reached and the `bypass_queue` parameter is set to `False`, the endpoint returns a 429 Too Many Requests response. +- **Exceptions during processing**: If an exception occurs during the image-to-video conversion process, the endpoint returns a 500 Internal Server Error response with the error message. ## 6. Usage Notes - The `image_url` parameter must be a valid URL pointing to an image file. - The `length` parameter specifies the duration of the output video in seconds and must be between 1 and 60. -- The `frame_rate` parameter determines the frame rate of the output video and must be between 15 and 60 frames per second. -- The `zoom_speed` parameter controls the speed of the zoom effect applied to the image during the video conversion. It is a value between 0 and 100, where 0 means no zoom, and 100 is the maximum zoom speed. -- If the `webhook_url` parameter is provided, a webhook notification will be sent to the specified URL upon completion of the conversion process. -- The `id` parameter is an optional identifier that can be associated with the request for tracking purposes. +- The `frame_rate` parameter specifies the frame rate of the output video and must be between 15 and 60. +- The `zoom_speed` parameter controls the speed of the zoom effect and must be between 0 and 100. +- The `webhook_url` parameter is optional and can be used to receive a notification when the conversion is complete. +- The `id` parameter is optional and can be used to identify the request. ## 7. Common Issues -- Providing an invalid or inaccessible `image_url`. -- Setting the `length` parameter to an extremely high value, which may result in long processing times or resource exhaustion. -- Specifying an unsupported image format. -- Network or connectivity issues that may cause the image download or video upload to fail. +- Providing an invalid or inaccessible `image_url` will result in an error during processing. +- Specifying invalid parameter values outside the allowed ranges will result in a 400 Bad Request response. +- If the maximum queue length is reached and the `bypass_queue` parameter is set to `False`, the request will be rejected with a 429 Too Many Requests response. ## 8. Best Practices -- Validate the `image_url` parameter before submitting the request to ensure it points to a valid and accessible image file. -- Set reasonable values for the `length` and `frame_rate` parameters based on your requirements and available resources. -- Consider using the `webhook_url` parameter to receive notifications about the conversion process, especially for long-running or asynchronous requests. -- Implement proper error handling and retry mechanisms in your client application to handle potential failures or network issues. -- Monitor the API logs for any errors or warnings related to the image-to-video conversion process. -- Use the `id` parameter to associate requests with specific users, sessions, or other identifiers for better tracking and debugging. \ No newline at end of file +- Validate the `image_url` parameter before sending the request to ensure it points to a valid and accessible image file. +- Use the `webhook_url` parameter to receive notifications about the completion of the conversion process, rather than polling the API repeatedly. +- Provide the `id` parameter to easily identify and track the request in logs or notifications. +- Consider setting the `bypass_queue` parameter to `True` for time-sensitive requests to bypass the queue and process the request immediately. \ No newline at end of file diff --git a/docs/media/media_transcribe.md b/docs/media/media_transcribe.md index 6f61b5e..2fc3bf2 100644 --- a/docs/media/media_transcribe.md +++ b/docs/media/media_transcribe.md @@ -1,22 +1,260 @@ -# Media Transcription Endpoint +# Media Transcription API Documentation -## 1. Overview +## Overview +The Media Transcription endpoint is part of the v1 API suite, providing audio/video transcription and translation capabilities. This endpoint leverages a queuing system for handling long-running transcription tasks, with webhook support for asynchronous processing. It's integrated into the main Flask application as a Blueprint and supports both direct response and cloud storage options for the transcription results. -The `/v1/media/transcribe` endpoint is part of the Flask API application and is responsible for transcribing media files (audio or video) into text format. It supports various options such as generating SRT subtitles, including word-level timestamps, and translating the transcription into different languages. The endpoint can return the transcription results directly or upload them to cloud storage and provide the URLs. +## Endpoint +- **URL**: `/v1/media/transcribe` +- **Method**: `POST` +- **Blueprint**: `v1_media_transcribe_bp` -## 2. Endpoint +## Request -- **URL Path**: `/v1/media/transcribe` -- **HTTP Method**: `POST` +### Headers +- `x-api-key`: Required. Authentication key for API access. +- `Content-Type`: Required. Must be `application/json`. -## 3. Request +### Body Parameters -### Headers +#### Required Parameters +- `media_url` (string) + - Format: URI + - Description: URL of the media file to be transcribed -- `x-api-key` (required): The API key for authentication. +#### Optional Parameters +- `task` (string) + - Allowed values: `"transcribe"`, `"translate"` + - Default: `"transcribe"` + - Description: Specifies whether to transcribe or translate the audio + +- `include_text` (boolean) + - Default: `true` + - Description: Include plain text transcription in the response + +- `include_srt` (boolean) + - Default: `false` + - Description: Include SRT format subtitles in the response + +- `include_segments` (boolean) + - Default: `false` + - Description: Include timestamped segments in the response + +- `word_timestamps` (boolean) + - Default: `false` + - Description: Include timestamps for individual words + +- `response_type` (string) + - Allowed values: `"direct"`, `"cloud"` + - Default: `"direct"` + - Description: Whether to return results directly or as cloud storage URLs + +- `language` (string) + - Optional + - Description: Source language code for transcription + +- `webhook_url` (string) + - Format: URI + - Description: URL to receive the transcription results asynchronously + +- `id` (string) + - Description: Custom identifier for the transcription job -### Body Parameters +### Example Request + +```bash +curl -X POST "https://api.example.com/v1/media/transcribe" \ + -H "x-api-key: your_api_key" \ + -H "Content-Type: application/json" \ + -d '{ + "media_url": "https://example.com/media/file.mp3", + "task": "transcribe", + "include_text": true, + "include_srt": true, + "include_segments": true, + "response_type": "cloud", + "webhook_url": "https://your-webhook.com/callback", + "id": "custom-job-123" + }' +``` + +## Response + +### Immediate Response (202 Accepted) +When a webhook URL is provided, the API returns an immediate acknowledgment: + +```json +{ + "code": 202, + "id": "custom-job-123", + "job_id": "550e8400-e29b-41d4-a716-446655440000", + "message": "processing", + "pid": 12345, + "queue_id": 67890, + "max_queue_length": "unlimited", + "queue_length": 1, + "build_number": "1.0.0" +} +``` + +### Success Response (via Webhook) +For direct response_type: + +```json +{ + "endpoint": "/v1/transcribe/media", + "code": 200, + "id": "custom-job-123", + "job_id": "550e8400-e29b-41d4-a716-446655440000", + "response": { + "text": "Transcribed text content...", + "srt": "SRT formatted content...", + "segments": [...], + "text_url": null, + "srt_url": null, + "segments_url": null + }, + "message": "success", + "pid": 12345, + "queue_id": 67890, + "run_time": 5.234, + "queue_time": 0.123, + "total_time": 5.357, + "queue_length": 0, + "build_number": "1.0.0" +} +``` + +For cloud response_type: + +```json +{ + "endpoint": "/v1/transcribe/media", + "code": 200, + "id": "custom-job-123", + "job_id": "550e8400-e29b-41d4-a716-446655440000", + "response": { + "text": null, + "srt": null, + "segments": null, + "text_url": "https://storage.example.com/text.txt", + "srt_url": "https://storage.example.com/subtitles.srt", + "segments_url": "https://storage.example.com/segments.json" + }, + "message": "success", + "pid": 12345, + "queue_id": 67890, + "run_time": 5.234, + "queue_time": 0.123, + "total_time": 5.357, + "queue_length": 0, + "build_number": "1.0.0" +} +``` + +### Error Responses + +#### Queue Full (429 Too Many Requests) +```json +{ + "code": 429, + "id": "custom-job-123", + "job_id": "550e8400-e29b-41d4-a716-446655440000", + "message": "MAX_QUEUE_LENGTH (100) reached", + "pid": 12345, + "queue_id": 67890, + "queue_length": 100, + "build_number": "1.0.0" +} +``` + +#### Server Error (500 Internal Server Error) +```json +{ + "endpoint": "/v1/transcribe/media", + "code": 500, + "id": "custom-job-123", + "job_id": "550e8400-e29b-41d4-a716-446655440000", + "response": null, + "message": "Error message details", + "pid": 12345, + "queue_id": 67890, + "run_time": 0.123, + "queue_time": 0.056, + "total_time": 0.179, + "queue_length": 1, + "build_number": "1.0.0" +} +``` + +## Error Handling + +### Common Errors +- **Invalid API Key**: 401 Unauthorized +- **Invalid JSON Payload**: 400 Bad Request +- **Missing Required Fields**: 400 Bad Request +- **Invalid media_url**: 400 Bad Request +- **Queue Full**: 429 Too Many Requests +- **Processing Error**: 500 Internal Server Error + +### Validation Errors +The endpoint performs strict validation of the request payload using JSON Schema. Common validation errors include: +- Invalid URI format for media_url or webhook_url +- Invalid task value (must be "transcribe" or "translate") +- Invalid response_type value (must be "direct" or "cloud") +- Unknown properties in the request body + +## Usage Notes + +1. **Webhook Processing** + - When a webhook_url is provided, the request is processed asynchronously + - The API returns an immediate 202 response with a job_id + - Final results are sent to the webhook_url when processing completes + +2. **Queue Management** + - Requests with webhook_url are queued for processing + - MAX_QUEUE_LENGTH environment variable controls queue size + - Set MAX_QUEUE_LENGTH to 0 for unlimited queue size + +3. **File Management** + - For cloud response_type, temporary files are automatically cleaned up + - Results are uploaded to cloud storage before deletion + - URLs in the response provide access to the stored files + +## Common Issues + +1. **Media Access** + - Ensure media_url is publicly accessible + - Verify media file format is supported + - Check for media file corruption + +2. **Webhook Delivery** + - Ensure webhook_url is publicly accessible + - Implement webhook endpoint retry logic + - Monitor webhook endpoint availability + +3. **Resource Usage** + - Large media files may take significant processing time + - Monitor queue length for production deployments + - Consider implementing request size limits + +## Best Practices + +1. **Request Handling** + - Always provide a unique id for job tracking + - Implement webhook retry logic + - Store job_id for result correlation + +2. **Resource Management** + - Monitor queue length in production + - Implement appropriate timeout handling + - Use cloud response_type for large files -The request body should be a JSON object with the following parameters: +3. **Error Handling** + - Implement comprehensive webhook error handling + - Log job_id with all related operations + - Monitor processing times and error rates -| Parameter | Type | Required | Description \ No newline at end of file +4. **Security** + - Use HTTPS for media_url and webhook_url + - Implement webhook authentication + - Validate media file types before processing \ No newline at end of file diff --git a/docs/media/transform/media_to_mp3.md b/docs/media/transform/media_to_mp3.md index 23ebffd..92630ec 100644 --- a/docs/media/transform/media_to_mp3.md +++ b/docs/media/transform/media_to_mp3.md @@ -2,7 +2,7 @@ ## 1. Overview -The `/v1/media/transform/mp3` endpoint is a part of the API's media transformation functionality. It allows users to convert various media files (audio or video) to MP3 format. This endpoint fits into the overall API structure as a part of the `v1` namespace, which is dedicated to the first version of the API. +The `/v1/media/transform/mp3` endpoint is a part of the API's media transformation functionality. It allows users to convert various media files (audio or video) to MP3 format. This endpoint fits into the overall API structure as a part of the `v1` namespace, which represents the first version of the API. ## 2. Endpoint @@ -21,7 +21,7 @@ POST /v1/media/transform/mp3 - `media_url` (required, string): The URL of the media file to be converted. - `webhook_url` (optional, string): The URL to receive a webhook notification upon completion. - `id` (optional, string): A unique identifier for the request. -- `bitrate` (optional, string): The desired bitrate for the MP3 output, specified in the format `k` (e.g., `128k`). If not provided, defaults to `128k`. +- `bitrate` (optional, string): The desired bitrate for the output MP3 file, in the format `k` (e.g., `128k`). If not provided, defaults to `128k`. The `validate_payload` directive in the routes file enforces the following JSON schema for the request body: @@ -62,28 +62,26 @@ curl -X POST \ ### Success Response -Upon successful conversion, the endpoint returns a JSON response with the following structure: +The success response follows the general response structure defined in `app.py`. Here's an example: ```json { "endpoint": "/v1/media/transform/mp3", "code": 200, "id": "unique-request-id", - "job_id": "generated-job-id", - "response": "https://cloud-storage.com/converted-file.mp3", + "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", + "response": "https://cloud-storage.example.com/converted-file.mp3", "message": "success", "pid": 12345, - "queue_id": 1234567890, - "run_time": 5.123, - "queue_time": 0.456, - "total_time": 5.579, + "queue_id": 6789, + "run_time": 5.234, + "queue_time": 0.123, + "total_time": 5.357, "queue_length": 0, "build_number": "1.0.0" } ``` -The `response` field contains the URL of the converted MP3 file uploaded to cloud storage. - ### Error Responses - **400 Bad Request**: Returned when the request payload is invalid or missing required parameters. @@ -95,7 +93,13 @@ Example error response: ```json { "code": 400, - "message": "Invalid request payload: 'media_url' is a required property" + "id": "unique-request-id", + "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", + "message": "Invalid request payload: 'media_url' is a required property", + "pid": 12345, + "queue_id": 6789, + "queue_length": 0, + "build_number": "1.0.0" } ``` @@ -106,26 +110,27 @@ The endpoint handles the following common errors: - Missing or invalid `media_url` parameter: Returns a 400 Bad Request error. - Invalid `bitrate` parameter: Returns a 400 Bad Request error. - Authentication failure: Returns a 401 Unauthorized error. -- Unexpected exceptions during the conversion process: Returns a 500 Internal Server Error error. +- Unexpected exceptions during the conversion process: Returns a 500 Internal Server Error. -Additionally, the main application context (`app.py`) includes error handling for queue overload. If the maximum queue length is reached, the endpoint returns a 429 Too Many Requests error. +Additionally, the main application context (`app.py`) includes error handling for queue overload. If the maximum queue length is reached, the endpoint will return a 429 Too Many Requests error. ## 6. Usage Notes -- The `media_url` parameter should point to a valid media file (audio or video) that can be processed by the conversion service. -- The `webhook_url` parameter is optional and can be used to receive a notification when the conversion is complete. -- The `id` parameter is optional and can be used to associate the request with a specific identifier. -- The `bitrate` parameter is optional and defaults to `128k` if not provided. +- The `media_url` parameter should point to a valid media file (audio or video) that can be converted to MP3 format. +- If the `webhook_url` parameter is provided, a webhook notification will be sent to the specified URL upon completion of the conversion process. +- The `id` parameter can be used to uniquely identify the request, which can be helpful for tracking and logging purposes. +- The `bitrate` parameter allows you to specify the desired bitrate for the output MP3 file. If not provided, the default bitrate of 128k will be used. ## 7. Common Issues - Providing an invalid or inaccessible `media_url`. -- Specifying an invalid `bitrate` value (e.g., not following the `k` format). -- Reaching the maximum queue length, resulting in a 429 Too Many Requests error. +- Attempting to convert unsupported media formats. +- Exceeding the maximum queue length, resulting in a 429 Too Many Requests error. ## 8. Best Practices - Validate the `media_url` parameter before sending the request to ensure it points to a valid and accessible media file. -- Use the `id` parameter to associate requests with specific identifiers for better tracking and debugging. -- Consider providing a `webhook_url` to receive notifications about the conversion status and the resulting file URL. -- Monitor the queue length and adjust the `MAX_QUEUE_LENGTH` environment variable accordingly to prevent overloading the system. \ No newline at end of file +- Consider providing a `webhook_url` parameter to receive notifications about the conversion process completion. +- Use a unique `id` parameter for each request to facilitate tracking and logging. +- Implement retry mechanisms in case of transient errors or queue overload situations. +- Monitor the API logs for any errors or issues during the conversion process. \ No newline at end of file diff --git a/docs/toolkit/authenticate.md b/docs/toolkit/authenticate.md index dec3e3b..0bfcfb5 100644 --- a/docs/toolkit/authenticate.md +++ b/docs/toolkit/authenticate.md @@ -1,19 +1,19 @@ -# Authenticate Endpoint Documentation +# Authenticate Endpoint ## 1. Overview -The `/v1/toolkit/authenticate` endpoint is a part of the `v1_toolkit_auth` blueprint in the Flask application. Its purpose is to authenticate requests by verifying the provided API key against a predefined value. This endpoint serves as a gatekeeper for accessing other protected endpoints within the API. +The `/v1/toolkit/authenticate` endpoint is a part of the `v1_toolkit_auth` blueprint in the API structure. Its purpose is to authenticate requests by verifying the provided API key against a predefined value. This endpoint serves as a gatekeeper, ensuring that only authorized clients can access the API's resources. ## 2. Endpoint -- **URL Path**: `/v1/toolkit/authenticate` -- **HTTP Method**: `GET` +- URL Path: `/v1/toolkit/authenticate` +- HTTP Method: `GET` ## 3. Request ### Headers -- `X-API-Key` (required): The API key to be verified for authentication. +- `X-API-Key` (required): The API key used for authentication. ### Body Parameters @@ -29,7 +29,7 @@ curl -X GET -H "X-API-Key: YOUR_API_KEY" http://localhost:8080/v1/toolkit/authen ### Success Response -If the provided API key matches the predefined value (`API_KEY` environment variable), the endpoint will return a 200 OK status code with the following response: +If the provided API key matches the predefined value, the endpoint will return a 200 OK status code with the following response: ```json { @@ -44,13 +44,14 @@ If the provided API key matches the predefined value (`API_KEY` environment vari "response": "Authorized", "run_time": 0.001, "total_time": 0.001, + "queue_time": 0, "build_number": "1.0.0" } ``` ### Error Responses -If the provided API key does not match the predefined value, the endpoint will return a 401 Unauthorized status code with the following response: +If the provided API key is invalid or missing, the endpoint will return a 401 Unauthorized status code with the following response: ```json { @@ -65,27 +66,27 @@ If the provided API key does not match the predefined value, the endpoint will r "response": null, "run_time": 0.001, "total_time": 0.001, + "queue_time": 0, "build_number": "1.0.0" } ``` ## 5. Error Handling -The main error that can occur with this endpoint is providing an invalid or missing API key. In such cases, the endpoint will return a 401 Unauthorized status code with an appropriate error message. +The main error that can occur with this endpoint is providing an invalid or missing API key. In this case, the endpoint will return a 401 Unauthorized status code with an appropriate error message. ## 6. Usage Notes -- This endpoint is designed to be used as a gatekeeper for other protected endpoints within the API. -- The API key must be provided in the `X-API-Key` header for every request to this endpoint. -- The API key is typically a secret value that should be kept secure and not shared publicly. +- This endpoint is designed to be used as a gatekeeper for the API, ensuring that only authorized clients can access the API's resources. +- The API key should be kept secure and should not be shared with unauthorized parties. ## 7. Common Issues - Forgetting to include the `X-API-Key` header in the request. -- Providing an incorrect or invalid API key. +- Using an invalid or expired API key. ## 8. Best Practices -- Keep the API key secure and avoid storing it in plaintext or committing it to version control systems. -- Consider implementing additional security measures, such as rate limiting or IP whitelisting, to further protect the authentication endpoint. -- Regularly rotate or update the API key to enhance security. \ No newline at end of file +- Rotate API keys periodically to enhance security. +- Store API keys securely and avoid committing them to version control systems. +- Consider implementing additional security measures, such as rate limiting or IP whitelisting, to further protect the API. \ No newline at end of file diff --git a/docs/toolkit/test.md b/docs/toolkit/test.md index b661a72..a304273 100644 --- a/docs/toolkit/test.md +++ b/docs/toolkit/test.md @@ -2,7 +2,7 @@ ## 1. Overview -The `/v1/toolkit/test` endpoint is a part of the NCA Toolkit API and is designed to test the API setup. It creates a temporary file, uploads it to cloud storage, and then returns the upload URL. This endpoint serves as a simple test to ensure that the API is properly configured and functioning correctly. +The `/v1/toolkit/test` endpoint is a part of the NCA Toolkit API and is designed to test the API setup. It creates a temporary file, uploads it to cloud storage, and then returns the upload URL. This endpoint serves as a simple test to ensure that the API is properly configured and can perform basic file operations and cloud storage interactions. ## 2. Endpoint @@ -23,8 +23,8 @@ This endpoint does not require any request body parameters. ```bash curl -X GET \ - https://api.example.com/v1/toolkit/test \ - -H 'x-api-key: YOUR_API_KEY' + https://your-api-url.com/v1/toolkit/test \ + -H 'x-api-key: your-api-key' ``` ## 4. Response @@ -37,10 +37,10 @@ curl -X GET \ "code": 200, "id": null, "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", - "response": "https://storage.example.com/success.txt", + "response": "https://cloud-storage.com/success.txt", "message": "success", "pid": 12345, - "queue_id": 1234567890, + "queue_id": 67890, "run_time": 0.123, "queue_time": 0.0, "total_time": 0.123, @@ -56,7 +56,7 @@ curl -X GET \ ```json { "code": 401, - "message": "Unauthorized" + "message": "Unauthorized: Invalid or missing API key" } ``` @@ -65,32 +65,25 @@ curl -X GET \ ```json { "code": 500, - "id": null, - "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", - "message": "Error message", - "pid": 12345, - "queue_id": 1234567890, - "queue_length": 0, - "build_number": "1.0.0" + "message": "An error occurred while processing the request" } ``` ## 5. Error Handling -- **401 Unauthorized**: This error occurs when the provided `x-api-key` is missing or invalid. -- **500 Internal Server Error**: This error occurs when an unexpected exception is raised during the execution of the endpoint. +- **Missing or Invalid API Key (401 Unauthorized)**: If the `x-api-key` header is missing or invalid, the API will return a 401 Unauthorized error. +- **Internal Server Error (500)**: If an unexpected error occurs during the file creation, upload, or any other operation, the API will return a 500 Internal Server Error with the error message. ## 6. Usage Notes -This endpoint is primarily used for testing purposes and does not require any specific input parameters. It can be called to verify that the API is set up correctly and can successfully upload files to cloud storage. +This endpoint is primarily used for testing purposes and does not require any specific input parameters. It can be called to verify that the API is set up correctly and can perform basic operations. ## 7. Common Issues -- Incorrect or missing `x-api-key` header. -- Temporary file creation or upload failures due to permissions or storage issues. +- Incorrect or missing API key: Ensure that the `x-api-key` header is included with a valid API key. +- Temporary file creation or upload issues: If there are any issues with creating or uploading the temporary file, the API will return an error. ## 8. Best Practices -- Use this endpoint as a simple health check to ensure the API is functioning correctly before attempting more complex operations. -- Regularly test the API setup, especially after updates or changes to the environment. -- Monitor the API logs for any errors or exceptions that may occur during the test. \ No newline at end of file +- Use this endpoint during the initial setup and testing phase of the API integration to ensure that the API is configured correctly. +- Regularly test the API setup using this endpoint to catch any potential issues or configuration changes that may affect the API's functionality. \ No newline at end of file diff --git a/docs/video/caption_video.md b/docs/video/caption_video.md index 43f1431..c295a13 100644 --- a/docs/video/caption_video.md +++ b/docs/video/caption_video.md @@ -17,7 +17,7 @@ The `/v1/video/caption` endpoint is part of the Video API and is responsible for ### Body Parameters -The request body should be a JSON object with the following properties: +The request body must be a JSON object with the following properties: - `video_url` (string, required): The URL of the video file to be captioned. - `captions` (string, optional): The caption text to be added to the video. @@ -67,7 +67,7 @@ The request body should be a JSON object with the following properties: "angle": {"type": "integer"}, "shadow_offset": {"type": "integer"} }, - "additionalProperties": False + "additionalProperties": false } ``` @@ -113,44 +113,44 @@ The request body should be a JSON object with the following properties: ```bash curl -X POST \ - https://api.example.com/v1/video/caption \ - -H 'x-api-key: YOUR_API_KEY' \ - -H 'Content-Type: application/json' \ - -d '{ - "video_url": "https://example.com/video.mp4", - "captions": "This is a sample caption text.", - "settings": { - "line_color": "#FFFFFF", - "word_color": "#000000", - "outline_color": "#000000", - "all_caps": false, - "max_words_per_line": 10, - "x": 20, - "y": 40, - "position": "bottom_left", - "alignment": "left", - "font_family": "Arial", - "font_size": 24, - "bold": false, - "italic": false, - "underline": false, - "strikeout": false, - "style": "classic", - "outline_width": 2, - "spacing": 2, - "angle": 0, - "shadow_offset": 2 - }, - "replace": [ - { - "find": "sample", - "replace": "example" - } - ], - "webhook_url": "https://example.com/webhook", - "id": "request-123", - "language": "en" -}' + -H "x-api-key: YOUR_API_KEY" \ + -H "Content-Type: application/json" \ + -d '{ + "video_url": "https://example.com/video.mp4", + "captions": "This is a sample caption text.", + "settings": { + "line_color": "#FFFFFF", + "word_color": "#000000", + "outline_color": "#000000", + "all_caps": false, + "max_words_per_line": 10, + "x": 20, + "y": 40, + "position": "bottom_left", + "alignment": "left", + "font_family": "Arial", + "font_size": 24, + "bold": false, + "italic": false, + "underline": false, + "strikeout": false, + "style": "classic", + "outline_width": 2, + "spacing": 2, + "angle": 0, + "shadow_offset": 2 + }, + "replace": [ + { + "find": "sample", + "replace": "example" + } + ], + "webhook_url": "https://example.com/webhook", + "id": "request-123", + "language": "en" + }' \ + https://your-api-endpoint.com/v1/video/caption ``` ## 4. Response @@ -168,19 +168,21 @@ The response will be a JSON object with the following properties: - `queue_id` (integer): The ID of the queue used for processing the request. - `run_time` (float): The time taken to process the request (in seconds). - `queue_time` (float): The time the request spent in the queue (in seconds). -- `total_time` (float): The total time taken to process the request, including queue time (in seconds). +- `total_time` (float): The total time taken for the request (in seconds). - `queue_length` (integer): The current length of the processing queue. - `build_number` (string): The build number of the application. +Example: + ```json { "code": 200, "id": "request-123", "job_id": "d290f1ee-6c54-4b01-90e6-d701748f0851", - "response": "https://storage.example.com/captioned-video.mp4", + "response": "https://cloud.example.com/captioned-video.mp4", "message": "success", - "pid": 123, - "queue_id": 140567495862272, + "pid": 12345, + "queue_id": 140682639937472, "run_time": 5.234, "queue_time": 0.012, "total_time": 5.246, @@ -191,70 +193,51 @@ The response will be a JSON object with the following properties: ### Error Responses -#### 400 Bad Request +#### Missing or Invalid Parameters -Returned when there is an issue with the request payload, such as missing or invalid parameters. +**Status Code:** 400 Bad Request ```json { "code": 400, "id": "request-123", "job_id": "d290f1ee-6c54-4b01-90e6-d701748f0851", - "message": "Invalid request payload", - "pid": 123, - "queue_id": 140567495862272, + "message": "Missing or invalid parameters", + "pid": 12345, + "queue_id": 140682639937472, "queue_length": 0, "build_number": "1.0.0" } ``` -#### 400 Font Error +#### Font Error -Returned when there is an issue with the requested font family. +**Status Code:** 400 Bad Request ```json { - "error": "The requested font family 'InvalidFont' is not available. Please choose from the available fonts.", - "available_fonts": ["Arial", "Times New Roman", "Courier New", ...], "code": 400, - "id": "request-123", - "job_id": "d290f1ee-6c54-4b01-90e6-d701748f0851", - "pid": 123, - "queue_id": 140567495862272, + "error": "The requested font 'InvalidFont' is not available. Please choose from the available fonts.", + "available_fonts": ["Arial", "Times New Roman", "Courier New", ...], + "pid": 12345, + "queue_id": 140682639937472, "queue_length": 0, "build_number": "1.0.0" } ``` -#### 429 Too Many Requests - -Returned when the processing queue has reached its maximum length. - -```json -{ - "code": 429, - "id": "request-123", - "job_id": "d290f1ee-6c54-4b01-90e6-d701748f0851", - "message": "MAX_QUEUE_LENGTH (100) reached", - "pid": 123, - "queue_id": 140567495862272, - "queue_length": 100, - "build_number": "1.0.0" -} -``` - -#### 500 Internal Server Error +#### Internal Server Error -Returned when an unexpected error occurs during the captioning process. +**Status Code:** 500 Internal Server Error ```json { "code": 500, "id": "request-123", "job_id": "d290f1ee-6c54-4b01-90e6-d701748f0851", - "message": "An unexpected error occurred during the captioning process.", - "pid": 123, - "queue_id": 140567495862272, + "error": "An unexpected error occurred during the captioning process.", + "pid": 12345, + "queue_id": 140682639937472, "queue_length": 0, "build_number": "1.0.0" } @@ -264,30 +247,32 @@ Returned when an unexpected error occurs during the captioning process. The endpoint handles the following common errors: -- **Missing or invalid parameters**: If any required parameters are missing or invalid, a 400 Bad Request error is returned. -- **Font error**: If the requested font family is not available, a 400 Bad Request error is returned, along with a list of available fonts. -- **Queue limit reached**: If the processing queue has reached its maximum length (determined by the `MAX_QUEUE_LENGTH` environment variable), a 429 Too Many Requests error is returned. -- **Unexpected errors**: If an unexpected error occurs during the captioning process, a 500 Internal Server Error is returned. +- **Missing or Invalid Parameters**: If any required parameters are missing or invalid, a 400 Bad Request error is returned with a descriptive error message. +- **Font Error**: If the requested font is not available, a 400 Bad Request error is returned with a list of available fonts. +- **Internal Server Error**: If an unexpected error occurs during the captioning process, a 500 Internal Server Error is returned with an error message. + +Additionally, the main application context (`app.py`) includes error handling for queue overload. If the maximum queue length (`MAX_QUEUE_LENGTH`) is set and the queue size reaches that limit, a 429 Too Many Requests error is returned with a descriptive message. ## 6. Usage Notes -- The `video_url` parameter is required, and it should be a valid URL pointing to a video file. -- The `captions` parameter is optional. If not provided, the video will be processed without captions. -- The `settings` parameter allows you to customize the appearance and behavior of the captions. Refer to the settings schema for available options. -- The `replace` parameter allows you to specify text replacements to be made in the captions. -- The `webhook_url` parameter is optional. If provided, a webhook notification will be sent to the specified URL when the captioning process is complete. +- The `video_url` parameter must be a valid URL pointing to a video file. +- The `captions` parameter is optional. If not provided, the video will be returned without captions. +- The `settings` parameter allows for customization of the caption appearance and behavior. +- The `replace` parameter can be used to perform text replacements in the captions. +- The `webhook_url` parameter is optional and can be used to receive a notification when the captioning process is complete. - The `id` parameter is optional and can be used to identify the request. -- The `language` parameter is optional and specifies the language of the captions. If not provided, the language will be automatically detected. +- The `language` parameter is optional and can be used to specify the language of the captions. If not provided, the language will be automatically detected. ## 7. Common Issues -- **Invalid video URL**: Ensure that the `video_url` parameter points to a valid and accessible video file. -- **Unsupported video format**: The captioning process may not support certain video formats. If you encounter issues, try converting the video to a more common format like MP4 or AVI. -- **Font availability**: The captioning process may not have access to certain font families. If you encounter a font error, choose from the list of available fonts provided in the error response. +- Providing an invalid or inaccessible `video_url`. +- Requesting an unavailable font in the `settings` object. +- Exceeding the maximum queue length, resulting in a 429 Too Many Requests error. ## 8. Best Practices -- **Validate input**: Always validate the input parameters to ensure they meet the expected format and requirements. -- **Use webhooks**: Utilize the `webhook_url` parameter to receive notifications when the captioning process is complete, rather than polling for the result. -- **Optimize settings**: Adjust the caption settings based on the video resolution and content to ensure readability and aesthetics. -- **Test locally**: Before deploying to production, test the captioning process locally with various video files and settings to identify and resolve any issues. \ No newline at end of file +- Validate the `video_url` parameter before sending the request to ensure it points to a valid and accessible video file. +- Use the `webhook_url` parameter to receive notifications about the captioning process, rather than polling the API for updates. +- Provide descriptive and meaningful `id` values to easily identify requests in logs and responses. +- Use the `replace` parameter judiciously to avoid unintended text replacements in the captions. +- Consider caching the captioned video files for frequently requested videos to improve performance and reduce processing time. \ No newline at end of file diff --git a/docs/video/concatenate.md b/docs/video/concatenate.md index bae4c1c..eb998a7 100644 --- a/docs/video/concatenate.md +++ b/docs/video/concatenate.md @@ -2,7 +2,7 @@ ## 1. Overview -The `/v1/video/concatenate` endpoint is part of the Video API and is responsible for combining multiple video files into a single video file. This endpoint fits into the overall API structure by providing a way to concatenate videos, which can be useful in various scenarios, such as creating video compilations or combining multiple video segments into a single file. +The `/v1/video/concatenate` endpoint is a part of the Video API and is responsible for combining multiple video files into a single video file. This endpoint fits into the overall API structure as a part of the version 1 (v1) routes, specifically under the `/v1/video` namespace. ## 2. Endpoint @@ -20,10 +20,10 @@ The `/v1/video/concatenate` endpoint is part of the Video API and is responsible The request body must be a JSON object with the following properties: - `video_urls` (required, array of objects): An array of video URLs to be concatenated. Each object in the array must have a `video_url` property (string, URI format) containing the URL of the video file. -- `webhook_url` (optional, string, URI format): The URL to receive a webhook notification when the video concatenation process is complete. -- `id` (optional, string): A unique identifier for the request. +- `webhook_url` (optional, string, URI format): The URL to which the response should be sent as a webhook. +- `id` (optional, string): An identifier for the request. -The `validate_payload` decorator in the route file enforces the following JSON schema for the request body: +The `validate_payload` decorator in the routes file enforces the following JSON schema for the request body: ```json { @@ -58,7 +58,7 @@ The `validate_payload` decorator in the route file enforces the following JSON s {"video_url": "https://example.com/video3.mp4"} ], "webhook_url": "https://example.com/webhook", - "id": "unique-request-id" + "id": "request-123" } ``` @@ -73,7 +73,7 @@ curl -X POST \ {"video_url": "https://example.com/video3.mp4"} ], "webhook_url": "https://example.com/webhook", - "id": "unique-request-id" + "id": "request-123" }' \ https://your-api-endpoint.com/v1/video/concatenate ``` @@ -88,12 +88,12 @@ The success response follows the general response format defined in the `app.py` { "endpoint": "/v1/video/concatenate", "code": 200, - "id": "unique-request-id", + "id": "request-123", "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", - "response": "https://cloud-storage.com/combined-video.mp4", + "response": "https://cloud-storage.example.com/combined-video.mp4", "message": "success", "pid": 12345, - "queue_id": 67890, + "queue_id": 6789, "run_time": 10.234, "queue_time": 2.345, "total_time": 12.579, @@ -107,53 +107,74 @@ The `response` field contains the URL of the combined video file uploaded to clo ### Error Responses - **400 Bad Request**: Returned when the request body is missing or invalid. + + ```json + { + "code": 400, + "message": "Invalid request payload" + } + ``` + - **401 Unauthorized**: Returned when the `x-api-key` header is missing or invalid. -- **429 Too Many Requests**: Returned when the maximum queue length is reached (if configured). -- **500 Internal Server Error**: Returned when an unexpected error occurs during the video concatenation process. -Example error response: + ```json + { + "code": 401, + "message": "Unauthorized" + } + ``` -```json -{ - "code": 400, - "id": "unique-request-id", +- **429 Too Many Requests**: Returned when the maximum queue length is reached. + + ```json + { + "code": 429, + "id": "request-123", "job_id": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6", - "message": "Invalid request payload: 'video_urls' is a required property", + "message": "MAX_QUEUE_LENGTH (100) reached", "pid": 12345, - "queue_id": 67890, - "queue_length": 0, + "queue_id": 6789, + "queue_length": 100, "build_number": "1.0.0" -} -``` + } + ``` + +- **500 Internal Server Error**: Returned when an unexpected error occurs during the video concatenation process. + + ```json + { + "code": 500, + "message": "An error occurred during video concatenation" + } + ``` ## 5. Error Handling The endpoint handles the following common errors: -- Missing or invalid request parameters: Returns a 400 Bad Request error with a descriptive error message. -- Authentication failure: Returns a 401 Unauthorized error if the `x-api-key` header is missing or invalid. -- Queue length exceeded: Returns a 429 Too Many Requests error if the maximum queue length is reached (if configured). -- Unexpected exceptions during video concatenation: Returns a 500 Internal Server Error with the exception message. +- **Missing or invalid request body**: If the request body is missing or does not conform to the expected JSON schema, a 400 Bad Request error is returned. +- **Missing or invalid API key**: If the `x-api-key` header is missing or invalid, a 401 Unauthorized error is returned. +- **Queue length exceeded**: If the maximum queue length is reached (determined by the `MAX_QUEUE_LENGTH` environment variable), a 429 Too Many Requests error is returned. +- **Unexpected errors during video concatenation**: If an unexpected error occurs during the video concatenation process, a 500 Internal Server Error is returned with the error message. -The main application context (`app.py`) also includes error handling for queue-related errors, such as reaching the maximum queue length. +The main application context (`app.py`) also includes error handling for the task queue. If the queue length exceeds the `MAX_QUEUE_LENGTH` limit, the request is rejected with a 429 Too Many Requests error. ## 6. Usage Notes +- The video files to be concatenated must be accessible via the provided URLs. - The order of the video files in the `video_urls` array determines the order in which they will be concatenated. -- The endpoint supports various video file formats, but the specific supported formats may depend on the underlying video processing library (e.g., FFmpeg). -- The combined video file will be uploaded to cloud storage, and the response will include the URL of the uploaded file. -- If a `webhook_url` is provided, a webhook notification will be sent to that URL when the video concatenation process is complete. +- If the `webhook_url` parameter is provided, the response will be sent as a webhook to the specified URL. +- The `id` parameter can be used to identify the request in the response. ## 7. Common Issues -- Providing invalid or inaccessible video URLs in the `video_urls` array. -- Exceeding the maximum queue length, if configured, which can result in a 429 Too Many Requests error. -- Encountering issues during the video concatenation process, such as unsupported video formats or corrupted video files. +- Providing invalid or inaccessible video URLs. +- Exceeding the maximum queue length, which can lead to requests being rejected with a 429 Too Many Requests error. +- Encountering unexpected errors during the video concatenation process, which can result in a 500 Internal Server Error. ## 8. Best Practices -- Validate the video URLs before sending the request to ensure they are accessible and in a supported format. -- Monitor the queue length and adjust the maximum queue length as needed to prevent overloading the system. -- Implement retry mechanisms for failed requests or consider using a message queue system for better reliability and scalability. -- Implement proper error handling and logging to aid in troubleshooting and monitoring. -- Consider implementing rate limiting or throttling mechanisms to prevent abuse or excessive load on the system. \ No newline at end of file +- Validate the video URLs before sending the request to ensure they are accessible and in the correct format. +- Monitor the queue length and adjust the `MAX_QUEUE_LENGTH` value accordingly to prevent requests from being rejected due to a full queue. +- Implement retry mechanisms for handling temporary errors or failures during the video concatenation process. +- Provide meaningful and descriptive `id` values to easily identify requests in the response. \ No newline at end of file diff --git a/requirements.txt b/requirements.txt index 0aef430..1858ee7 100644 --- a/requirements.txt +++ b/requirements.txt @@ -4,10 +4,8 @@ requests ffmpeg-python openai-whisper gunicorn -APScheduler srt numpy -torch google-auth google-auth-oauthlib google-auth-httplib2 @@ -15,5 +13,4 @@ google-api-python-client google-cloud-storage psutil boto3 -Pillow -matplotlib \ No newline at end of file +Pillow \ No newline at end of file diff --git a/routes/v1/media/media_transcribe.py b/routes/v1/media/media_transcribe.py index dce2fe4..4c52b6a 100644 --- a/routes/v1/media/media_transcribe.py +++ b/routes/v1/media/media_transcribe.py @@ -53,7 +53,10 @@ def transcribe(job_id, data): result_json = { "text": result[0], "srt": result[1], - "segments": result[2] + "segments": result[2], + "text_url": None, + "srt_url": None, + "segments_url": None, } return result_json, "/v1/transcribe/media", 200 @@ -61,9 +64,12 @@ def transcribe(job_id, data): else: cloud_urls = { - "text": upload_file(result[0]) if include_text is True else None, - "srt": upload_file(result[1]) if include_srt is True else None, - "segments": upload_file(result[2]) if include_segments is True else None, + "text": None, + "srt": None, + "segments": None, + "text_url": upload_file(result[0]) if include_text is True else None, + "srt_url": upload_file(result[1]) if include_srt is True else None, + "segments_url": upload_file(result[2]) if include_segments is True else None, } if include_text is True: