From b511afc420ef0131ab554d9e32ae93ccacb146d8 Mon Sep 17 00:00:00 2001 From: szekelyzol Date: Fri, 18 Oct 2024 18:03:24 +0200 Subject: [PATCH 1/5] Add summary error documentation --- reference/navigation.yaml | 2 ++ reference/summary-already-exists.md | 47 +++++++++++++++++++++++++++++ 2 files changed, 49 insertions(+) create mode 100644 reference/summary-already-exists.md diff --git a/reference/navigation.yaml b/reference/navigation.yaml index 755a5cad..ede0001a 100644 --- a/reference/navigation.yaml +++ b/reference/navigation.yaml @@ -113,6 +113,8 @@ href: ./unrecognized-request-url.md - label: Webhook Limit Reached href: ./webhook-limit-reached.md + - label: Summary Already Exists + href: ./summary-already-exists.md - label: Video Ingestion Errors href: ./video-upload-errors.md collapsed: true diff --git a/reference/summary-already-exists.md b/reference/summary-already-exists.md new file mode 100644 index 00000000..a5c39e5c --- /dev/null +++ b/reference/summary-already-exists.md @@ -0,0 +1,47 @@ +--- +title: Summary already exists +meta: + description: This guide explains the cause and the possible solutions for the Summary already exists error. +--- + +# Summary already exists + +A summary already exists for the video that you tried adding a summary to. + +## Sample error response + +```json +{ + "type": "https://docs.api.video/reference/summary-already-exists", + "title": "A summary already exists or is being created on this video.", + "status": 409, + "detail": "You can delete the existing summary and generate a new one.", + "name": "videoId" +} +``` + +## Causes + +This error can occur in 3 scenarios: + +- When using [`POST /summaries`](reference/api/Summaries#generate-video-summary) on an already uploaded video that has a summary, or where summary generation is in progress +- When using [`PATCH /summaries/{summaryId}/source`](reference/api/Summaries#update-summary-details) on a video where a summary source is already generated, or where summary generation is in progress +- When using [`PATCH /videos/{videoId}`](reference/api/Videos#update-a-video-object) on a video where a summary source is already generated, or where summary generation is in progress + +The API does not allow updating already generated summaries. + +## Solution + +To regenerate or fix an incorrect summary for a video, you first need to delete the existing summary: + +```curl +curl -s --location --request DELETE 'https://ws.staging.api.video/summaries/summary_1CGHWuXjhxmeH4WiZ51234/source' \ +--header 'Content-Type: application/json' \ +--header 'Authorization: Basic {Your API key}' \ +``` + +After the API responds with 204 - Deleted, you can repeat your original request to `POST` or `PATCH` the summary. + +## Next steps + +- Learn more about the AI-driven [Summarization](/vod/create-summaries) feature \ No newline at end of file From 29830eb8faac4ca2ca5f694e133f8adb2d8a4155 Mon Sep 17 00:00:00 2001 From: szekelyzol Date: Fri, 18 Oct 2024 20:35:26 +0200 Subject: [PATCH 2/5] Fix wording --- reference/summary-already-exists.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/reference/summary-already-exists.md b/reference/summary-already-exists.md index a5c39e5c..9448b55a 100644 --- a/reference/summary-already-exists.md +++ b/reference/summary-already-exists.md @@ -32,7 +32,7 @@ The API does not allow updating already generated summaries. ## Solution -To regenerate or fix an incorrect summary for a video, you first need to delete the existing summary: +To recreate or fix an incorrect summary for a video, you first need to delete the existing summary: ```curl curl -s --location --request DELETE 'https://ws.staging.api.video/summaries/summary_1CGHWuXjhxmeH4WiZ51234/source' \ From 850a7e3ecd5f89e5d9f896408db538e15006b494 Mon Sep 17 00:00:00 2001 From: szekelyzol Date: Fri, 18 Oct 2024 20:36:00 +0200 Subject: [PATCH 3/5] Fix formatting --- reference/summary-already-exists.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/reference/summary-already-exists.md b/reference/summary-already-exists.md index 9448b55a..ca8ffcee 100644 --- a/reference/summary-already-exists.md +++ b/reference/summary-already-exists.md @@ -40,7 +40,7 @@ curl -s --location --request DELETE 'https://ws.staging.api.video/summaries/summ --header 'Authorization: Basic {Your API key}' \ ``` -After the API responds with 204 - Deleted, you can repeat your original request to `POST` or `PATCH` the summary. +After the API responds with `204 - Deleted`, you can repeat your original request to `POST` or `PATCH` the summary. ## Next steps From 458c4da7c450ed16ec2c1699cea40501a36f87f4 Mon Sep 17 00:00:00 2001 From: szekelyzol Date: Tue, 22 Oct 2024 18:38:07 +0200 Subject: [PATCH 4/5] Add summary feature guide --- reference/summary-already-exists.md | 6 +- vod/create-summaries.md | 151 ++++++++++++++++++++++++++++ vod/generate-transcripts.md | 2 +- vod/navigation.yaml | 2 + 4 files changed, 157 insertions(+), 4 deletions(-) create mode 100644 vod/create-summaries.md diff --git a/reference/summary-already-exists.md b/reference/summary-already-exists.md index ca8ffcee..eee891e6 100644 --- a/reference/summary-already-exists.md +++ b/reference/summary-already-exists.md @@ -24,9 +24,9 @@ A summary already exists for the video that you tried adding a summary to. This error can occur in 3 scenarios: -- When using [`POST /summaries`](reference/api/Summaries#generate-video-summary) on an already uploaded video that has a summary, or where summary generation is in progress -- When using [`PATCH /summaries/{summaryId}/source`](reference/api/Summaries#update-summary-details) on a video where a summary source is already generated, or where summary generation is in progress -- When using [`PATCH /videos/{videoId}`](reference/api/Videos#update-a-video-object) on a video where a summary source is already generated, or where summary generation is in progress +- When using [`POST /summaries`](/reference/api/Summaries#generate-video-summary) on an already uploaded video that has a summary, or where summary generation is in progress +- When using [`PATCH /summaries/{summaryId}/source`](/reference/api/Summaries#update-summary-details) on a video where a summary source is already generated, or where summary generation is in progress +- When using [`PATCH /videos/{videoId}`](/reference/api/Videos#update-a-video-object) on a video where a summary source is already generated, or where summary generation is in progress The API does not allow updating already generated summaries. diff --git a/vod/create-summaries.md b/vod/create-summaries.md new file mode 100644 index 00000000..a52b7b77 --- /dev/null +++ b/vod/create-summaries.md @@ -0,0 +1,151 @@ +--- +title: Create video summaries +meta: + description: This page explains how to generate video summaries manually or automatically using the Videos and Summaries endpoints and the api.video dashboard. +--- + +# Create video summaries + +api.video's an AI-driven summarization feature can generate concise and accurate outlines of your videos. Each summary consist of a title, an abstract, and key takeaways based on the exact contents of a video. All of these are available directly in the player, on your dashboard, and through the API. + +Provide your audience with detailed descriptions and key points of your videos, and also enable more inclusive and accessible content by inviting deaf or hard-of-hearing users! + + + +A summary is based on the transcription of a video. api.video uses [Whisper](https://openai.com/index/whisper/) for multilingual speech recognition and transcripton. We run Whisper's ASR models on our own infrastructure and do not expose data outside our service. You control who gets access to your videos and summaries. + + +## Summary data structure + +Summaries consist of 3 elements: + +- a **title** that clearly notes the topic of the video +- an **abstract** that provides a short outline of the contents of the video +- 3 **takeaways** in chronologial order, highlighting the key points of the video + +Each summary is identified by a `summaryId`. The API separates summary data into `summary` and `source` objects. A `summary` object contains information about the summary: + +```json +{ + "summaryId": "summary_1CGHWuXjhxmeH4WiZ51234", + "createdAt": "2024-07-14T23:36:07+00:00", + "updatedAt": "2024-07-14T23:36:07+00:00", + "videoId": "vilkR8K3N7yrRcxcMt91234", + "origin": "auto", + "sourceStatus": "completed" +} +``` + +The `source` contains the actual title, abstract, and takeaways: + +```json +{ + "title": "A short lecture on quantum theory", + "abstract": "In this lecture, we discuss how complicated quantum theory is, using the famous example of Schrödingers cat. We also discuss practical applications like quantum computing.", + "takeaways": [ + "Quantum theory is complicated.", + "Schrödinger's cat is neither dead, nor alive.", + "Quantum computers are super cool." + ] +} +``` + +Visit the [API reference](/reference/api/Summaries) for detailed explanations of each field and attribute in the `summary` object. + +## How to create summaries + +You have 3 methods to create summaries. You can use: + +- the dashboard to enable summary generation during video upload, and for already uploaded videos +- the `/videos` endpoint for automatic summarization via the API +- the `/summaries` endpoint for more granular control on summary creation via the API + +This guide explains all 3 methods. + +### Generate summaries in the dashboard + +When you use the dashboard to [upload videos](https://dashboard.api.video/videos), simply switch on the **Summary** toggle during the upload process. + + + +A summary is based on the transcription of a video. When you create summaries, we recommend that you also enable transcription. This enables our API to create video summaries faster. + + +For already updated videos, open the **Video Details** page and then the Summarization tab. Click on **Summarize** to generate a summary, or **Summarize again** to recreate a summary. + +### Automatic summarization + +When you use the API to upload videos, the easiest method to enable summary creation when you create a video object. + +To enable summarization, set the **optional** `transcriptSummary` parameter in your `POST` request to the [Create video object endpoint](/reference/api/Videos#create-a-video-object). We recommend that you also enable the **optional** `transcript` and `language` parameters. + +When you define the video `language`, the API creates a summary of the video using the language you specify. Check out the list of supported languages [here](/vod/generate-transcripts#supported-languages). If you do not specify a language for the video, the API will detect it automatically. + +| Field | Type | Description | +|---------------------|-----------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `transcriptSummary` | `boolean` | When `true`, the API generates a summary for the video. The default value is `false`. | +| `transcript` | `boolean` | When `true`, the API generates a transcript for the video. The default value is `false`. | +| `language` | `string` | A valid language identifier using [IETF language tags](https://en.wikipedia.org/wiki/IETF_language_tag). You can use primary subtags like `en` or `fr`.

When the value in your request does not match any covered language, the API returns an error. | + +You can also trigger automatic summarization for already uploaded videos. Use the same parameters in your `PATCH` request to the [Update video object endpoint](https://docs.api.video/reference/api/Videos#update-a-video-object). + +### Manual summarization + +This method enables you to control every step of the summary creation. We recommend that you use this method in scenarios where you want to manually create, update, or edit the generated summary. + +To summarize already uploaded videos manually, use a `POST` request to the [Summaries endpoint](/reference/api/Summaries). You have the option to define the origin of the summary generation with the `origin` request parameter. + +| Field | Type | Description | +|----------|----------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `origin` | `string` | Use this **optional** parameter to define how the API generates the summary. The only allowed value is `auto`, which means that the API generates a summary automatically. | + + + + +If you do not set the `origin` parameter in your request, **the API will not generate a summary automatically**. + +In this case, `sourceStatus` will return `missing`, and you have to manually add the summary source using the `PATCH /summaries/{summaryId}/source` endpoint operation. + + +## Manage summaries + +The API provides dedicated endpoints to list, update, and delete summaries. You can use: + +- the `/summaries` endpoint to [`GET` a list of all summaries](/reference/api/Summaries#list-summaries) in your project, and to filter the list based on `videoId`, `origin`, or `sourceStatus` +- the `/summaries/{summaryId}` endpoint to [`DELETE` a summary](/reference/api/Summaries#delete-video-summary) +- the `/summaries/{summaryId}/source` endpoint to [`GET` the source of a specific summary](/reference/api/Summaries#get-summary-details), and to [`PATCH` an existing summary](/reference/api/Summaries#update-summary-details). + +## Summary generation statuses + +The `summary` object returns the status of summary generation in the `sourceStatus` field. These are the available statuses: + +- `missing`: the source for a summary is not present. +- `waiting` : the source video is being processed and a summary will be generated. +- `failed`: a technical issue prevented summary generation. +- `completed`: the summary is generated. +- `unprocessable`: the source video is unsuitable for summary generation. An example for this is a source video that has no audio, or has less than 50 words. + + + +Using the `PATCH /summaries/{summaryId}/source` endpoint operation is only allowed for summaries where `sourceStatus` is `missing`. The main purpose of this endpoint is to provide a direct way to edit or update summary sources. + +If you want to edit a summary where a source is already present, you first need to [delete the existing summary](/reference/api/Summaries#delete-video-summary). + + +## Supported languages + +The API creates summaries using the same language as for transcripts. Check out the [Generate transcripts guide](/vod/generate-transcripts#supported-languages) for the list of languages + +## Limitations + + + +- The API requires at least 50 spoken words in the video you upload to create a transcript and a summary. +- The length of a summary `abstract` is between 20 to 300 words. The exact length of the abstract depends on the contents of the video and will be roughly 1/8th of the transcript. +- When you set the `language` parameter, make sure that it matches the actual language used in the video. Your setting forces the API to summarize in that language. Mismatching language settings or videos with dialogue in multiple languages can return low quality summaries. + + +## Next steps + +- Learn more about video transcription in the [Generate transcripts guide](/vod/generate-transcripts) +- Check out other AI-driven features like video translation and AI summary [in our blog](https://api.video/blog/product-updates/ai-video-features/) \ No newline at end of file diff --git a/vod/generate-transcripts.md b/vod/generate-transcripts.md index 2abde040..8efe00c4 100644 --- a/vod/generate-transcripts.md +++ b/vod/generate-transcripts.md @@ -4,7 +4,7 @@ meta: description: This page gets you started on how to enable automatic transcription for videos in multiple languages using the Videos endpoint. --- -# Generating video transcripts +# Generate video transcripts api.video's AI-driven transcription feature can generate video transcripts using a single API call. You can now avoid manually uploading captions, as the generated transcripts are available as video captions in the standard WebVTT format. diff --git a/vod/navigation.yaml b/vod/navigation.yaml index 801587bf..843475d7 100644 --- a/vod/navigation.yaml +++ b/vod/navigation.yaml @@ -89,6 +89,8 @@ items: - label: Transcripts href: /vod/generate-transcripts.md + - label: Summaries + href: /vod/create-summaries.md - label: Captions href: /vod/add-captions.md - label: Watermarks From 4a3928ab9d5cdd3b78981df7479bd61afce6e29c Mon Sep 17 00:00:00 2001 From: szekelyzol Date: Tue, 22 Oct 2024 18:51:20 +0200 Subject: [PATCH 5/5] Update API reference nav --- reference/navigation.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/reference/navigation.yaml b/reference/navigation.yaml index ede0001a..afe11fbe 100644 --- a/reference/navigation.yaml +++ b/reference/navigation.yaml @@ -30,6 +30,7 @@ - Videos - Watermarks - Captions + - Summaries - Chapters - Tags