Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/track docs #208

Merged
merged 5 commits into from
Oct 5, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ following platforms:

- Android, Android TV & FireTV
- iOS & tvOS (Apple TV)
- Web (currently only HTML5 browsers are tested)
- HTML5, Tizen & webOS (web, mobile web, smart TVs, set-top boxes and gaming consoles).

This document covers the creation of a minimal app including a `THEOplayerView` component,
and an overview of the accompanying example app with a user interface provided
Expand Down Expand Up @@ -103,5 +103,5 @@ and discussed in the next section. Finally, an overview of features, limitations
- [Digital Rights Management (DRM)](./doc/drm.md)
- [Migrating to `react-native-theoplayer` v2.x](./doc/migrating_v2.md)
- [Picture-in-Picture (PiP)](./doc/pip.md)
- [Styling subtitles and closed captions](./doc/texttrackstyles.md)
- [Limitations and known issues](./doc/limitations.md)
- [Subtitles, Closed Captions and Metadata tracks](./doc/texttracks.md)
- [Limitations and known issues](./doc/limitations.md)
239 changes: 239 additions & 0 deletions doc/texttracks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,239 @@
## Subtitles, Closed Captions and Metadata tracks

THEOplayer's `TextTrack` api gives developers the capability to manage and manipulate various types of text-based content.
Some of its key functionalities include text track selection, styling, and listening to track and cue events.

### Table of Contents
- [Types of text tracks](#types-of-text-tracks)
- [Side-loaded text tracks](#side-loaded-text-tracks)
- [Listening to text track events](#listening-to-text-track-events)
- [Enabling a text track](#enabling-a-text-track)
- [Preview thumbnails](#preview-thumbnails)
- [Styling subtitles and closed captions](#styling-subtitles-and-closed-captions)

### Types of text tracks

In this document we will differentiate between the following kinds of text tracks:

- **Subtitles**, which are text-based translations of the dialogue or narrative of a video;
- **Closed Captions (CC)**, which go beyond subtitles by providing additional information about sound effects, music, and other audio cues;
- **Metadata tracks**, which are used for providing non-textual information about the video, such as thumbnail urls or program boundaries in a live stream.

In addition, a distinction can be made between how a text track is delivered to the player:

- **In-band** refers to text tracks that are embedded within the video or audio media stream itself.
- **Out-band**, or *out-of-band*, is used to describe text tracks that are delivered separately from the media file. They are typically described in the playlist or manifest.
- **Side-loaded** text tracks are those that are manually added or loaded by the user through the source description.

### Side-loaded text tracks

In contrast to in-band and out-band text tracks, which are delivered to the player through the media content itself,
the side-loaded text tracks can be configured by the user when setting a source to the player:

```typescript
player.source = {
"sources": [
{
"src": "https://cdn.theoplayer.com/video/big_buck_bunny/big_buck_bunny.m3u8",
"type": "application/x-mpegurl"
}
],
"textTracks": [
{
"default": true,
"src": "https://cdn.theoplayer.com/dash/theoplayer/thumbnails/big_buck_bunny_thumbnails.vtt",
"label": "thumbnails",
"kind": "metadata",
"format": "webvtt",
"srclang": "en"
}
]
}
```

#### iOS/tvOS

On iOS and tvOS, support for side-loaded tracks is provided by a separate connector, which is enabled by adding the
feature to the `react-native-theoplayer.json` file in your app folder.

```json
{
"ios": {
"features": ["SIDELOADED_TEXTTRACKS"]
}
}
```

### Listening to text track events

Out-band and side-loaded text tracks become accessible via the text track API once the player has loaded the media
source. This event is signaled by the `PlayerEventType.LOADED_METADATA` event,
which is dispatched by the [THEOplayer API](../src/api/player/THEOplayer.ts):

```typescript
player.addEventListener(PlayerEventType.LOADED_METADATA, (event: LoadedMetadataEvent) => {
console.log(TAG, 'loadedmetadata', JSON.stringify(event));
})
```

Alternatively, the `PlayerEventType.TEXT_TRACK_LIST` event can be used to dynamically listen to text tracks that are being added,
removed or changed. In-band text tracks, in particular, are created when their first cue is loaded, which can occur at any
point during playback.

```typescript
player.addEventListener(PlayerEventType.TEXT_TRACK_LIST, (event: TextTrackListEvent) => {
const { subType, track } = event;
switch (subType) {
case TrackListEventType.ADD_TRACK: /*A text track has been added*/ break;
case TrackListEventType.REMOVE_TRACK: /*A text track has been removed*/ break;
case TrackListEventType.CHANGE_TRACK: /*A text track has changed*/ break;
}
});
```

Similarly for text track cues, the `PlayerEventType.TEXT_TRACK` event provides knowledge on cues
being added or removed.

```typescript
player.addEventListener(PlayerEventType.TEXT_TRACK, (event: TextTrackEvent) => {
const { subType, cue } = event;
switch (subType) {
case TextTrackEventType.ADD_CUE: /*A text track cue has been added to the text track*/ break;
case TextTrackEventType.ENTER_CUE: /*A text track cue has entered; it became active*/ break;
case TextTrackEventType.EXIT_CUE: /*A text track cue has exited; it became inactive*/ break;
case TextTrackEventType.REMOVE_CUE: /*A text track cue has been removed from the text track*/ break;
}
})
```

### Enabling a text track

A subtitle or closed caption can be enabled by setting the player's `selectedTextTrack` property to
the `uid` of the track, or `undefined` to select none.

```typescript
const track: TextTrack;
player.selectedTextTrack = track.uid;
```

Note that [THEOplayer React-Native UI](https://github.com/THEOplayer/react-native-theoplayer-ui) package
provides a visual representation of the list of available text tracks, as well as the ability to enable/disable them.

### Preview thumbnails

Preview thumbnails is a feature that allows users to see a visual representation of different points or scenes within
a video, making it easier to navigate and locate specific content of interest.

The thumbnails are typically contained in a dedicated thumbnail track, which is a text track of kind `metadata` with label
`thumbnails`.
The track can be either be side-loaded to the stream source, or contained in a stream manifest, as the demo
sources in the [example application](./example-app.md) demonstrate:

```json
{
"sources": [
{
"src": "https://cdn.theoplayer.com/video/big_buck_bunny/big_buck_bunny.m3u8",
"type": "application/x-mpegurl"
}
],
"textTracks": [
{
"default": true,
"src": "https://cdn.theoplayer.com/dash/theoplayer/thumbnails/big_buck_bunny_thumbnails.vtt",
"label": "thumbnails",
"kind": "metadata",
"format": "webvtt"
}
]
}
```

The [`@theoplayer/react-native-ui`](https://github.com/THEOplayer/react-native-theoplayer-ui) package provides a
thumbnail viewer the interprets thumbnail cues and displays them above the progress bar.

[//]: # ( TODO: insert picture from react-native-theoplayer-ui)
[//]: # (# ![]())

### Styling subtitles and closed captions

In this section, we showcase the functionality of THEOplayer's `TextTrackStyle` API,
which enables the customization of subtitles and closed captions' appearance. We will illustrate the practical implementation of this API through some examples.

#### Style properties

| Property | Purpose | Value | Support |
|-------------------|-------------------------------------------------------------------|-----------------------------------------------------------------------------------------|:-------------------------------------------------------------------|
| `fontFamily` | The font of the text track. | A string, e.g. `'serif'`, `'monospace'`, `'arial'`. | All platforms, |
| `fontColor` | The font color of the text track. | A string, e.g.`'red'`, `'#ff0000'`, `'#ffc50f'` or `'rgba(255, 0, 0, 1)`. | All platforms. The `rgba()` formatting is supported on Web only. |
| `fontSize` | The font size of the text track. | A percentage string, e.g.`'50%'`, `'100%'` (default), `'175%'`. | All platforms, |
| `backgroundColor` | The color of the area directly behind the text. | A string, e.g.`'red'`, `'#ff0000'`, `'#ffc50f'` or `'rgba(255, 0, 0, 1)`. | All platforms. The `rgba()` formatting is supported on Web only. |
| `windowColor` | The color of the area covering the full width of the text track. | A string, e.g.`'red'`, `'#ff0000'`, `'#ffc50f'` or `'rgba(255, 0, 0, 1)`. | Web and Android. The `rgba()` formatting is supported on Web only. |
| `edgeStyle` | The edge style of the text. | An enum value, either `none` (default), `dropshadow`, `raised`, `depressed`, `uniform`. | All platforms. |
| `marginTop` | The top margin of the area where subtitles are being rendered. | A positive numeric value specifying the number of pixels. | All platforms. |
| `marginLeft` | The left margin of the area where subtitles are being rendered. | A positive numeric value specifying the number of pixels. | All platforms. |
| `marginBottom` | The bottom margin of the area where subtitles are being rendered. | A positive numeric value specifying the number of pixels. | Web only. |
| `marginRight` | The right margin of the area where subtitles are being rendered. | A positive numeric value specifying the number of pixels. | Web only. |

#### Examples

<table>

<tr valign="top">

<td>

![background](./texttrackstyle_background.png)

</td>

<td>

```typescript
player.textTrackStyle.backgroundColor = 'red';
player.textTrackStyle.marginLeft = 45;
```

</td>

</tr>

<tr valign="top">

<td>

![size](./texttrackstyle_size.png)

</td>

<td>

```typescript
player.textTrackStyle.windowColor = 'black';
player.textTrackStyle.fontColor = '#ffff00ff';
player.textTrackStyle.fontSize = '250%';
```

</td>

</tr>

<tr valign="top">

<td>

![size](./texttrackstyle_family.png)

</td>

<td>

```typescript
player.textTrackStyle.fontFamily = 'monospace';
```

</td>

</tr>

</table>
86 changes: 0 additions & 86 deletions doc/texttrackstyles.md

This file was deleted.