Skip to content

Commit

Permalink
Refactor examples generation, start fixing guides
Browse files Browse the repository at this point in the history
  • Loading branch information
@WojciechBarczynski
WojciechBarczynski committed Aug 14, 2024
1 parent 0ced10f commit 97b649f
Show file tree
Hide file tree
Showing 36 changed files with 641 additions and 867 deletions.
2 changes: 1 addition & 1 deletion docs/pages/api/components/Tiles.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ hide_table_of_contents: true
---
import Docs from "@site/pages/api/generated/component-Tiles.md"
import AbsolutePositionDefinition from "@site/pages/common/absolute-position.md"
import TilesExample from "./tile_transition.webp"
import TilesExample from "./assets/tile_transitions.webp"

# Tiles

Expand Down
Empty file added docs/pages/api/components/assets/tile_transitions.webp
View file
Empty file.
Binary file removed docs/pages/api/components/tile_transition.webp
View file
Binary file not shown.
Empty file added docs/pages/guides/assets/layouts_1.webp
View file
Empty file.
Empty file added docs/pages/guides/assets/layouts_2.webp
View file
Empty file.
Empty file added docs/pages/guides/assets/layouts_3.webp
View file
Empty file.
Empty file added docs/pages/guides/assets/layouts_4.webp
View file
Empty file.
Empty file added docs/pages/guides/assets/layouts_5.webp
View file
Empty file.
Empty file added docs/pages/guides/assets/quick_start_1.webp
View file
Empty file.
Empty file added docs/pages/guides/assets/quick_start_2.webp
View file
Empty file.
Empty file added docs/pages/guides/assets/transitions_1.webp
View file
Empty file.
Empty file added docs/pages/guides/assets/transitions_2.webp
View file
Empty file.
Empty file added docs/pages/guides/assets/transitions_3.webp
View file
Empty file.
Empty file added docs/pages/guides/assets/transitions_4.webp
View file
Empty file.
255 changes: 55 additions & 200 deletions docs/pages/guides/simple-scene.md → docs/pages/guides/layouts.md
View file
Original file line number Diff line number Diff line change
@@ -1,205 +1,24 @@
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import SimpleSceneEmpty from "./simple_scene_1.webp"
import SimpleSceneOverflow from "./simple_scene_2.webp"
import SimpleSceneFitted from "./simple_scene_3.webp"
import SimpleSceneBothInputs from "./simple_scene_4.webp"
import SimpleSceneAbsolutePosition from "./simple_scene_5.webp"
import LayoutsEmpty from "./assets/layouts_1.webp"
import LayoutsOverflow from "./assets/layouts_2.webp"
import LayoutsFitted from "./assets/layouts_3.webp"
import LayoutsBothInputs from "./assets/layouts_4.webp"
import LayoutsAbsolutePosition from "./assets/layouts_5.webp"

# Simple scene
# Layouts

This guide will explain how to create simple scene that is combining input streams in a simple layout into single output stream.
This guide will explain how to combine input streams in a simple layout and mix audio into single output stream.

## Configure inputs and output

### Start the compositor

<Tabs queryString="lang">
<TabItem value="http" label="HTTP">
[Start the compositor server](../deployment/overview.md). Check out [configuration page](../deployment/configuration.md) for available configuration options.
</TabItem>
<TabItem value="membrane" label="Membrane Framework">
Following code snippets are implementing `handle_init/2` or `handle_setup/2` callbacks. Those
are just examples, you can use any [`Membrane.Pipeline` callbacks](https://hexdocs.pm/membrane_core/Membrane.Pipeline.html#callbacks)
instead.

```elixir
alias Membrane.LiveCompositor

def handle_init(ctx, opts) do
spec = [
...

child(:live_compositor, %LiveCompositor{
framerate: {30, 1}
}),

...
]
{[spec: spec], %{}}
end
```
</TabItem>
</Tabs>

## Update output to show an input

### Register input stream `input_1`.

<Tabs queryString="lang">
<TabItem value="http" label="HTTP">
```http
POST: /api/input/input_1/register
Content-Type: application/json

{
"type": "rtp_stream",
"transport_protocol": "tcp_server",
"port": 9001,
"video": {
"decoder": "ffmpeg_h264"
}
}
```

After receiving the response you can establish the connection and start sending the stream. Check out [how to deliver input streams](./deliver-input.md) to learn more.

In this example we are using RTP over TCP, but it could be easily replaced by UDP.
</TabItem>
<TabItem value="membrane" label="Membrane Framework">
```elixir
def handle_init(ctx, opts) do
spec = [
...

input_1_spec
|> via_in(Pad.ref(:video_input, "input_1"))
|> get_child(:live_compositor),

...
]
{[spec: spec], %{}}
end
```
where `input_1_spec` is an element that produces H264 video.
</TabItem>
</Tabs>

### Register input stream `input_2`.

<Tabs queryString="lang">
<TabItem value="http" label="HTTP">
```http
POST: /api/input/input_2/register
Content-Type: application/json

{
"type": "rtp_stream",
"transport_protocol": "tcp_server",
"port": 9002,
"video": {
"decoder": "ffmpeg_h264"
}
}
```
After receiving the response you can establish the connection and start sending the stream. Check out [how to deliver input streams](./deliver-input.md) to learn more.

In this example, we are using RTP over TCP, but it could be easily replaced by UDP.
</TabItem>
<TabItem value="membrane" label="Membrane Framework">
```elixir
def handle_init(ctx, opts) do
spec = [
...

input_2_spec
|> via_in(Pad.ref(:video_input, "input_2"))
|> get_child(:live_compositor),

...
]
{[spec: spec], %{}}
end
```
where `input_2_spec` is an element that produces H264 video.
</TabItem>
</Tabs>

### Register output stream `output_1`.

Configure it to render just an empty [`View`](../api/components/View.md) component with a background color set to `#4d4d4d` (gray).

<Tabs queryString="lang">
<TabItem value="http" label="HTTP">
```http
POST: /api/output/output_1/register
Content-Type: application/json

{
"type": "rtp_stream",
"transport_protocol": "tcp_server",
"port": 9003,
"video": {
"resolution": { "width": 1280, "height": 720 },
"encoder": {
"type": "ffmpeg_h264",
"preset": "ultrafast",
"initial": {
"root": {
"type": "view",
"background_color_rgba": "#4d4d4dff",
}
}
}
}
}
```
You can can configure output framerate with [`LIVE_COMPOSITOR_OUTPUT_FRAMERATE` environmental variable](../deployment/configuration.md#live_compositor_output_framerate).

After receiving the response you can establish the connection and start listening for the stream. Check out [how to receive output streams](./receive-output.md) to learn more.

In this example we are using RTP over TCP. If you prefer to use UDP you need to start listening on the specified port before sending the register request to make sure you are not losing first frames.
</TabItem>
<TabItem value="membrane" label="Membrane Framework">
```elixir
def handle_init(ctx, opts) do
spec = [
...

get_child(:live_compositor),
|> via_out(Pad.ref(:video_output, "output_1"), options: [
width: 1280,
height: 720,
encoder: %LiveCompositor.Encoder.FFmpegH264{
preset: :ultrafast
},
initial: %{
root: %{
type: :view,
background_color_rgba: "#4d4d4dff",
}
}
])
|> output_1_spec

...
]
{[spec: spec], %{}}
end
```
where `output_1_spec` is an element that can consume H264 video.
</TabItem>
</Tabs>

`View` component does not have any children, so on the output you should see just a blank screen of a specified color as shown below.

<div style={{textAlign: 'center'}}>
<img src={SimpleSceneEmpty} style={{ width: 600 }} />
Output stream
</div>

## Update scene to show an input

Update output to render a [`View`](../api/components/View.md) component with an [`InputStream`](../api/components/InputStream.md) as its child.
Update output to:
- render a [`View`](../api/components/View.md) component with an [`InputStream`](../api/components/InputStream.md) as its child
- pass through audio from `input_1`

<Tabs queryString="lang">
<TabItem value="http" label="HTTP">
Expand All @@ -216,15 +35,20 @@ Update output to render a [`View`](../api/components/View.md) component with an
{ "type": "input_stream", "input_id": "input_1" }
]
}
},
"audio": {
"inputs": [
{ "input_id": "input_1" }
]
}
}
```
</TabItem>
<TabItem value="membrane" label="Membrane Framework">
```elixir
def handle_setup(ctx, state) do
request = %LiveCompositor.Request.UpdateVideoOutput{
output_id: "output_1",
video_request = %LiveCompositor.Request.UpdateVideoOutput{
output_id: "video_output_1",
root: %{
type: :view,
background_color_rgba: "#4d4d4dff",
Expand All @@ -233,7 +57,19 @@ Update output to render a [`View`](../api/components/View.md) component with an
]
}
}
{[notify_child: {:live_compositor, request}], state}
audio_request = %LiveCompositor.Request.UpdateAudioOutput{
output_id: "audio_output_1",
inputs: [
%{ input_id: "audio_input_1" }
]
}

actions = [
notify_child: {:live_compositor, video_request},
notify_child: {:live_compositor, audio_request}
]

{actions, state}
end
```
</TabItem>
Expand All @@ -242,13 +78,14 @@ Update output to render a [`View`](../api/components/View.md) component with an
The input stream in the example has a resolution `1920x1080` and it is rendered on the `1270x720` output. As a result only part of the stream is visible.

<div style={{textAlign: 'center'}}>
<img src={SimpleSceneOverflow} style={{ width: 600 }} />
<img src={LayoutsOverflow} style={{ width: 600 }} />
Output stream
</div>

## Resize input stream to fit inside the output
## Resize input stream to fit inside the output and mix audio

Wrap an [`InputStream`](../api/components/InputStream.md) component with a [`Rescaler`](../api/components/Rescaler.md).
Mix sightly silenced audio from `input_1` with audio from `input_2`.

<Tabs queryString="lang">
<TabItem value="http" label="HTTP">
Expand All @@ -268,6 +105,12 @@ Wrap an [`InputStream`](../api/components/InputStream.md) component with a [`Res
}
]
}
},
"audio": {
"inputs": [
{ "input_id": "input_1", volume: 0.8 },
{ "input_id": "input_2" }
]
}
}
```
Expand Down Expand Up @@ -297,7 +140,7 @@ Wrap an [`InputStream`](../api/components/InputStream.md) component with a [`Res
Input stream now fully fits inside the output.

<div style={{textAlign: 'center'}}>
<img src={SimpleSceneFitted} style={{ width: 600 }} />
<img src={LayoutsFitted} style={{ width: 600 }} />
Output stream
</div>

Expand Down Expand Up @@ -333,6 +176,12 @@ Add another [`InputStream`](../api/components/InputStream.md) wrapped with [`Res
}
]
}
},
"audio": {
"inputs": [
{ "input_id": "input_1", volume: 0.8 },
{ "input_id": "input_2" }
]
}
}
```
Expand Down Expand Up @@ -371,7 +220,7 @@ In an example below we can see that:
- Each `Rescaler` component has a size `640x720` (half of `1280x720`), but it needs to fit an input stream with `16:9` aspect ratio.

<div style={{textAlign: 'center'}}>
<img src={SimpleSceneBothInputs} style={{ width: 600 }} />
<img src={LayoutsBothInputs} style={{ width: 600 }} />
Output stream
</div>

Expand Down Expand Up @@ -405,6 +254,12 @@ Specify `width` and `height` of one of the `Rescaler` components and position it
}
]
}
},
"audio": {
"inputs": [
{ "input_id": "input_1", volume: 0.8 },
{ "input_id": "input_2" }
]
}
}
```
Expand Down Expand Up @@ -447,6 +302,6 @@ As a result:
- The second component takes the specified size, and it is positioned in the top-right corner (20px from border).

<div style={{textAlign: 'center'}}>
<img src={SimpleSceneAbsolutePosition} style={{ width: 600 }} />
<img src={LayoutsAbsolutePosition} style={{ width: 600 }} />
Output stream
</div>
Loading

0 comments on commit 97b649f

Please sign in to comment.