Skip to content

Commit

Permalink
Merge pull request #10 from dyte-in/feat/WEB-3445-symbl-custom-connec…
Browse files Browse the repository at this point in the history
…tionId-userId

feat(custom-connectionId-userId): WEB-3445 added support for custom connectionId, userId along with some fixes
  • Loading branch information
palashgo authored Dec 9, 2023
2 parents 4e02ab6 + a73a7fa commit b67a38a
Show file tree
Hide file tree
Showing 12 changed files with 699 additions and 696 deletions.
82 changes: 56 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,34 +1,19 @@
# Dyte <> Symbl.ai transcriptions

## How to test this?

Go to terminal

Install the packages.
`npm install`

Run it locally
`npm run dev`

It will run a server on localhost:3000 serving the HTML containing the sample integration from index.html.

Use the following URL to test.
`
http://localhost:3000/?authToken=PUT_DYTE_PARTICIPANT_AUTH_TOKEN_HERE&symblAccessToken=PUT_SYMBL_ACCESS_TOKEN_HERE
`
A quick and easy solution to integrate Symbl.ai's transcriptions and conversational AI services with Dyte's SDK.

## How to use it with Dyte?

Find the Dyte integration logic in your codebase which may look like this
1. Please find the Dyte integration logic in your codebase which may look like the following.

```
```js
// Somewhere in your codebase
const meeting = await DyteClient.init(...)
```

On top of the file where integration was found, import this package.
2. On top of the file where integration was found, import this package.

```
```js
import {
activateTranscriptions,
deactivateTranscriptions,
Expand All @@ -37,18 +22,26 @@ import {
} from '@dytesdk/symbl-transcription';
```

Now you can activate Symbl transcriptions.

```
3. Now you can activate Symbl transcriptions.

```js
activateTranscriptions({
meeting: meeting, // From DyteClient.init
symblAccessToken: 'ACCESS_TOKEN_FROM_SYMBL_AI',
connectionId: 'SOME_ARBITRARY_CONNECTION_ID', // optional,
speakerUserId: 'SOME_ARBITRARY_USER_ID_FOR_SPEAKER', // optional
});
```

This would ensure that your audio gets translated and resultant transcriptions get sent to all participants including `self` being referred by `meeting.self`.
This method internally connects with Symbl using Websocket connection & automatically forwards the audio to them, while your Mic is on. On receiving transcriptions from Symbl, we broadcast those transcriptions to all the participants of the meeting, including the speaker, being referred by `meeting.self` .

`connectionId` field is optional. If not passed, value of `meeting.meta.roomName` will be used as `connectionId`.

`speakerUserId` field is optional. If not passed, value of `meeting.self.clientSpecificId` will be used as `speakerUserId`.


If you want to show transcriptions to a participant or for `self`, you can do so using the following snippet.
4. If you want to show transcriptions to a participant or for `self`, you can do so using the following snippet.

```
addTranscriptionsListener({
Expand All @@ -58,6 +51,11 @@ addTranscriptionsListener({
})
```

Above code snippet helps you segregate speakers from listeners.

For example, If you know that a participant is only meant to act as a listener, you can avoid calling `activateTranscriptions` and simply only call `addTranscriptionsListener` that runs solely over Dyte, thus reducing concurrent connections to Symbl thus giving you a potential cost benefit.


Using `transcriptionsCallback` you can populate the transcriptions in your app/website at any desired place.

<b>NOTE</b>: For every partial or complete sentence, `transcriptionsCallback` will be called, with all formatted transcriptions.
Expand All @@ -67,10 +65,9 @@ Once meeting is over, deactivate the transcription generation.
```
deactivateTranscriptions({
meeting: meeting, // From DyteClient.init
symblAccessToken: 'ACCESS_TOKEN_FROM_SYMBL_AI',
});
```
In similar fashion, remove the transcriptions listener, once the meeting is over.
In a similar fashion, remove the transcriptions listener, once the meeting is over.

```
removeTranscriptionsListener({meeting: meeting});
Expand All @@ -93,3 +90,36 @@ curl -k -X POST "https://api.symbl.ai/oauth2/token:generate" \
"appSecret": "YOUR_APP_SECRET"
}'
```

# How to subscribe to transcriptions of this conversation?

Please pass a unique `connectionId` for this meeting and a unique `speakerUserId` for the speaker while activating treanscriptions using `activateTranscriptions` method.

This would help you use subscribe API of Symbl, located at https://docs.symbl.ai/reference/subscribe-api along with better control over the speakers.

# How to test Symbl integration quickly without having to integrate Dyte beforehand?

To see the demo or to test the Symbl integration, please go to https://github.com/dyte-in/symbl-transcription and clone the repo and run the npm script named `dev`.

```sh
git clone https://github.com/dyte-in/symbl-transcription.git
cd symbl-transcription
npm install
npm run dev
```

It will run a server on localhost:3000 serving the HTML containing the sample integration from index.html.

Please use the following URL to see the Default Dyte Meeting interface.

```text
http://localhost:3000/?authToken=PUT_DYTE_PARTICIPANT_AUTH_TOKEN_HERE&symblAccessToken=PUT_SYMBL_ACCESS_TOKEN_HERE
```

In case you are still using v1 meetings, please use the following URL.
```text
http://localhost:3000/?authToken=PUT_DYTE_PARTICIPANT_AUTH_TOKEN_HERE&symblAccessToken=PUT_SYMBL_ACCESS_TOKEN_HERE&roomName=PUT_DYTE_ROOM_NAME_HERE
```

Once the Dyte UI is loaded, please turn on the Mic and grant permissions, if asked. Post that, try speaking sentences in English (default) to see the transcriptions.
85 changes: 85 additions & 0 deletions demo/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
import DyteClient from '@dytesdk/web-core';
import { defineCustomElements } from '@dytesdk/ui-kit/loader/index.es2017';
import {
activateTranscriptions,
addTranscriptionsListener,
deactivateTranscriptions,
removeTranscriptionsListener,
} from '../src/index';

defineCustomElements();

const init = async () => {
try {
const params = new URLSearchParams(window.location.search);
const roomName = params.get('roomName') || '';
const authToken = params.get('authToken') || '';
const symblAccessToken = params.get('symblAccessToken') || '';

if (!authToken || (roomName && !authToken)) {
alert('Please pass authToken (and roomName, if you are using v1 APIs) in query params');
return;
}
if (!symblAccessToken) {
alert('Please pass symblAccessToken in query params');
return;
}

const meeting = await DyteClient.init({
authToken,
roomName,
apiBase: 'https://api.dyte.io',
defaults: {
audio: false,
video: false,
},
});

(document.getElementById('my-meeting') as any).meeting = meeting;
Object.assign(window, { meeting });

// Initialize speech client
await activateTranscriptions({
meeting,
languageCode: 'en-US',
symblAccessToken,
});

await addTranscriptionsListener({
meeting,
noOfTranscriptionsToCache: 200,
transcriptionsCallback: (transcriptions) => {
const transcription = document.getElementById('dyte-transcriptions') as HTMLDivElement;
const list = transcriptions.slice(-3);
transcription.innerHTML = '';
list.forEach((item) => {
const speaker = document.createElement('span');
speaker.classList.add('dyte-transcription-speaker');
speaker.innerText = `${item.displayName}: `;

const text = document.createElement('span');
text.classList.add('dyte-transcription-text');
text.innerText = item.text.toString().trim() !== '' ? item.text.toString().trim() : '...';

const container = document.createElement('span');
container.classList.add('dyte-transcription-line');
container.appendChild(speaker);
container.appendChild(text);

transcription.appendChild(container);
});
},
});

meeting.self.on('roomLeft', () => {
const transcriptionsDiv = document.getElementById('dyte-transcriptions') as HTMLDivElement;
transcriptionsDiv.innerHTML = '';
deactivateTranscriptions({ meeting });
removeTranscriptionsListener({ meeting });
});
} catch (e) {
console.log(e);
}
};

init();
149 changes: 42 additions & 107 deletions index.html
Original file line number Diff line number Diff line change
@@ -1,109 +1,44 @@
<!DOCTYPE html>
<html>
<head>
<script type="module">
import { defineCustomElements } from 'https://cdn.jsdelivr.net/npm/@dytesdk/[email protected]/loader/index.es2017.js';
defineCustomElements();
</script>
<!-- Import Web Core via CDN too -->
<script src="https://cdn.dyte.in/core/dyte.js"></script>
<script type="module" src="/src/index.ts"></script>
<style>
#dyte-transcriptions{
position: absolute;
bottom: 15%;
left: 25%;
background-color: black;
opacity: 0.7;
width: auto;
padding: 10px 10px;
max-width: 50%;
min-width: 10%;
}
.dyte-transcription-line{
display: block;
}
.dyte-transcription-speaker{
font-weight: bold;
color: white;
}
.dyte-transcription-text{
color: white;
font-weight: normal;
}
</style>
</head>
<body>
<dyte-meeting id="my-meeting"></dyte-meeting>
<div id="dyte-transcriptions"></div>
<script>
const params = new Proxy(new URLSearchParams(window.location.search), {
get: (searchParams, prop) => searchParams.get(prop),
});

function transcriptionsListenerCB(transcriptions) {
const transcriptionsDiv = document.getElementById('dyte-transcriptions');

if (transcriptionsDiv) {
transcriptionsDiv.innerHTML = '';
transcriptionsDiv.classList.remove('hidden');

const transcriptionsToShow = transcriptions.slice(-3);

transcriptionsToShow.forEach((transcription) => {
const speakerSpan = document.createElement('span');
speakerSpan.classList.add('dyte-transcription-speaker');
speakerSpan.innerText = `${transcription.displayName}: `;

const transcriptionSpan = document.createElement('span');
transcriptionSpan.classList.add('dyte-transcription-text');
transcriptionSpan.innerText = transcription.text?.toString();

const transcriptionLine = document.createElement('span');
transcriptionLine.classList.add('dyte-transcription-line');
transcriptionLine.appendChild(speakerSpan).appendChild(transcriptionSpan);

transcriptionsDiv.appendChild(transcriptionLine);
});
}
}

const init = async () => {
if(!params.authToken || (params.roomName && !params.authToken)){
alert("Please pass authToken (and roomName, if you are using v1 APIs) in query params");
return;
}
if(!params.symblAccessToken){
alert("Please pass symblAccessToken in query params");
return;
}
const meeting = await DyteClient.init({
authToken: params.authToken,
roomName: params.roomName,
defaults: {
audio: true,
video: false,
},
});

document.getElementById('my-meeting').meeting = meeting;
Symbl.activateTranscriptions({
meeting: meeting, // From DyteClient.init
symblAccessToken: params.symblAccessToken
});
await Symbl.addTranscriptionsListener({
meeting: meeting,
noOfTranscriptionsToCache: 200,
transcriptionsCallback: transcriptionsListenerCB,
});
meeting.self.on('roomLeft', () => {
const transcriptionsDiv = document.getElementById('dyte-transcriptions')
transcriptionsDiv.innerHTML = '';
transcriptionsDiv.classList.add('hidden');
Symbl.deactivateTranscriptions({ meeting });
});
};
init();
</script>
</body>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Transcription Demo</title>
<style>
body {
height: 100vh;
width: 100vw;
}

#dyte-transcriptions{
position: absolute;
z-index: 99999;
bottom: 15%;
width: 100%;
display:flex;
flex-direction: column;
justify-content: center;
align-items:center;
}

.dyte-transcription-line{
display: block;
max-width: 80%;
text-align: center !important;
}
.dyte-transcription-speaker{
font-weight: 500;
color: orange;
}
.dyte-transcription-text{
color: white;
}
</style>
</head>
<body>
<dyte-meeting id="my-meeting" show-setup-screen="false"></dyte-meeting>
<div id="dyte-transcriptions"></div>
<script type="module" src="./demo/index.ts"></script>
</body>
</html>
Loading

0 comments on commit b67a38a

Please sign in to comment.