Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: differentiate shorts, lives and long videos #371

Closed
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]


### Fixed

- Diffrentiate user uploaded shorts, lives, & long videos (#367)
arjitcodes marked this conversation as resolved.
Show resolved Hide resolved
- corrected the short video resolution in the UI (#366)

### Fixed

- Check for empty playlists after filtering, and after downloading videos (#375)
Expand Down
3 changes: 3 additions & 0 deletions scraper/src/youtube2zim/schemas.py
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,9 @@
banner_path: str | None = None
joined_date: str
main_playlist: str | None = None
user_long_uploads_playlist: str | None = None
user_short_uploads_playlist: str | None = None
user_lives_playlist: str | None = None

Check warning on line 111 in scraper/src/youtube2zim/schemas.py

View check run for this annotation

Codecov / codecov/patch

scraper/src/youtube2zim/schemas.py#L109-L111

Added lines #L109 - L111 were not covered by tests
playlist_count: int


Expand Down
23 changes: 23 additions & 0 deletions scraper/src/youtube2zim/scraper.py
Original file line number Diff line number Diff line change
Expand Up @@ -171,6 +171,9 @@
# process-related
self.playlists = []
self.uploads_playlist_id = None
self.user_long_uploads_playlist_id = None
self.user_short_uploads_playlist_id = None
self.user_lives_playlist_id = None

Check warning on line 176 in scraper/src/youtube2zim/scraper.py

View check run for this annotation

Codecov / codecov/patch

scraper/src/youtube2zim/scraper.py#L174-L176

Added lines #L174 - L176 were not covered by tests
self.videos_ids = []
self.video_ids_count = 0
self.videos_processed = 0
Expand Down Expand Up @@ -553,6 +556,9 @@
self.playlists,
self.main_channel_id,
self.uploads_playlist_id,
self.user_long_uploads_playlist_id,
self.user_short_uploads_playlist_id,
self.user_lives_playlist_id,
self.is_playlist,
) = extract_playlists_details_from(self.youtube_id)

Expand Down Expand Up @@ -1045,6 +1051,7 @@
author = videos_channels[video_id]
subtitles_list = get_subtitles(video_id)
channel_data = get_channel_json(author["channelId"])

return Video(
id=video_id,
title=video["snippet"]["title"],
Expand Down Expand Up @@ -1155,6 +1162,10 @@
home_playlist_list = []

main_playlist_slug = None
user_long_uploads_playlist_slug = None
user_short_uploads_playlist_slug = None
user_lives_playlist_slug = None

Check warning on line 1167 in scraper/src/youtube2zim/scraper.py

View check run for this annotation

Codecov / codecov/patch

scraper/src/youtube2zim/scraper.py#L1165-L1167

Added lines #L1165 - L1167 were not covered by tests

empty_playlists = list(
filter(lambda playlist: len(get_videos_list(playlist)) == 0, self.playlists)
)
Expand Down Expand Up @@ -1195,6 +1206,15 @@
# modify playlist object for preview on homepage
playlist_obj.videos = playlist_obj.videos[:12]

if playlist.playlist_id == self.user_long_uploads_playlist_id:
user_long_uploads_playlist_slug = playlist_slug

Check warning on line 1210 in scraper/src/youtube2zim/scraper.py

View check run for this annotation

Codecov / codecov/patch

scraper/src/youtube2zim/scraper.py#L1210

Added line #L1210 was not covered by tests

if playlist.playlist_id == self.user_short_uploads_playlist_id:
user_short_uploads_playlist_slug = playlist_slug

Check warning on line 1213 in scraper/src/youtube2zim/scraper.py

View check run for this annotation

Codecov / codecov/patch

scraper/src/youtube2zim/scraper.py#L1213

Added line #L1213 was not covered by tests

if playlist.playlist_id == self.user_lives_playlist_id:
user_lives_playlist_slug = playlist_slug

Check warning on line 1216 in scraper/src/youtube2zim/scraper.py

View check run for this annotation

Codecov / codecov/patch

scraper/src/youtube2zim/scraper.py#L1216

Added line #L1216 was not covered by tests

if playlist.playlist_id == self.uploads_playlist_id:
main_playlist_slug = (
playlist_slug # set uploads playlist as main playlist
Expand Down Expand Up @@ -1242,6 +1262,9 @@
profile_path="profile.jpg",
banner_path="banner.jpg",
main_playlist=main_playlist_slug,
user_long_uploads_playlist=user_long_uploads_playlist_slug,
user_short_uploads_playlist=user_short_uploads_playlist_slug,
user_lives_playlist=user_lives_playlist_slug,
playlist_count=len(self.playlists),
joined_date=channel_data["snippet"]["publishedAt"],
).model_dump_json(by_alias=True, indent=2),
Expand Down
42 changes: 39 additions & 3 deletions scraper/src/youtube2zim/youtube.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
# vim: ai ts=4 sts=4 et sw=4 nu

from http import HTTPStatus
from datetime import datetime

Check warning on line 5 in scraper/src/youtube2zim/youtube.py

View check run for this annotation

Codecov / codecov/patch

scraper/src/youtube2zim/youtube.py#L5

Added line #L5 was not covered by tests

import requests
from dateutil import parser as dt_parser
Expand Down Expand Up @@ -56,6 +57,10 @@
@classmethod
def from_id(cls, playlist_id):
playlist_json = get_playlist_json(playlist_id)
if playlist_json is None:
raise PlaylistNotFoundError(

Check warning on line 61 in scraper/src/youtube2zim/youtube.py

View check run for this annotation

Codecov / codecov/patch

scraper/src/youtube2zim/youtube.py#L61

Added line #L61 was not covered by tests
f"Invalid playlistId `{playlist_id}`: Not Found"
) from None
arjitcodes marked this conversation as resolved.
Show resolved Hide resolved
return Playlist(
playlist_id=playlist_id,
title=playlist_json["snippet"]["title"],
Expand Down Expand Up @@ -176,10 +181,13 @@
req.raise_for_status()
try:
playlist_json = req.json()["items"][0]
total_results = req.json().get("pageInfo", {}).get("totalResults", 0)

Check warning on line 184 in scraper/src/youtube2zim/youtube.py

View check run for this annotation

Codecov / codecov/patch

scraper/src/youtube2zim/youtube.py#L184

Added line #L184 was not covered by tests
if total_results == 0:
logger.error(f"Playlist `{playlist_id}`: No Item Available")
return None

Check warning on line 187 in scraper/src/youtube2zim/youtube.py

View check run for this annotation

Codecov / codecov/patch

scraper/src/youtube2zim/youtube.py#L186-L187

Added lines #L186 - L187 were not covered by tests
except IndexError:
raise PlaylistNotFoundError(
f"Invalid playlistId `{playlist_id}`: Not Found"
) from None
logger.error(f"Invalid playlistId `{playlist_id}`: Not Found")
return None

Check warning on line 190 in scraper/src/youtube2zim/youtube.py

View check run for this annotation

Codecov / codecov/patch

scraper/src/youtube2zim/youtube.py#L189-L190

Added lines #L189 - L190 were not covered by tests
save_json(YOUTUBE.cache_dir, fname, playlist_json)
return playlist_json

Expand Down Expand Up @@ -347,6 +355,31 @@
playlist_ids = [
p["id"] for p in get_channel_playlists_json(main_channel_id)
]

# Get special playlists JSON objects
user_long_uploads_json = get_playlist_json("UULF" + main_channel_id[2:])
user_short_uploads_json = get_playlist_json("UUSH" + main_channel_id[2:])
user_lives_json = get_playlist_json("UULV" + main_channel_id[2:])

Check warning on line 362 in scraper/src/youtube2zim/youtube.py

View check run for this annotation

Codecov / codecov/patch

scraper/src/youtube2zim/youtube.py#L360-L362

Added lines #L360 - L362 were not covered by tests

# Extract special playlists IDs if the JSON objects are not None
user_long_uploads_playlist_id = (

Check warning on line 365 in scraper/src/youtube2zim/youtube.py

View check run for this annotation

Codecov / codecov/patch

scraper/src/youtube2zim/youtube.py#L365

Added line #L365 was not covered by tests
user_long_uploads_json["id"] if user_long_uploads_json else None
)
user_short_uploads_playlist_id = (

Check warning on line 368 in scraper/src/youtube2zim/youtube.py

View check run for this annotation

Codecov / codecov/patch

scraper/src/youtube2zim/youtube.py#L368

Added line #L368 was not covered by tests
user_short_uploads_json["id"] if user_short_uploads_json else None
)
user_lives_playlist_id = user_lives_json["id"] if user_lives_json else None

Check warning on line 371 in scraper/src/youtube2zim/youtube.py

View check run for this annotation

Codecov / codecov/patch

scraper/src/youtube2zim/youtube.py#L371

Added line #L371 was not covered by tests

# Add special playlists if they exists
playlist_ids += filter(

Check warning on line 374 in scraper/src/youtube2zim/youtube.py

View check run for this annotation

Codecov / codecov/patch

scraper/src/youtube2zim/youtube.py#L374

Added line #L374 was not covered by tests
None,
[
user_long_uploads_playlist_id,
user_short_uploads_playlist_id,
user_lives_playlist_id,
],
)

# we always include uploads playlist (contains everything)
playlist_ids += [
channel_json["contentDetails"]["relatedPlaylists"]["uploads"]
Expand All @@ -371,5 +404,8 @@
[Playlist.from_id(playlist_id) for playlist_id in dict.fromkeys(playlist_ids)],
main_channel_id,
uploads_playlist_id,
user_long_uploads_playlist_id,
user_short_uploads_playlist_id,
user_lives_playlist_id,
is_playlist,
)
1 change: 1 addition & 0 deletions zimui/src/assets/main.css
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
html {
overflow: auto !important;
font-family: 'Roboto', sans-serif;
overflow-y: scroll !important;
}

body {
Expand Down
35 changes: 35 additions & 0 deletions zimui/src/assets/vjs-youtube.css
Original file line number Diff line number Diff line change
Expand Up @@ -54,3 +54,38 @@
.vjs-youtube .vjs-tech canvas {
border-radius: 8px;
}


.video-js.vjs-fluid,
.video-js.vjs-16-9,
.video-js.vjs-4-3,
video.video-js,
video.vjs-tech {
max-height: calc(100vh - 64px);
position: relative !important;
width: 100%;
height: auto !important;
max-width: 100% !important;
padding-top: 0 !important;
line-height: 0;
}
.vjs-control-bar {
line-height: 1;
}

/* Fullscreen styles */
.video-js.vjs-fullscreen {
display: flex;
align-items: center;
justify-content: center;
background-color: black;
text-align: center;
}

.video-js.vjs-fullscreen video {
margin: auto;
width: auto !important;
height: 100% !important;
max-height: 100vh;
object-fit: contain;
}
35 changes: 23 additions & 12 deletions zimui/src/components/channel/ChannelHeader.vue
Original file line number Diff line number Diff line change
Expand Up @@ -20,23 +20,34 @@ onMounted(async () => {
}
})

const tabs = [
{
id: 0,
title: 'Videos',
to: { name: 'videos' }
},
{
id: 1,
title: 'Playlists',
to: { name: 'playlists' }
// Computed tabs array based on store data
const tabs = computed(() => {
const baseTabs = [
{ id: 0, title: 'Home', to: { name: 'home' } }
];

if (main.channel?.userLongUploadsPlaylist) {
baseTabs.push({ id: 1, title: 'Videos', to: { name: 'videos' } });
}

if (main.channel?.userShortUploadsPlaylist) {
baseTabs.push({ id: 2, title: 'Shorts', to: { name: 'shorts' } });
}

if (main.channel?.userLivesPlaylist) {
baseTabs.push({ id: 3, title: 'Lives', to: { name: 'lives' } });
}
]

baseTabs.push({ id: 4, title: 'Playlists', to: { name: 'playlists' } });

return baseTabs;
});


// Hide tabs if there is only one playlist
const hideTabs = computed(() => main.channel?.playlistCount === 1)

const tab = ref<number>(tabs[0].id)
const tab = ref<number>(tabs.value[0]?.id || 0);
</script>

<template>
Expand Down
19 changes: 19 additions & 0 deletions zimui/src/components/channel/tabs/ChannelHomeTab.vue
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
<script setup lang="ts">
import { useMainStore } from '@/stores/main'
import { computed } from 'vue'

import ChannelHomeGridTab from './ChannelHomeGridTab.vue'
import ChannelHomeListTab from './ChannelHomeListTab.vue'

const main = useMainStore()
const hideTabs = computed(() => main.channel?.playlistCount === 1)
</script>

<template>
<div v-if="hideTabs">
<channel-home-grid-tab />
</div>
<div v-else>
<channel-Home-list-tab />
</div>
</template>
7 changes: 7 additions & 0 deletions zimui/src/components/channel/tabs/LivesTab.vue
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
<script setup lang="ts">
import TabView from '@/views/TabView.vue'
</script>

<template>
<tab-view :playlist-slug="'userLivesPlaylist'" title-prefix="Lives" />
</template>
7 changes: 7 additions & 0 deletions zimui/src/components/channel/tabs/ShortsTab.vue
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
<script setup lang="ts">
import TabView from '@/views/TabView.vue'
</script>

<template>
<tab-view :playlist-slug="'userShortUploadsPlaylist'" title-prefix="Shorts" />
</template>
16 changes: 2 additions & 14 deletions zimui/src/components/channel/tabs/VideosTab.vue
Original file line number Diff line number Diff line change
@@ -1,19 +1,7 @@
<script setup lang="ts">
import { useMainStore } from '@/stores/main'
import { computed } from 'vue'

import VideosGridTab from './VideosGridTab.vue'
import VideosListTab from './VideosListTab.vue'

const main = useMainStore()
const hideTabs = computed(() => main.channel?.playlistCount === 1)
import TabView from '@/views/TabView.vue'
</script>

<template>
<div v-if="hideTabs">
<videos-grid-tab />
</div>
<div v-else>
<videos-list-tab />
</div>
<tab-view :playlist-slug="'userLongUploadsPlaylist'" title-prefix="Videos" />
</template>
20 changes: 19 additions & 1 deletion zimui/src/router/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,10 @@ import HomeView from '../views/HomeView.vue'
import VideoPlayerView from '../views/VideoPlayerView.vue'
import NotFoundView from '../views/NotFoundView.vue'

import ChannelHomeTab from '@/components/channel/tabs/ChannelHomeTab.vue'
import VideosTab from '@/components/channel/tabs/VideosTab.vue'
import ShortsTab from '@/components/channel/tabs/ShortsTab.vue'
import LivesTab from '@/components/channel/tabs/LivesTab.vue'
import PlaylistsTab from '@/components/channel/tabs/PlaylistsTab.vue'
import PlaylistView from '@/views/PlaylistView.vue'

Expand All @@ -15,13 +18,28 @@ const router = createRouter({
path: '/',
name: 'home',
component: HomeView,
redirect: '/videos',
redirect: '/channel-home',
children: [
{
path: 'channel-home',
name: 'channel-home',
component: ChannelHomeTab
},
{
path: 'videos',
name: 'videos',
component: VideosTab
},
{
path: 'shorts',
name: 'shorts',
component: ShortsTab
},
{
path: 'lives',
name: 'lives',
component: LivesTab
},
{
path: 'playlists',
name: 'playlists',
Expand Down
3 changes: 3 additions & 0 deletions zimui/src/types/Channel.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@ export interface Channel {
bannerPath?: string
joinedDate: string
mainPlaylist?: string
userLongUploadsPlaylist?:string
userShortUploadsPlaylist?:string
userLivesPlaylist?:string
playlistCount: number
}

Expand Down
Loading
Loading