Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Notification support #446

Draft
wants to merge 75 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
73a938b
Add NotificationEvent Enum
andrii-i Oct 20, 2023
b37aa2b
Mention enum in the docstring
andrii-i Oct 20, 2023
2ff82d4
add Notification class
andrii-i Oct 20, 2023
ce7d5e1
Add notification-related fields to RuntimeEnv
andrii-i Oct 20, 2023
3bbfd76
add notification field to JobDef models
andrii-i Oct 20, 2023
cf70931
add notification to Job models
andrii-i Oct 20, 2023
04a9bc8
add notification to Job and JobDef orm classes
andrii-i Oct 20, 2023
199be36
add __str__ to NotificationEvent
andrii-i Oct 20, 2023
bedc97f
add notifications to front-end models
andrii-i Oct 20, 2023
276188e
add BaseModel to Notification class
andrii-i Oct 20, 2023
6eea203
make NotificationEvent a StrEnum
andrii-i Oct 20, 2023
3ed5f7f
use list comprehension for notification_events
andrii-i Oct 20, 2023
c473c6c
create notitifactions-picker component
andrii-i Oct 20, 2023
98731d9
add notifications_picker to create-job
andrii-i Oct 20, 2023
b341f49
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 20, 2023
011a315
add NotificationsPicker component
andrii-i Oct 20, 2023
d58fb03
ad NotificationsPicker to create job form
andrii-i Oct 20, 2023
bfdde0d
add Notification type, use it in ICreateJobModel
andrii-i Oct 20, 2023
7e4237c
disable notifications by default
andrii-i Oct 20, 2023
873882f
code formatriing
andrii-i Oct 20, 2023
415f9e3
Use (str, Enum) not StrEnum to work w Python <3.11
andrii-i Oct 20, 2023
1e157ec
addd notification events chips
andrii-i Oct 23, 2023
be04bf6
use NotificationPicker
andrii-i Oct 23, 2023
dca3492
make sendTo optional and add enableNotification to models
andrii-i Oct 23, 2023
d331ad8
account for sendTo being optional
andrii-i Oct 23, 2023
d86a6f4
add enable notifications toggle
andrii-i Oct 23, 2023
702dec1
make sendTo a string[] in all models
andrii-i Oct 23, 2023
e002bf6
save sendTo input to model on blur or on enter
andrii-i Oct 23, 2023
9fc56b6
add option to Include Output
andrii-i Oct 23, 2023
766a2e6
use Notification, not Notifications in UI elements
andrii-i Oct 23, 2023
a191dda
add notification to detail models
andrii-i Oct 23, 2023
a933dc4
check if enableNotification and set it if not
andrii-i Oct 23, 2023
bbb5bb6
display notification info in JobDetail
andrii-i Oct 23, 2023
32eb7f6
clean-up code
andrii-i Oct 23, 2023
5a1fa30
Update Send To, Event indexes to start from 1
andrii-i Oct 23, 2023
eaa6bc9
set notifications_enabled to False by default
andrii-i Oct 23, 2023
e9ddade
add attrib-s description to Notification, NotificationEvent
andrii-i Oct 23, 2023
b0a0f5a
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 23, 2023
b77d3b1
adjust docstring
andrii-i Oct 23, 2023
2eb0507
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 23, 2023
52c6aa1
default notification_events to []
andrii-i Oct 25, 2023
8a417bc
rename "Notification" into "NotificationSettings"
andrii-i Oct 26, 2023
6ee5055
Rename INotification --> INotificationsSettings
andrii-i Oct 26, 2023
848b71e
run prettier
andrii-i Oct 26, 2023
57f6a75
update local var names to notificationsSettings
andrii-i Oct 26, 2023
b07e516
convert props to types from interfaces
andrii-i Oct 26, 2023
711ca24
Use Sentence case, not Title Case for field labels
andrii-i Oct 26, 2023
878bca5
conver arrow functions into plain old functions, move props type decl…
andrii-i Oct 26, 2023
f200838
extract SelectedEventsChips props type into a separate type declaration
andrii-i Oct 26, 2023
319fa80
NotificationsSettings --> NotificationsConfig
andrii-i Oct 27, 2023
be78d8b
rename NotificationsSettings --> NotificationsConfig
andrii-i Oct 27, 2023
142f444
notifications_settings --> notifications_config
andrii-i Oct 27, 2023
c8acf42
rename INotificationsConfigDetailProps
andrii-i Oct 27, 2023
eb0a72a
make NotificationsConfig a separate table
andrii-i Oct 31, 2023
c68d7cf
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 31, 2023
63ce389
change validator signature
andrii-i Oct 31, 2023
1401b86
add pydantic validators
andrii-i Oct 31, 2023
041ba11
correct validator function naming
andrii-i Oct 31, 2023
e63c507
eager load NotificationsConfig relationship by default
andrii-i Oct 31, 2023
95204d7
Utilize more targeted model update function for NotificationsConfig
andrii-i Oct 31, 2023
8757418
convert NotificationsConfig to dict
andrii-i Nov 1, 2023
3503123
adjust RuntimeEnvironment
andrii-i Nov 1, 2023
9b3ca4f
fix orm to pydantic model conversion
andrii-i Nov 1, 2023
d905ff5
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 1, 2023
1696c51
fix notifications config pydantic to orm data conversion for create_j…
andrii-i Nov 1, 2023
740bad8
use props.model.notificationsConfig as a basis for NotificationsConfi…
andrii-i Nov 2, 2023
711e3d5
remove useCallback to simplify as no perf gains
andrii-i Nov 2, 2023
3ef4269
adjust SelectedEventsChips's onChange
andrii-i Nov 2, 2023
2e1b2f8
fix chip delete function
andrii-i Nov 2, 2023
c434fca
remove test defaults
andrii-i Nov 2, 2023
02dccfe
Display "Send to" and "Notification events" as Chips
andrii-i Nov 2, 2023
29331cf
remvoe NotificationsConfigItem
andrii-i Nov 2, 2023
7002908
use LabeledValue for all Notifications Settings items
andrii-i Nov 2, 2023
10b3239
represent events as comma-separated list
andrii-i Nov 2, 2023
24f9fdb
display send to as a comma-separated list of strings in details
andrii-i Nov 2, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 68 additions & 2 deletions jupyter_scheduler/models.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
import os
from enum import Enum
from typing import Dict, List, Optional, Union
from typing import Any, Dict, List, Optional, Type, Union

from pydantic import BaseModel, root_validator
from pydantic import BaseModel, root_validator, validator

Tags = List[str]
EnvironmentParameterValues = Union[int, float, bool, str]
Expand All @@ -11,6 +11,65 @@
SCHEDULE_RE = ""


class NotificationEvent(str, Enum):
"""
Enum that represents events triggering notifications. Implementers can extend
this enum to include additional notification events as needed.

Attributes:
SUCCESS (str): Sent when a job completes successfully.
FAILURE (str): Sent on job failure.
STOPPED (str): Sent when a job is manually stopped.
"""

SUCCESS = "Success"
FAILURE = "Failure"
STOPPED = "Stopped"


class NotificationsConfig(BaseModel):
"""Represents configuration for notifications.

Attributes:
send_to (List[str]): A list of symbols (e.g., email addresses) to which notifications should be sent.
events (List[NotificationEvent]): A list of events that should trigger the sending of notifications.
include_output (bool): A flag indicating whether a output should be included in the notification. Default is False.
"""

send_to: List[str] = []
events: List[NotificationEvent] = []
include_output: bool = False

class Config:
orm_mode = True

@validator("send_to")
def validate_send_to(cls, v):
if len(v) > 100:
raise ValueError("Too many 'Send to' addressee identifiers. Maximum allowed is 100.")
return v

@validator("send_to", each_item=True)
def validate_send_to_items(cls, v):
if len(v) > 100:
raise ValueError(
"Each 'Send to' addressee identifier should be at most 100 characters long."
)
return v

@validator("events")
def validate_events(cls, v):
if len(v) > 100:
raise ValueError("Too many notification events. Maximum allowed is 100.")
return v

@validator("events", each_item=True)
def validate_events_items(cls, v):
if len(v.value) > 100:
raise ValueError("Each notification event should be at most 100 characters long.")
return v


class RuntimeEnvironment(BaseModel):
"""Defines a runtime context where job
execution will happen. For example, conda
Expand All @@ -26,6 +85,8 @@ class RuntimeEnvironment(BaseModel):
compute_types: Optional[List[str]]
default_compute_type: Optional[str] # Should be a member of the compute_types list
utc_only: Optional[bool]
notifications_enabled: bool = False
notification_events: List[Type[NotificationEvent]] = []

def __str__(self):
return self.json()
Expand Down Expand Up @@ -85,6 +146,7 @@ class CreateJob(BaseModel):
name: str
output_filename_template: Optional[str] = OUTPUT_FILENAME_TEMPLATE
compute_type: Optional[str] = None
notifications_config: Optional[NotificationsConfig] = None

@root_validator
def compute_input_filename(cls, values) -> Dict:
Expand Down Expand Up @@ -145,6 +207,7 @@ class DescribeJob(BaseModel):
status: Status = Status.CREATED
status_message: Optional[str] = None
downloaded: bool = False
notifications_config: Optional[NotificationsConfig] = None

class Config:
orm_mode = True
Expand Down Expand Up @@ -209,6 +272,7 @@ class CreateJobDefinition(BaseModel):
compute_type: Optional[str] = None
schedule: Optional[str] = None
timezone: Optional[str] = None
notifications_config: Optional[NotificationsConfig] = None

@root_validator
def compute_input_filename(cls, values) -> Dict:
Expand All @@ -234,6 +298,7 @@ class DescribeJobDefinition(BaseModel):
create_time: int
update_time: int
active: bool
notifications_config: Optional[NotificationsConfig] = None

class Config:
orm_mode = True
Expand All @@ -253,6 +318,7 @@ class UpdateJobDefinition(BaseModel):
active: Optional[bool] = None
compute_type: Optional[str] = None
input_uri: Optional[str] = None
notifications_config: Optional[NotificationsConfig] = None


class ListJobDefinitionsQuery(BaseModel):
Expand Down
22 changes: 20 additions & 2 deletions jupyter_scheduler/orm.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,14 @@
from uuid import uuid4

import sqlalchemy.types as types
from sqlalchemy import Boolean, Column, Integer, String, create_engine
from sqlalchemy.orm import declarative_base, declarative_mixin, registry, sessionmaker
from sqlalchemy import Boolean, Column, ForeignKey, Integer, String, create_engine
from sqlalchemy.orm import (
declarative_base,
declarative_mixin,
registry,
relationship,
sessionmaker,
)

from jupyter_scheduler.models import EmailNotifications, Status
from jupyter_scheduler.utils import get_utc_timestamp
Expand Down Expand Up @@ -67,6 +73,14 @@ def process_result_value(self, value, dialect):
mapper_registry = registry()


class NotificationsConfigTable(Base):
__tablename__ = "notifications_config"
id = Column(String(36), primary_key=True, default=generate_uuid)
include_output = Column(Boolean, default=False)
send_to = Column(JsonType, nullable=False)
events = Column(JsonType, nullable=False)


@declarative_mixin
class CommonColumns:
runtime_environment_name = Column(String(256), nullable=False)
Expand Down Expand Up @@ -98,6 +112,8 @@ class Job(CommonColumns, Base):
url = Column(String(256), default=generate_jobs_url)
pid = Column(Integer)
idempotency_token = Column(String(256))
notifications_config_id = Column(String(36), ForeignKey("notifications_config.id"))
notifications_config = relationship("NotificationsConfigTable", lazy="joined")


class JobDefinition(CommonColumns, Base):
Expand All @@ -108,6 +124,8 @@ class JobDefinition(CommonColumns, Base):
url = Column(String(256), default=generate_job_definitions_url)
create_time = Column(Integer, default=get_utc_timestamp)
active = Column(Boolean, default=True)
notifications_config_id = Column(String(36), ForeignKey("notifications_config.id"))
notifications_config = relationship("NotificationsConfigTable", lazy="joined")


def create_tables(db_url, drop_tables=False):
Expand Down
34 changes: 31 additions & 3 deletions jupyter_scheduler/scheduler.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,12 @@
UpdateJob,
UpdateJobDefinition,
)
from jupyter_scheduler.orm import Job, JobDefinition, create_session
from jupyter_scheduler.orm import (
Job,
JobDefinition,
NotificationsConfigTable,
create_session,
)
from jupyter_scheduler.utils import create_output_directory, create_output_filename


Expand Down Expand Up @@ -396,7 +401,17 @@ def create_job(self, model: CreateJob) -> str:
if not model.output_formats:
model.output_formats = []

job = Job(**model.dict(exclude_none=True, exclude={"input_uri"}))
orm_notifications_config = None
if model.notifications_config:
orm_notifications_config = NotificationsConfigTable(
**model.notifications_config.dict()
)
session.add(orm_notifications_config)

job_data = model.dict(exclude={"input_uri", "notifications_config"})
job_data["notifications_config"] = orm_notifications_config

job = Job(**job_data)
session.add(job)
session.commit()

Expand Down Expand Up @@ -534,7 +549,20 @@ def create_job_definition(self, model: CreateJobDefinition) -> str:
if not self.file_exists(model.input_uri):
raise InputUriError(model.input_uri)

job_definition = JobDefinition(**model.dict(exclude_none=True, exclude={"input_uri"}))
orm_notifications_config = None
if model.notifications_config:
orm_notifications_config = NotificationsConfigTable(
**model.notifications_config.dict()
)
session.add(orm_notifications_config)
session.flush()

job_definition_data = model.dict(
exclude={"input_uri", "notifications_config"}, exclude_none=True
)
job_definition = JobDefinition(
**job_definition_data, notifications_config=orm_notifications_config
)
session.add(job_definition)
session.commit()

Expand Down
45 changes: 45 additions & 0 deletions src/components/notifications-config-detail.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
import React from 'react';

import { Card, CardContent, Stack, FormLabel } from '@mui/material';
import { useTranslator } from '../hooks';
import { Scheduler } from '../handler';
import { LabeledValue } from './labeled-value';

type INotificationsConfigDetailProps = {
notificationsConfig: Scheduler.INotificationsConfig;
};

export function NotificationsConfigDetail(
props: INotificationsConfigDetailProps
): JSX.Element {
const trans = useTranslator('jupyterlab');
const sendTo = props.notificationsConfig.send_to.join(', ');
const events = props.notificationsConfig.events.join(', ');

return (
<Card>
<CardContent>
<FormLabel component="legend" sx={{ mb: 2 }}>
{trans.__('Notifications Settings')}
</FormLabel>
<Stack spacing={2}>
<LabeledValue
label={trans.__('Send to')}
value={sendTo}
style={{ flex: '1 1 100%' }}
/>
<LabeledValue
label={trans.__('Notification events')}
value={events}
style={{ flex: '1 1 100%' }}
/>
<LabeledValue
label={trans.__('Include output')}
value={props.notificationsConfig.include_output ? 'True' : 'False'}
style={{ flex: '1 1 100%' }}
/>
</Stack>
</CardContent>
</Card>
);
}
Loading
Loading