Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(place): event scrape #118

Open
wants to merge 26 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
4681de3
feat(guest scrape): initial commit
pkheav Mar 10, 2021
1750ca1
feat(guest scrape spec): add StaffAPI to spec
pkheav Mar 10, 2021
ef510df
feat(guest scrape): try to get systems from zones
pkheav Mar 11, 2021
9554a98
feat(guest scrape): use System model
pkheav Mar 11, 2021
983fdb3
fix(guest scrape): add System properly
pkheav Mar 11, 2021
50da4b9
chore(guest scrape): add comment re. array union
pkheav Mar 11, 2021
de3d651
feat(staff api): add module query methods
pkheav Mar 11, 2021
e20f841
feat(guest scrape): don't convert to json
pkheav Mar 11, 2021
7bba7f1
fix(guest scrape): only get system ids with Booking modules
pkheav Mar 11, 2021
d8c7982
chore(guest scrape): add comments
pkheav Mar 11, 2021
96bac74
feat(staff api): add get_module_state method
pkheav Mar 11, 2021
927be22
feat(staff api): add lookup param for get_module_state
pkheav Mar 11, 2021
58de43f
feat(guest scrape): get bookings from modules
pkheav Mar 11, 2021
c00bece
fix(guest scrape): get module bookings properly
pkheav Mar 11, 2021
043c66c
feat(guest scrape): basic spec for get_bookings
pkheav Mar 11, 2021
fb7693a
fix(guest scrape): cast module bookings correctly
pkheav Mar 11, 2021
5475bb8
fix(guest_scrape): deal with no bookings case
pkheav Mar 12, 2021
5a90b79
fix(guest scrape): cast json booking resp
pkheav Mar 12, 2021
ccd89e4
feat(guest scrape): use event model
pkheav Mar 12, 2021
41a3d32
fix(guest_scrape): use correct params for mailer
pkheav Mar 12, 2021
03746c8
fix(guest scrape): convert event_start to Int
pkheav Mar 12, 2021
5be46e7
feat(guest scrape): don't send to email to host
pkheav Mar 12, 2021
3d94bc1
chore(event scrape): rename from Guest to Event
pkheav Mar 15, 2021
0340904
feat(event scrape): update response structure
pkheav Mar 15, 2021
934633c
feat(event scrape): add filtering of events by time
pkheav Mar 15, 2021
0e234bc
fix(event scrape): simplify bookings parsing
pkheav Mar 15, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 84 additions & 0 deletions drivers/place/event_scrape.cr
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
require "place_calendar"

class Place::EventScrape < PlaceOS::Driver
descriptive_name "PlaceOS Event Scrape"
generic_name :EventScrape

default_settings({
zone_ids: ["placeos-zone-id"],
internal_domains: ["PlaceOS.com"]
})

accessor staff_api : StaffAPI_1

@zone_ids = [] of String
@internal_domains = [] of String

alias Event = PlaceCalendar::Event

struct SystemWithEvents
include JSON::Serializable

def initialize(@name : String, @zones : Array(String), @events : Array(Event))
end
end

def on_load
on_update
end

def on_update
@zone_ids = setting?(Array(String), :zone_ids) || [] of String
@internal_domains = setting?(Array(String), :internal_domains) || [] of String
end

def todays_bookings
response = {
internal_domains: @internal_domains,
systems: {} of String => SystemWithEvents
}

now = Time.local
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's unlikely the the server TZ will be the same as the users TZ.

May be worth pulling a larger time slice, then filtering on front-ends, or support passing a timezone either to this method call, or applying as a setting for the driver instance.

start_epoch = now.at_beginning_of_day.to_unix
end_epoch = now.at_end_of_day.to_unix

@zone_ids.each do |z_id|
staff_api.systems(zone_id: z_id).get.as_a.each do |sys|
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The #get here will block while each request completes, forcing them to run sequentially. It looks like these could happen safely concurrently.

It should be possible to reorganise this so that you map @zone_ids to an array of Futures (the staff_api.systems(..) call, then combine these into your response.

sys_id = sys["id"].as_s
# In case the same system is in multiple zones
next if response[:systems][sys_id]?

response[:systems][sys_id] = SystemWithEvents.new(
name: sys["name"].as_s,
zones: Array(String).from_json(sys["zones"].to_json),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a roundtrip through some JSON serialisation here that doesn't look to be required.

events: get_system_bookings(sys_id, start_epoch, end_epoch)
)
end
end

response
end

def get_system_bookings(sys_id : String, start_epoch : Int64?, end_epoch : Int64?) : Array(Event)
booking_module = staff_api.modules_from_system(sys_id).get.as_a.find { |mod| mod["name"] == "Bookings" }
# If the system has a booking module with bookings
if booking_module && (bookings = staff_api.get_module_state(booking_module["id"].as_s).get["bookings"]?)
Comment on lines +63 to +65
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may need to expand the staff API driver to support this, but this can collapse into a single query via this endpoint:

https://github.com/PlaceOS/rest-api/blob/5d3acc0cff8d949d360a8f6de9c2e916c52f5cf1/src/placeos-rest-api/controllers/systems.cr#L314-L321

bookings = Array(Event).from_json(bookings.as_s)

# If both start_epoch and end_epoch are passed
if start_epoch && end_epoch
# Convert start/end_epoch to Time object as Event.event_start.class == Time
start_time = Time.unix(start_epoch)
end_time = Time.unix(end_epoch)
range = (start_time..end_time)
# Only select bookings within start_epoch and end_epoch
bookings.select! { |b| range.includes?(b.event_start) }
bookings
end

bookings
else
[] of Event
end
end
end
68 changes: 68 additions & 0 deletions drivers/place/event_scrape_spec.cr
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
DriverSpecs.mock_driver "Place::EventScrape" do
system({
StaffAPI: {StaffAPI}
})

exec(:get_bookings)
end

class StaffAPI < DriverSpecs::MockDriver
def systems(zone_id : String)
logger.info { "requesting zone #{zone_id}" }

sys_1 = {
id: "sys-1",
name: "Room 1",
zones: ["placeos-zone-id"]
}

if zone_id == "placeos-zone-id"
[sys_1]
else
[
sys_1,
{
id: "sys-2",
name: "Room 2",
zones: ["zone-1"]
}
]
end
end

def modules_from_system(system_id : String)
[
{
id: "mod-1",
control_system_id: system_id,
name: "Calendar"
},
{
id: "mod-2",
control_system_id: system_id,
name: "Bookings"
},
{
id: "mod-3",
control_system_id: system_id,
name: "Bookings"
}
]
end

def get_module_state(module_id : String, lookup : String? = nil)
now = Time.local
start = now.at_beginning_of_day.to_unix
ending = now.at_end_of_day.to_unix

{
bookings: [{
event_start: start,
event_end: ending,
id: "booking-1",
host: "[email protected]",
title: "Test in #{module_id}"
}].to_json
}
end
end
78 changes: 78 additions & 0 deletions drivers/place/staff_api.cr
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,26 @@ class Place::StaffAPI < PlaceOS::Driver
end
end

def systems(q : String? = nil,
limit : Int32 = 1000,
offset : Int32 = 0,
zone_id : String? = nil,
module_id : String? = nil,
features : String? = nil,
capacity : Int32? = nil,
bookable : Bool? = nil)
placeos_client.systems.search(
q: q,
limit: limit,
offset: offset,
zone_id: zone_id,
module_id: module_id,
features: features,
capacity: capacity,
bookable: bookable
)
end

# Staff details returns the information from AD
def staff_details(email : String)
response = get("/api/staff/v1/people/#{email}", headers: {
Expand Down Expand Up @@ -163,6 +183,64 @@ class Place::StaffAPI < PlaceOS::Driver
)
end

# ===================================
# MODULE INFORMATION
# ===================================
def module(module_id : String)
response = get("/api/engine/v2/modules/#{module_id}", headers: {
"Accept" => "application/json",
"Authorization" => "Bearer #{token}",
})

raise "unexpected response for module id #{module_id}: #{response.status_code}\n#{response.body}" unless response.success?

begin
JSON.parse(response.body)
rescue error
logger.debug { "issue parsing module #{module_id}:\n#{response.body.inspect}" }
raise error
end
end

def modules_from_system(system_id : String)
response = get("/api/engine/v2/modules?control_system_id=#{system_id}", headers: {
"Accept" => "application/json",
"Authorization" => "Bearer #{token}",
})

raise "unexpected response for modules for #{system_id}: #{response.status_code}\n#{response.body}" unless response.success?

begin
JSON.parse(response.body)
rescue error
logger.debug { "issue getting modules for #{system_id}:\n#{response.body.inspect}" }
raise error
end
end

def get_module_state(module_id : String, lookup : String? = nil)
placeos_client.modules.state(module_id, lookup)
end

# TODO: figure out why these 2 methods don't work
# def module(module_id : String)
# placeos_client.modules.fetch module_id
# end

# def modules(q : String? = nil,
# limit : Int32 = 20,
# offset : Int32 = 0,
# control_system_id : String? = nil,
# driver_id : String? = nil)
# placeos_client.modules.search(
# q: q,
# limit: limit,
# offset: offset,
# control_system_id: control_system_id,
# driver_id: driver_id
# )
# end

# ===================================
# BOOKINGS ACTIONS
# ===================================
Expand Down