Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatické postovanie do fóra. #7

Merged
merged 14 commits into from
Feb 4, 2016
Merged
2 changes: 2 additions & 0 deletions Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ source 'https://rubygems.org'
gem 'nokogiri'
gem 'curb'
gem 'dotenv'
gem 'discourse_api'
gem 'abstract_type'

group :development do
gem 'rspec'
Expand Down
13 changes: 13 additions & 0 deletions Gemfile.lock
Original file line number Diff line number Diff line change
@@ -1,20 +1,31 @@
GEM
remote: https://rubygems.org/
specs:
abstract_type (0.0.7)
ast (2.2.0)
codeclimate-test-reporter (0.4.7)
simplecov (>= 0.7.1, < 1.0.0)
curb (0.9.1)
diff-lcs (1.2.5)
discourse_api (0.7.0)
faraday (~> 0.9.0)
faraday_middleware (~> 0.9)
rack (~> 1.5)
docile (1.1.5)
dotenv (2.1.0)
faraday (0.9.2)
multipart-post (>= 1.2, < 3)
faraday_middleware (0.10.0)
faraday (>= 0.7.4, < 0.10)
json (1.8.3)
mini_portile2 (2.0.0)
multipart-post (2.0.0)
nokogiri (1.6.7.2)
mini_portile2 (~> 2.0.0.rc2)
parser (2.3.0.2)
ast (~> 2.2)
powerpack (0.1.1)
rack (1.6.4)
rainbow (2.1.0)
rake (10.4.2)
rspec (3.4.0)
Expand Down Expand Up @@ -46,8 +57,10 @@ PLATFORMS
ruby

DEPENDENCIES
abstract_type
codeclimate-test-reporter
curb
discourse_api
dotenv
nokogiri
rake
Expand Down
40 changes: 0 additions & 40 deletions lib/slack_notifier.rb

This file was deleted.

32 changes: 0 additions & 32 deletions lib/uvo_parser.rb

This file was deleted.

28 changes: 0 additions & 28 deletions lib/uvo_scraper.rb

This file was deleted.

34 changes: 9 additions & 25 deletions lib/uvobot.rb
Original file line number Diff line number Diff line change
@@ -1,26 +1,10 @@
class Uvobot
def initialize(notifier, scraper)
@notifier = notifier
@scraper = scraper
end

def run(release_date)
if @scraper.issue_ready?(release_date)
notify_announcements(release_date)
else
@notifier.new_issue_not_published
end
end

private

def notify_announcements(release_date)
page_info, announcements = @scraper.get_announcements(release_date)

if announcements.count > 0
@notifier.matching_announcements_found(page_info, announcements)
else
@notifier.no_announcements_found
end
end
require_relative 'uvobot/worker'
require_relative 'uvobot/uvo_scraper'
require_relative 'uvobot/uvo_parser'
require_relative 'uvobot/discourse_client'
require_relative 'uvobot/notifications/discourse_notifier'
require_relative 'uvobot/notifications/slack_notifier'
require_relative 'uvobot/worker'

module Uvobot
end
15 changes: 15 additions & 0 deletions lib/uvobot/discourse_client.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
require 'discourse_api'

module Uvobot
class DiscourseClient
def initialize(host, api_key, api_username)
@client = DiscourseApi::Client.new(host, api_key, api_username)
end

def create_topic(args = {})
@client.create_topic(args)
rescue DiscourseApi::Error
return nil
end
end
end
55 changes: 55 additions & 0 deletions lib/uvobot/notifications/discourse_notifier.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
require_relative 'notifier'
require_relative '../uvo_scraper'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Toto tu uz byt nemusi.


module Uvobot
module Notifications
class DiscourseNotifier < Notifier
def initialize(discourse_client, category, scraper)
@client = discourse_client
@category = category
@scraper = scraper
end

def no_announcements_found
# noop
end

def new_issue_not_published
# noop
end

def matching_announcements_found(_page_info, announcements)
announcements.each do |a|
topic = announcement_to_topic(a)
@client.create_topic(
title: topic[:title],
raw: topic[:body],
category: @category
)
end
end

private

def announcement_to_topic(announcement)
detail = @scraper.get_announcement_detail(announcement[:link][:href])

{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tu mi chyba to osetrenie, ze ked je detail nil = nepodarilo sa ho extrahovat tak to vytvori topic s linkom ale default spravou, ze k tomu obstarku sa nepodarilo nic viac vytiahnut.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Je to tam ponate tak, ze metoda detail_message da namiesto ceny spravu o nepodarenej extrakcii. A link na detail/zdroj obstaravania, maju hned pod tym.

To fakt chces tvorit zvlast topic s oznamom ze sa nepodarilo vytiahnut cenu pre kazde obstaravanie co ma iny format?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inak, ak sa nepodari vytiahnut ziadne data tak by to malo vytvorit stale topic. Ak sa nepodari len cenu tak je to v pohode tak ako to je teraz.

EDIT: Ah! uz to vidim. Dobre. Nic.

title: announcement[:procurement_subject].to_s,
body: ["**Obstarávateľ:** #{announcement[:procurer]} ",
"**Predmet obstarávania:** #{announcement[:procurement_subject]} ",
detail_message(detail),
"**Zdroj:** [#{announcement[:link][:text]}](#{announcement[:link][:href]})"].join("\n")
}
end

def detail_message(detail)
if detail
"**Cena:** #{detail[:amount]} EUR "
else
"**Detaily sa nepodarilo extrahovať.** "
end
end
end
end
end
13 changes: 13 additions & 0 deletions lib/uvobot/notifications/notifier.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
require 'abstract_type'

module Uvobot
module Notifications
class Notifier
include AbstractType

abstract_method :matching_announcements_found
abstract_method :no_announcements_found
abstract_method :new_issue_not_published
end
end
end
45 changes: 45 additions & 0 deletions lib/uvobot/notifications/slack_notifier.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
require 'json'
require 'curb'
require_relative 'notifier'

module Uvobot
module Notifications
class SlackNotifier < Notifier
def initialize(slack_webhook, http_client = Curl)
@slack_webhook = slack_webhook
@http_client = http_client
end

def new_issue_not_published
send_message('*Fíha, dnes na ÚVO nevyšlo nové vydanie vestníka?*')
end

def matching_announcements_found(page, announcements)
send_message("Našiel som niečo nové na ÚVO! (#{page})")

announcements.each do |a|
send_message("<#{a[:link][:href]}|#{a[:link][:text]}>: *#{a[:procurer]}* #{a[:procurement_subject]}")
end
end

def no_announcements_found
send_message('Dnes som nenašiel žiadne nové IT zákazky.')
end

private

def send_message(text)
@http_client.post(@slack_webhook, payload(text))
end

def payload(text)
{
text: text,
channel: '#general',
username: 'uvobot',
icon_emoji: ':mag_right:'
}.to_json
end
end
end
end
45 changes: 45 additions & 0 deletions lib/uvobot/uvo_parser.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
require 'nokogiri'

module Uvobot
class UvoParser
def self.parse_announcements(html)
announcements = []

doc(html).css('.oznamenie').each do |a|
link = a.css('.ozn1 a').first
procurer = a.css('.ozn2').text.strip
procurement_subject = a.css('.ozn3').text.strip

announcements << {
link: { text: link.text, href: link['href'] },
procurer: procurer,
procurement_subject: procurement_subject
}
end
announcements
end

def self.parse_detail(html)
# there are multiple formats of detail page, this method does not handle them all for now
detail = {}
h_doc = doc(html)
amount_node = h_doc.xpath('//div[text()="Hodnota "]').css('span').first
return nil if amount_node.nil?

detail[:amount] = amount_node.text
detail
end

def self.parse_page_info(html)
doc(html).css('.search-results').first.text.strip
end

def self.parse_issue_header(html)
doc(html).css('h1')[1].text
end

def self.doc(html)
Nokogiri::HTML(html)
end
end
end
Loading