Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatické postovanie do fóra. #7

Merged
merged 14 commits into from
Feb 4, 2016
Merged
2 changes: 2 additions & 0 deletions Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ source 'https://rubygems.org'
gem 'nokogiri'
gem 'curb'
gem 'dotenv'
gem 'discourse_api'
gem 'abstract_type'

group :development do
gem 'rspec'
Expand Down
13 changes: 13 additions & 0 deletions Gemfile.lock
Original file line number Diff line number Diff line change
@@ -1,20 +1,31 @@
GEM
remote: https://rubygems.org/
specs:
abstract_type (0.0.7)
ast (2.2.0)
codeclimate-test-reporter (0.4.7)
simplecov (>= 0.7.1, < 1.0.0)
curb (0.9.1)
diff-lcs (1.2.5)
discourse_api (0.7.0)
faraday (~> 0.9.0)
faraday_middleware (~> 0.9)
rack (~> 1.5)
docile (1.1.5)
dotenv (2.1.0)
faraday (0.9.2)
multipart-post (>= 1.2, < 3)
faraday_middleware (0.10.0)
faraday (>= 0.7.4, < 0.10)
json (1.8.3)
mini_portile2 (2.0.0)
multipart-post (2.0.0)
nokogiri (1.6.7.2)
mini_portile2 (~> 2.0.0.rc2)
parser (2.3.0.2)
ast (~> 2.2)
powerpack (0.1.1)
rack (1.6.4)
rainbow (2.1.0)
rake (10.4.2)
rspec (3.4.0)
Expand Down Expand Up @@ -46,8 +57,10 @@ PLATFORMS
ruby

DEPENDENCIES
abstract_type
codeclimate-test-reporter
curb
discourse_api
dotenv
nokogiri
rake
Expand Down
40 changes: 0 additions & 40 deletions lib/slack_notifier.rb

This file was deleted.

32 changes: 0 additions & 32 deletions lib/uvo_parser.rb

This file was deleted.

28 changes: 0 additions & 28 deletions lib/uvo_scraper.rb

This file was deleted.

33 changes: 8 additions & 25 deletions lib/uvobot.rb
Original file line number Diff line number Diff line change
@@ -1,26 +1,9 @@
class Uvobot
def initialize(notifier, scraper)
@notifier = notifier
@scraper = scraper
end

def run(release_date)
if @scraper.issue_ready?(release_date)
notify_announcements(release_date)
else
@notifier.new_issue_not_published
end
end

private

def notify_announcements(release_date)
page_info, announcements = @scraper.get_announcements(release_date)

if announcements.count > 0
@notifier.matching_announcements_found(page_info, announcements)
else
@notifier.no_announcements_found
end
end
require_relative './uvobot/worker'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Toto ./uvobot/worker nepoznam, ja pouzivam require_relative 'uvobot/worker'

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nechapem. Nepaci sa ti ze tam volam cely namespace naraz, alebo ze tam je bodka v ceste?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ta bodka, podla mna to je zbytocne.

require_relative './uvobot/uvo_scraper'
require_relative './uvobot/uvo_parser'
require_relative './uvobot/discourse_client'
require_relative './uvobot/notifications'
require_relative './uvobot/worker'

module Uvobot
end
16 changes: 16 additions & 0 deletions lib/uvobot/discourse_client.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
require 'discourse_api'

module Uvobot
class DiscourseClient
def initialize(host, api_key = nil, api_username = nil)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nie su api_key a api_username povinne?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pravda. Po prerabke to malo byt povinne.

@client = DiscourseApi::Client.new(host, api_key, api_username)
end

def create_topic(args = {})
@client.create_topic(args)
rescue DiscourseApi::Error => e
# puts e.message
return nil
end
end
end
8 changes: 8 additions & 0 deletions lib/uvobot/notifications.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
require_relative 'notifications/notifier'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cely tento subor notifiactions.rb podla mna moze ist prec.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moze. Pouzivam to na skupinovy require, ked rozpisem jednotlive notifiers nebude ho treba.

require_relative 'notifications/slack_notifier'
require_relative 'notifications/discourse_notifier'

module Uvobot
module Notifications
end
end
46 changes: 46 additions & 0 deletions lib/uvobot/notifications/discourse_notifier.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
require_relative './notifier'
require_relative '../uvo_scraper'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Toto tu uz byt nemusi.


module Uvobot
module Notifications
class DiscourseNotifier < Notifier
def initialize(discourse_client, category = 'Štátne projekty', scraper = Uvobot::UvoScraper.new)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tu by som v klude tie default parametre dal prec a pleskol ich do top-levelu. Nech to je explicitne.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok.

@client = discourse_client
@category = category
@scraper = scraper
end

def no_announcements_found
# noop
end

def new_issue_not_published
# noop
end

def matching_announcements_found(_page_info, announcements)
announcements.each do |a|
topic = announcement_to_topic(a)
@client.create_topic(
title: topic[:title],
raw: topic[:body],
category: @category
)
end
end

private

def announcement_to_topic(announcement)
detail = @scraper.get_announcement_detail(announcement[:link][:href])
{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tu mi chyba to osetrenie, ze ked je detail nil = nepodarilo sa ho extrahovat tak to vytvori topic s linkom ale default spravou, ze k tomu obstarku sa nepodarilo nic viac vytiahnut.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Je to tam ponate tak, ze metoda detail_message da namiesto ceny spravu o nepodarenej extrakcii. A link na detail/zdroj obstaravania, maju hned pod tym.

To fakt chces tvorit zvlast topic s oznamom ze sa nepodarilo vytiahnut cenu pre kazde obstaravanie co ma iny format?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inak, ak sa nepodari vytiahnut ziadne data tak by to malo vytvorit stale topic. Ak sa nepodari len cenu tak je to v pohode tak ako to je teraz.

EDIT: Ah! uz to vidim. Dobre. Nic.

title: announcement[:procurement_subject].to_s,
body: ["**Obstarávateľ:** #{announcement[:procurer]} ",
"**Predmet obstarávania:** #{announcement[:procurement_subject]} ",
"**Cena:** #{detail[:amount]} EUR ",
"**Zdroj:** [#{announcement[:link][:text]}](#{announcement[:link][:href]})"].join("\n")
}
end
end
end
end
13 changes: 13 additions & 0 deletions lib/uvobot/notifications/notifier.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
require 'abstract_type'

module Uvobot
module Notifications
class Notifier
include AbstractType

abstract_method :matching_announcements_found
abstract_method :no_announcements_found
abstract_method :new_issue_not_published
end
end
end
45 changes: 45 additions & 0 deletions lib/uvobot/notifications/slack_notifier.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
require 'json'
require 'curb'
require_relative './notifier'

module Uvobot
module Notifications
class SlackNotifier < Notifier
def initialize(slack_webhook, http_client = Curl)
@slack_webhook = slack_webhook
@http_client = http_client
end

def new_issue_not_published
send_message('*Fíha, dnes na ÚVO nevyšlo nové vydanie vestníka?*')
end

def matching_announcements_found(page, announcements)
send_message("Našiel som niečo nové na ÚVO! (#{page})")

announcements.each do |a|
send_message("<#{a[:link][:href]}|#{a[:link][:text]}>: *#{a[:procurer]}* #{a[:procurement_subject]}")
end
end

def no_announcements_found
send_message('Dnes som nenašiel žiadne nové IT zákazky.')
end

private

def send_message(text)
@http_client.post(@slack_webhook, payload(text))
end

def payload(text)
{
text: text,
channel: '#general',
username: 'uvobot',
icon_emoji: ':mag_right:'
}.to_json
end
end
end
end
42 changes: 42 additions & 0 deletions lib/uvobot/uvo_parser.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
require 'nokogiri'

module Uvobot
class UvoParser
def self.parse_announcements(html)
announcements = []

doc(html).css('.oznamenie').each do |a|
link = a.css('.ozn1 a').first
procurer = a.css('.ozn2').text.strip
procurement_subject = a.css('.ozn3').text.strip

announcements << {
link: { text: link.text, href: link['href'] },
procurer: procurer,
procurement_subject: procurement_subject
}
end
announcements
end

def self.parse_detail(html)
detail = { amount: 'Extrakcia sa nepodarila.' }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sumu sa nepodarilo automaticky ziskat.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok.

h_doc = doc(html)
# unstable, there are multiple formats of detail page
detail[:amount] = h_doc.xpath('//div[text()="Hodnota "]').css('span').first.text
detail
end

def self.parse_page_info(html)
doc(html).css('.search-results').first.text.strip
end

def self.parse_issue_header(html)
doc(html).css('h1')[1].text
end

def self.doc(html)
Nokogiri::HTML(html)
end
end
end
Loading