-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Automatické postovanie do fóra. #7
Changes from 9 commits
9bcbfd9
cea5844
6412f1b
dcfd7bd
7e3cb8c
b788a24
083d9c8
dfcd0f0
bffdcb3
9aa28ca
a9921e8
29af179
9cdbc24
0c3eacd
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
This file was deleted.
This file was deleted.
This file was deleted.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,26 +1,9 @@ | ||
class Uvobot | ||
def initialize(notifier, scraper) | ||
@notifier = notifier | ||
@scraper = scraper | ||
end | ||
|
||
def run(release_date) | ||
if @scraper.issue_ready?(release_date) | ||
notify_announcements(release_date) | ||
else | ||
@notifier.new_issue_not_published | ||
end | ||
end | ||
|
||
private | ||
|
||
def notify_announcements(release_date) | ||
page_info, announcements = @scraper.get_announcements(release_date) | ||
|
||
if announcements.count > 0 | ||
@notifier.matching_announcements_found(page_info, announcements) | ||
else | ||
@notifier.no_announcements_found | ||
end | ||
end | ||
require_relative './uvobot/worker' | ||
require_relative './uvobot/uvo_scraper' | ||
require_relative './uvobot/uvo_parser' | ||
require_relative './uvobot/discourse_client' | ||
require_relative './uvobot/notifications' | ||
require_relative './uvobot/worker' | ||
|
||
module Uvobot | ||
end |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
require 'discourse_api' | ||
|
||
module Uvobot | ||
class DiscourseClient | ||
def initialize(host, api_key = nil, api_username = nil) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. nie su There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Pravda. Po prerabke to malo byt povinne. |
||
@client = DiscourseApi::Client.new(host, api_key, api_username) | ||
end | ||
|
||
def create_topic(args = {}) | ||
@client.create_topic(args) | ||
rescue DiscourseApi::Error => e | ||
# puts e.message | ||
return nil | ||
end | ||
end | ||
end |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
require_relative 'notifications/notifier' | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Cely tento subor There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Moze. Pouzivam to na skupinovy require, ked rozpisem jednotlive notifiers nebude ho treba. |
||
require_relative 'notifications/slack_notifier' | ||
require_relative 'notifications/discourse_notifier' | ||
|
||
module Uvobot | ||
module Notifications | ||
end | ||
end |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,46 @@ | ||
require_relative './notifier' | ||
require_relative '../uvo_scraper' | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Toto tu uz byt nemusi. |
||
|
||
module Uvobot | ||
module Notifications | ||
class DiscourseNotifier < Notifier | ||
def initialize(discourse_client, category = 'Štátne projekty', scraper = Uvobot::UvoScraper.new) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Tu by som v klude tie default parametre dal prec a pleskol ich do top-levelu. Nech to je explicitne. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ok. |
||
@client = discourse_client | ||
@category = category | ||
@scraper = scraper | ||
end | ||
|
||
def no_announcements_found | ||
# noop | ||
end | ||
|
||
def new_issue_not_published | ||
# noop | ||
end | ||
|
||
def matching_announcements_found(_page_info, announcements) | ||
announcements.each do |a| | ||
topic = announcement_to_topic(a) | ||
@client.create_topic( | ||
title: topic[:title], | ||
raw: topic[:body], | ||
category: @category | ||
) | ||
end | ||
end | ||
|
||
private | ||
|
||
def announcement_to_topic(announcement) | ||
detail = @scraper.get_announcement_detail(announcement[:link][:href]) | ||
{ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Tu mi chyba to osetrenie, ze ked je detail There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Je to tam ponate tak, ze metoda detail_message da namiesto ceny spravu o nepodarenej extrakcii. A link na detail/zdroj obstaravania, maju hned pod tym. To fakt chces tvorit zvlast topic s oznamom ze sa nepodarilo vytiahnut cenu pre kazde obstaravanie co ma iny format? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Inak, ak sa nepodari vytiahnut ziadne data tak by to malo vytvorit stale topic. Ak sa nepodari len cenu tak je to v pohode tak ako to je teraz. EDIT: Ah! uz to vidim. Dobre. Nic. |
||
title: announcement[:procurement_subject].to_s, | ||
body: ["**Obstarávateľ:** #{announcement[:procurer]} ", | ||
"**Predmet obstarávania:** #{announcement[:procurement_subject]} ", | ||
"**Cena:** #{detail[:amount]} EUR ", | ||
"**Zdroj:** [#{announcement[:link][:text]}](#{announcement[:link][:href]})"].join("\n") | ||
} | ||
end | ||
end | ||
end | ||
end |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
require 'abstract_type' | ||
|
||
module Uvobot | ||
module Notifications | ||
class Notifier | ||
include AbstractType | ||
|
||
abstract_method :matching_announcements_found | ||
abstract_method :no_announcements_found | ||
abstract_method :new_issue_not_published | ||
end | ||
end | ||
end |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
require 'json' | ||
require 'curb' | ||
require_relative './notifier' | ||
|
||
module Uvobot | ||
module Notifications | ||
class SlackNotifier < Notifier | ||
def initialize(slack_webhook, http_client = Curl) | ||
@slack_webhook = slack_webhook | ||
@http_client = http_client | ||
end | ||
|
||
def new_issue_not_published | ||
send_message('*Fíha, dnes na ÚVO nevyšlo nové vydanie vestníka?*') | ||
end | ||
|
||
def matching_announcements_found(page, announcements) | ||
send_message("Našiel som niečo nové na ÚVO! (#{page})") | ||
|
||
announcements.each do |a| | ||
send_message("<#{a[:link][:href]}|#{a[:link][:text]}>: *#{a[:procurer]}* #{a[:procurement_subject]}") | ||
end | ||
end | ||
|
||
def no_announcements_found | ||
send_message('Dnes som nenašiel žiadne nové IT zákazky.') | ||
end | ||
|
||
private | ||
|
||
def send_message(text) | ||
@http_client.post(@slack_webhook, payload(text)) | ||
end | ||
|
||
def payload(text) | ||
{ | ||
text: text, | ||
channel: '#general', | ||
username: 'uvobot', | ||
icon_emoji: ':mag_right:' | ||
}.to_json | ||
end | ||
end | ||
end | ||
end |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,42 @@ | ||
require 'nokogiri' | ||
|
||
module Uvobot | ||
class UvoParser | ||
def self.parse_announcements(html) | ||
announcements = [] | ||
|
||
doc(html).css('.oznamenie').each do |a| | ||
link = a.css('.ozn1 a').first | ||
procurer = a.css('.ozn2').text.strip | ||
procurement_subject = a.css('.ozn3').text.strip | ||
|
||
announcements << { | ||
link: { text: link.text, href: link['href'] }, | ||
procurer: procurer, | ||
procurement_subject: procurement_subject | ||
} | ||
end | ||
announcements | ||
end | ||
|
||
def self.parse_detail(html) | ||
detail = { amount: 'Extrakcia sa nepodarila.' } | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sumu sa nepodarilo automaticky ziskat. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ok. |
||
h_doc = doc(html) | ||
# unstable, there are multiple formats of detail page | ||
detail[:amount] = h_doc.xpath('//div[text()="Hodnota "]').css('span').first.text | ||
detail | ||
end | ||
|
||
def self.parse_page_info(html) | ||
doc(html).css('.search-results').first.text.strip | ||
end | ||
|
||
def self.parse_issue_header(html) | ||
doc(html).css('h1')[1].text | ||
end | ||
|
||
def self.doc(html) | ||
Nokogiri::HTML(html) | ||
end | ||
end | ||
end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Toto
./uvobot/worker
nepoznam, ja pouzivamrequire_relative 'uvobot/worker'
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nechapem. Nepaci sa ti ze tam volam cely namespace naraz, alebo ze tam je bodka v ceste?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ta bodka, podla mna to je zbytocne.