-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Automatické postovanie do fóra. #7
Merged
Merged
Changes from all commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
9bcbfd9
Spec helper is required in .rspec config. Removing require from specs.
martinsabo cea5844
Discourse publisher draft.
martinsabo 6412f1b
Discourse client.
martinsabo dcfd7bd
Style fixes.
martinsabo 7e3cb8c
Refactoring of notifiers.
martinsabo b788a24
New namespacing. Abstract type gem. Lazy detail parsing.
martinsabo 083d9c8
Style fix.
martinsabo dfcd0f0
Refactoring iteration.
martinsabo bffdcb3
Style fix.
martinsabo 9aa28ca
Require fixes. Refactored error handling and constructor signatures.
martinsabo a9921e8
Removed notifications module file.
martinsabo 29af179
Details parsing should return nil when failed.
martinsabo 9cdbc24
Category loading from ENV.
martinsabo 0c3eacd
Removed unneeded require.
martinsabo File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,26 +1,10 @@ | ||
class Uvobot | ||
def initialize(notifier, scraper) | ||
@notifier = notifier | ||
@scraper = scraper | ||
end | ||
|
||
def run(release_date) | ||
if @scraper.issue_ready?(release_date) | ||
notify_announcements(release_date) | ||
else | ||
@notifier.new_issue_not_published | ||
end | ||
end | ||
|
||
private | ||
|
||
def notify_announcements(release_date) | ||
page_info, announcements = @scraper.get_announcements(release_date) | ||
|
||
if announcements.count > 0 | ||
@notifier.matching_announcements_found(page_info, announcements) | ||
else | ||
@notifier.no_announcements_found | ||
end | ||
end | ||
require_relative 'uvobot/worker' | ||
require_relative 'uvobot/uvo_scraper' | ||
require_relative 'uvobot/uvo_parser' | ||
require_relative 'uvobot/discourse_client' | ||
require_relative 'uvobot/notifications/discourse_notifier' | ||
require_relative 'uvobot/notifications/slack_notifier' | ||
require_relative 'uvobot/worker' | ||
|
||
module Uvobot | ||
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
require 'discourse_api' | ||
|
||
module Uvobot | ||
class DiscourseClient | ||
def initialize(host, api_key, api_username) | ||
@client = DiscourseApi::Client.new(host, api_key, api_username) | ||
end | ||
|
||
def create_topic(args = {}) | ||
@client.create_topic(args) | ||
rescue DiscourseApi::Error | ||
return nil | ||
end | ||
end | ||
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,54 @@ | ||
require_relative 'notifier' | ||
|
||
module Uvobot | ||
module Notifications | ||
class DiscourseNotifier < Notifier | ||
def initialize(discourse_client, category, scraper) | ||
@client = discourse_client | ||
@category = category | ||
@scraper = scraper | ||
end | ||
|
||
def no_announcements_found | ||
# noop | ||
end | ||
|
||
def new_issue_not_published | ||
# noop | ||
end | ||
|
||
def matching_announcements_found(_page_info, announcements) | ||
announcements.each do |a| | ||
topic = announcement_to_topic(a) | ||
@client.create_topic( | ||
title: topic[:title], | ||
raw: topic[:body], | ||
category: @category | ||
) | ||
end | ||
end | ||
|
||
private | ||
|
||
def announcement_to_topic(announcement) | ||
detail = @scraper.get_announcement_detail(announcement[:link][:href]) | ||
|
||
{ | ||
title: announcement[:procurement_subject].to_s, | ||
body: ["**Obstarávateľ:** #{announcement[:procurer]} ", | ||
"**Predmet obstarávania:** #{announcement[:procurement_subject]} ", | ||
detail_message(detail), | ||
"**Zdroj:** [#{announcement[:link][:text]}](#{announcement[:link][:href]})"].join("\n") | ||
} | ||
end | ||
|
||
def detail_message(detail) | ||
if detail | ||
"**Cena:** #{detail[:amount]} EUR " | ||
else | ||
"**Detaily sa nepodarilo extrahovať.** " | ||
end | ||
end | ||
end | ||
end | ||
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
require 'abstract_type' | ||
|
||
module Uvobot | ||
module Notifications | ||
class Notifier | ||
include AbstractType | ||
|
||
abstract_method :matching_announcements_found | ||
abstract_method :no_announcements_found | ||
abstract_method :new_issue_not_published | ||
end | ||
end | ||
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
require 'json' | ||
require 'curb' | ||
require_relative 'notifier' | ||
|
||
module Uvobot | ||
module Notifications | ||
class SlackNotifier < Notifier | ||
def initialize(slack_webhook, http_client = Curl) | ||
@slack_webhook = slack_webhook | ||
@http_client = http_client | ||
end | ||
|
||
def new_issue_not_published | ||
send_message('*Fíha, dnes na ÚVO nevyšlo nové vydanie vestníka?*') | ||
end | ||
|
||
def matching_announcements_found(page, announcements) | ||
send_message("Našiel som niečo nové na ÚVO! (#{page})") | ||
|
||
announcements.each do |a| | ||
send_message("<#{a[:link][:href]}|#{a[:link][:text]}>: *#{a[:procurer]}* #{a[:procurement_subject]}") | ||
end | ||
end | ||
|
||
def no_announcements_found | ||
send_message('Dnes som nenašiel žiadne nové IT zákazky.') | ||
end | ||
|
||
private | ||
|
||
def send_message(text) | ||
@http_client.post(@slack_webhook, payload(text)) | ||
end | ||
|
||
def payload(text) | ||
{ | ||
text: text, | ||
channel: '#general', | ||
username: 'uvobot', | ||
icon_emoji: ':mag_right:' | ||
}.to_json | ||
end | ||
end | ||
end | ||
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
require 'nokogiri' | ||
|
||
module Uvobot | ||
class UvoParser | ||
def self.parse_announcements(html) | ||
announcements = [] | ||
|
||
doc(html).css('.oznamenie').each do |a| | ||
link = a.css('.ozn1 a').first | ||
procurer = a.css('.ozn2').text.strip | ||
procurement_subject = a.css('.ozn3').text.strip | ||
|
||
announcements << { | ||
link: { text: link.text, href: link['href'] }, | ||
procurer: procurer, | ||
procurement_subject: procurement_subject | ||
} | ||
end | ||
announcements | ||
end | ||
|
||
def self.parse_detail(html) | ||
# there are multiple formats of detail page, this method does not handle them all for now | ||
detail = {} | ||
h_doc = doc(html) | ||
amount_node = h_doc.xpath('//div[text()="Hodnota "]').css('span').first | ||
return nil if amount_node.nil? | ||
|
||
detail[:amount] = amount_node.text | ||
detail | ||
end | ||
|
||
def self.parse_page_info(html) | ||
doc(html).css('.search-results').first.text.strip | ||
end | ||
|
||
def self.parse_issue_header(html) | ||
doc(html).css('h1')[1].text | ||
end | ||
|
||
def self.doc(html) | ||
Nokogiri::HTML(html) | ||
end | ||
end | ||
end |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tu mi chyba to osetrenie, ze ked je detail
nil
= nepodarilo sa ho extrahovat tak to vytvori topic s linkom ale default spravou, ze k tomu obstarku sa nepodarilo nic viac vytiahnut.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Je to tam ponate tak, ze metoda detail_message da namiesto ceny spravu o nepodarenej extrakcii. A link na detail/zdroj obstaravania, maju hned pod tym.
To fakt chces tvorit zvlast topic s oznamom ze sa nepodarilo vytiahnut cenu pre kazde obstaravanie co ma iny format?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Inak, ak sa nepodari vytiahnut ziadne data tak by to malo vytvorit stale topic. Ak sa nepodari len cenu tak je to v pohode tak ako to je teraz.
EDIT: Ah! uz to vidim. Dobre. Nic.