Allow to search for URLs in Hoaxy mode

[glciampaglia]

From @filmenczer on May 5, 2018 0:6

When searching for a URL (eg, one of the URLs in the dashboard) using Hoaxy search, the top article selected by default is NOT necessarily the one with the queried URL. This is a problem as it is misleading to the user. Can this be fixed?

Copied from original issue: IUNetSci/hoaxy-botometer#244

[glciampaglia]

As far as I remember (and @shaochengcheng please correct me if I'm wrong), the URL was never an indexed field in Lucene. I think the only fields we used were the contents of the <TITLE></TITLE> tag, and the article body.

I have tried to search by URL in the past, and sometimes it works and sometimes it doesn't. I think that what is happening in those cases is that somehow Lucene is able to treat parts of the URL as if they were keywords.

We would need to see if we can add the URLs as a field to the index.

[glciampaglia]

From @filmenczer on May 5, 2018 22:58

Unless we can add this functionality in a very short time (I doubt it), I think we should remove the "Search Link" altogether (at least in Hoaxy search). It is misleading. The search by title should work as intended.

For Twitter search, I think it should be okay to keep both.

[glciampaglia]

From @filmenczer on May 6, 2018 16:46

@mihaivavram, could you please remove the "Search Link" links just under "Popular Claims" and "Popular Fact-Checks" in the homepage dashboard? Thanks!

[glciampaglia]

From @mihaivavram on May 6, 2018 16:48

No problem, I will do this.

On Sun, May 6, 2018 at 12:46 PM, filmenczer [email protected]
wrote:

@mihaivavram https://github.com/mihaivavram, could you please remove
the "Search Link" links just under "Popular Claims" and "Popular
Fact-Checks" in the homepage dashboard? Thanks!

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/IUNetSci/hoaxy-botometer/issues/244#issuecomment-386892680,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AXpgRwaJGk_mmSs1yRTMbJR2aSnG2AtVks5tvykBgaJpZM4TzaWp
.

[glciampaglia]

From @mihaivavram on May 6, 2018 16:53

Request from @filmenczer is done, removing this from Launch and adding to Utopia.

[marcmccarty]

Maybe the Python3 fixes or using different news scrapers fixed this, but I tried this with an article on the front page (https://www.theblaze.com/news/networks-will-snub-trump-salute-to-america) and it worked.

I'll close this, but we can reopen if there's discussion needed or if someone finds a counter-example.

[filmenczer]

It works in some cases, not in others (as Giovanni wrote in a previous comment). For example if you try this other example from the front page (https://www.breitbart.com/economy/2019/07/05/america-created-224000-jobs-in-june/), it fails. Reopening.

[benabus]

This worked fine for me: https://hoaxy.iuni.iu.edu/#query=https%3A%2F%2Fwww.breitbart.com%2Feconomy%2F2019%2F07%2F05%2Famerica-created-224000-jobs-in-june%2F&sort=relevant&type=Hoaxy

[filmenczer]

As you can read in the issue description, the problem is not that it does not return something. The problem is that what it returns does not match the URL article. If you click on the button "1 article visualized" you will see that the top article is something else from Infowars, not the Breitbart article.

[marcmccarty]

@benabus Check my example. The URL you searched isn't the article visualized in the list (where there's like 20). I'm not sure why mine worked but not that one, though. At least not yet.

[benabus]

My mistake. I only read the issue title.