Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Writeup? #4

Open
DataStrategist opened this issue Apr 21, 2021 · 3 comments
Open

Writeup? #4

DataStrategist opened this issue Apr 21, 2021 · 3 comments

Comments

@DataStrategist
Copy link

Hola Lluis. Divertidisimo tu bot!

I was exactly about to build something like this when I came accross this repo. I was wondering, have you written up a blogpost or something where you give a few more details about how you have operationalized this (is it a VM or serverless, what were your thoughts on storing the tweets in the json file, etc). I'd be very interested!

Thanks!

@lluiscamino
Copy link
Owner

Hola @DataStrategist!

I haven't written a blog post about the bot. That's a great idea though, I should definitely try to do it. 😀

In respect of the operationalization of Archillect Context, it's currently hosted in a Google Cloud VM Instance. In the past, it was hosted in an AWS EC2 instance.

Regarding the storage of the tweets, I've tried different strategies over the life of the bot. The idea behind this is to be able to serve the tweets on a web application that lets users search by keyword.

Firstly, I used to store the tweets and the keywords in a relational database but I had to stop using this strategy due to the elevated costs of running a PostgreSQL database and a webserver to display the tweets.

Recently, I've been storing the tweets in different file formats (CSV and JSON). Currently, the web app that is supposed to serve the analyzed tweets (https://context.monster) is broken but I'm going to try using the JSON file to serve this content, although the "search by keyword" feature will probably be worse in terms of efficiency.

I hope that I've been able to explain myself well, let me know if you want to know anything else about the bot!

@DataStrategist
Copy link
Author

No that's perfect! Couple quick questions... if you already have a VM, why not install a simple db on that instance and serve it like that? Although indeed... this is a very small dataset, you can just check the csv files directly.

In terms of things to do... hrm... do u have a lot of the back-data? I like network analysis, if you want to collaborate we could make a big network of all the metadata for all the images and then display the pictures themselves within their position of the network? If you want to share the metadata, I could build the network using this: https://datastorm-open.github.io/visNetwork/image_icon.html. It wouldn't be live, but it could still be cool I think, no?

@lluiscamino
Copy link
Owner

That seems cool! I'll join the data from the different data sources that I have and make it public so that we can build the metadata network.

I will send you a message when I have all the data available 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants