-
Notifications
You must be signed in to change notification settings - Fork 1
A tiny FTP crawler and indexer with a web UI
License
kisscool/tiny-garbage
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
== Tiny-Garbage Note : You should check the successors of Tiny-Garbage : Tiny-Garbage2 for MongoDB (http://github.com/kisscool/tiny-garbage2) and Tiny-Garbage3 for Redis (http://github.com/kisscool/tiny-garbage3). Just a little FTP crawler, with a little Sinatra based search interface in order to browse the FTP index. Nothing else, nothing more. == Dependencies Required gems are : - dm-core - dm-migrations - dm-is-tree - dm-aggregates - dm-sqlite-adapter (or dm-mysql-adapter, depending on your need) - sinatra - haml - sass - net-ftp-list - Rack If you have Bundler installed on your system, you can track down all those dependencies by launching the following command from inside the project directory : $ bundle install --path vendor If you want only MySQL dependencies, you can use : * bundle install --path vendor --without sqlite (idem if you do want only sqlite dependencies) == Install = The crawler part Do a clone of the project git repository, install the missing dependencies then create the configuration file by copying config.rb.sample as config.rb and edit it to suit your taste. Configure your crontab to launch periodically the following commands : * "$path_to_project/scripts/crawler.rb ping" (with a recommenced frequency of every 10 minutes) * "$path_to_project/scripts/crawler.rb index" (with a recommended frequency of once a day) = The Web UI part For a first try you can just launch "rackup config.ru" in order to test the web UI with the pure ruby Webrick server. If you want to deploy it in production, you will want to check Unicorn, Thin or Phusion Passenger documentations for more solids options. == Thanks Thanks to Loic Gomez for the first Garbage Collector, which was so useful back in the days, and thanks to Zouchaoqun's ezFtpSearch (http://github.com/zouchaoqun/ezftpsearch) whose model bootstrapped our work.
About
A tiny FTP crawler and indexer with a web UI
Resources
License
Stars
Watchers
Forks
Packages 0
No packages published