Skip to content

Sufia Management Guide

Jump to bottom
hackmastera edited this page May 4, 2017 · 35 revisions

The Sufia Management Guide provides tips for how to manage, customize, and enhance your Sufia application.

Feature matrix

Feature matrix (moved to its own page)

Production concerns

In production or production-like (e.g., staging) environments, you may want to make changes to the following areas.

Identifier state

Sufia uses the ActiveFedora::Noid gem to mint Noid-style identifiers -- short, opaque identifiers -- for all user-created content (including GenericWorks, FileSets, and Collections). The identifier minter is stateful, meaning that it keeps track of where it is in the sequence of minting identifiers so that the minter can be "replayed," for example in a disaster recovery scenario. (Read more about the technical details.) The state also means that the minter, once it has minted an identifier, will never mint it again so there's no risk of identifier collisions.

Identifier state is tracked by default in a file that is located in a well-known directory in UNIX-like environments, /tmp/, but this may be insufficient in production-like environments where /tmp/ may be aggressively cleaned out. To prevent the chance of identifier collisions, it is recommended that you go one of two routes: specifying a less transient shared filesystem location or using a relational database to store minter state.

Filesystem-backed minter state

You can continue using the filesystem for minter state if you have a system location that your application can write to that is under its control (read: not /tmp/). If you are deploying via Capistrano, that location should not be in your application directory, which will change on each deployment. If you run multiple instances of your Sufia application, for instance in load-balanced scenarios, you will want to choose a filesystem location that all instances can access. You may change this by uncommenting and changing the value in this line from config/initializers/sufia.rb to a filesystem location other than /tmp/:

# config.minter_statefile = '/tmp/minter-state'

Database-backed minter state

Alternatively, to get around situations where you have multiple application instances attempting to obtain locks on a single, shared filesystem location, you may use the database-backed minter (new to ActiveFedora::Noid 2.x), which stores minter state information in your application's relational database.

To use it, you'll first need to run the install generator:

$ rails generate active_fedora:noid:install

This will create the necessary database tables and seed the database minter. To start minting identifiers with the new minter, override the AF::Noid configuration in e.g config/initializers/active_fedora-noid.rb:

require 'active_fedora/noid'

ActiveFedora::Noid.configure do |config|
  config.minter_class = ActiveFedora::Noid::Minter::Db
end

Using the database-backed minter can cause problems with your test suite, where it is often sensible to wipe out database rows between tests (which destroys the database-backed minter's state, which renders it unusable). To deal with this and still get the benefits of using the database-backed minter in development and production environments, you'll also want to add the following helper to your spec/spec_helper.rb:

require 'active_fedora/noid/rspec'

RSpec.configure do |config|
  include ActiveFedora::Noid::RSpec

  config.before(:suite) { disable_production_minter! }
  config.after(:suite)  { enable_production_minter! }
end

If you switch to the new database-backed minter and want to include in that minter the state of your current file-backed minter, AF::Noid 2.x provides a new rake task that will copy your minter's state from the filesystem to the database:

$ rake active_fedora:noid:migrate:file_to_database

Derivatives

In Sufia 7, derivatives are served from a directory on the filesystem rather than directly from the Fedora repository. If your production environment includes multiple Rails servers, you will want to make sure that they are using a shared filesystem for derivatives. To set this directory, change the value of config.derivatives_path in config/initializers/sufia.rb. The default value is tmp/derivatives/ within your application directory, which will cause unexpected behavior in a multi-server configuration, or if you deploy with capistrano.

Web server

The web server provided by Rails (whether that's WEBrick, Unicorn, or another) is not built to scale out very far, so you should consider alternatives such as Passenger with Apache httpd or nginx.

Database

The database provided by default is SQLite, and you may wish to swap in something built more for scale like PostgreSQL or MySQL, both of which have been used in other production Sufia applications.

Mailers

Sufia uses ActionMailer to send email to users. Some environments may need special configuration to enable your application to send messages. These changes are best made in one of your application's environment files. The configuration options are documented in the ActionMailer Rails Guide.

Background workers

Background Workers (moved to its own page)

Fixity checking

Sufia provides a service that iterates over all file sets in your repository and verifies fixities in the background. Sufia will not run this service for you, so you should use a cronjob (or similar, e.g., the whenever gem) to run this on a schedule that fits your needs and your content. The code that'll need to run is:

Sufia::RepositoryAuditService.audit_everything

Audiovisual transcoding

Sufia includes support for transcoding audio and video files via CurationConcerns. View CurationConcerns README for installation/configuration help.

Removing a work type

If you need to remove a work type that you created, e.g., via rails generate sufia:work MyWorkType, you can do so via:

rails destroy sufia:work MyWorkType

User interface

If you encounter problems with the in-browser content editor -- e.g., the About page, and the three blocks on the homepage -- you should remove turbolinks support from app/assets/javascripts/application.js if present by deleting the following line:

//= require turbolinks

Integration with Dropbox, Box, etc.

Sufia provides built-in support for the browse-everything gem, which provides a consolidated file picker experience for selecting files from DropBox, Skydrive, Google Drive, Box, and a server-side directory share.

To activate browse-everything in your sufia app, run the browse-everything install generator

rails g browse_everything:install --skip-assets

This will generate a file at config/browse_everything_providers.yml. Open that file and enter the API keys for the providers that you want to support in your app. For more info on configuring browse-everything, go to the project page on github.

After running the browse-everything install generator and setting the API keys for the desired providers, an extra tab will appear in your app's Upload page allowing users to pick files from those providers and submit them into your app's repository.

If your config/initializers/sufia.rb was generated with sufia 3.7.2 or earlier, then you need to add this line to an initializer (probably _config/initializers/sufia.rb _):

config.browse_everything = BrowseEverything.config

Geonames

In order for autocomplete to work in the location field, you must have a valid geonames account that can query the service and return possible matches. To setup an account, visit: http://www.geonames.org/login.

After registering and getting an account username, you'll need to enable the free web services by going to http://www.geonames.org/manageaccount and clicking the link to enable. Once that's done, and you've verified you can query the service via their REST Api, add the account username to the config/initializers/sufia.rb file under config.geonames_username.

Analytics and usage statistics

Sufia provides support for capturing usage information via Google Analytics and for displaying usage stats in the UI.

Capturing usage

To enable the Google Analytics javascript snippet, make sure that config.google_analytics_id is set in your app within the config/initializers/sufia.rb file. A Google Analytics ID typically looks like UA-99999999-1.

Displaying usage in the UI

To display data from Google Analytics in the UI, first head to the Google Developers Console and create a new project:

https://console.developers.google.com/project

Let's assume for now Google assigns it a project ID of foo-bar-123. It may take a few seconds for this to complete (watch the Activities bar near the bottom of the browser). Once it's complete, enable the Google+ and Google Analytics APIs here (note: this is an example URL -- you'll have to change the project ID to match yours):

https://console.developers.google.com/apis/library?project=foo-bar-123

Finally, click the 'credentials' menu item and create a new Service Account Key. This will give you the client ID, a client email address, a private key file, and a private key secret/password, which you will need in the next step.

Then copy this config into your app at config/analytics.yml (don't commit it to a public repo):

https://github.com/projecthydra/sufia/blob/v7.3.1/lib/generators/sufia/templates/config/analytics.yml

Edit the file to reflect the information that the Google Developer Console gave you earlier.

Lastly, you will need to set config.analytics = true and config.analytic_start_date in config/initializers/sufia.rb and ensure that the client email has the proper access within your Google Analyics account. To do so, go to the Admin tab for your Google Analytics account. Click on User Management, in the Account column, and add "Read & Analyze" permissions for the OAuth client email address.

Zotero integration

Integration with Zotero-managed publications is possible using Arkivo. Arkivo is a Node-based Zotero subscription service that monitors Zotero for changes and will feed those changes to your Sufia-based app. Read more about this work.

To enable Zotero integration, first register an OAuth client with Zotero, then install and start Arkivo-Sufia and then generate the Arkivo API in your Sufia-based application:

rails g sufia:arkivo_api

The generator does the following:

  • Enables the API in the Sufia initializer
  • Adds a database migration
  • Creates a routing constraint that allows you to control what clients can access the API
  • Copies a config file that allows you to specify the host and port Arkivo is running on
  • Copies a config file for your Zotero OAuth client credentials

Update your database schema with rake db:migrate.

Add unique Arkivo tokens for each of your existing user accounts with rake sufia:user:tokens. (New users will have tokens created as part of the account creation process.)

Edit the routing constraint in config/initializers/arkivo_constraint.rb so that your Sufia-based app will allow connections from Arkivo. Make sure this is restrictive as you are allowing access to an API that allows creates, updates and deletes.

Tweak config/arkivo.yml to point at the host and port your instance of Arkivo is running on.

Tweak config/zotero.yml to hold your Zotero OAuth client key and secret. Alternatively, if you'd rather not paste these into a file, you may use the environment variables ZOTERO_CLIENT_KEY and ZOTERO_CLIENT_SECRET.

Restart your app and it should now be able to pull in Zotero-managed publications on behalf of your users. Each user will need to link their Sufia app account with their Zotero accounts, which can be done in the "Edit Profile" page. After the accounts are linked, Arkivo will create a subscription to that user's Zotero-hosted "My Publications" collection. When users add items to their "My Publications" collection via the Zotero client, they will automatically be pushed into the Sufia-based repository application. Updates to these items will trigger updates to item metadata in your app, and deletes will delete the files from your app.

Customizing metadata

Chances are you will want to customize the default metadata provided by Sufia. Here's a guide to help you with that in Sufia >= 7.0.0. If you need similar instructions for Sufia 6.x, read more about customizing metadata in 6.x.

Admin users

See making admin users in Sufia.

Migrating data to PCDM in Sufia 7

WARNING: THIS IS IN PROGRESS AND UNTESTED

  1. Create a GenericWork for each GenericFile. The new GenericWork should have the same id as the old GenericFile so that URLs that users have saved will route them to the appropriate location.
  2. Create a FileSet for each GenericWork and add it to the ordered_members collection on the GenericWork.
  3. Move the binary from GenericFile#content to FileSet#original_file.

Here are more details on a proof of concept.

Learn more about Sufia!

Clone this wiki locally