Relaton::Index

This gem is a part of [Relaton](https://github.com/relaton) project. It provides a way to index Relaton documents' files and search them by references.

Installation

Install the gem and add it to the application’s Gemfile by executing:

$ bundle add relaton-index

If bundler is not being used to manage dependencies, install the gem by executing:

$ gem install relaton-index

Usage

Creating an index object

The gem provides the Relaton::Index.find_or_create {type}, url: {url}, file: {filename}, id_keys: {keys} method to create an index object. The first argument is the type of dataset (ISO, IEC, IHO, etc.). The second argument is the URL to the zipped remote index file. The third argument is the filename of the local index file. The fourth argument is an array of ID’s parts names. The URL, filename, and keys are optional.

If the URL is specified and the local file in a /{home}/.relaton/{type} dir doesn’t exist or is outdated, the index file will be downloaded from the URL saved as a local file and an index object will be created from the file. If the file in the /{home}/.relaton/{type} exists and is actual, the index object will be created from the local file.

If the URL isn’t specified, the index object will be created from the local file in the current dir ./{filename}. If the file doesn’t exist, the empty index object will be created.

If the filename isn’t specified, a default index,yaml filename will be used.

If the keys are specified, the local index file will be checked for the presence of the keys. If the keys are not found, the index file will be downloaded from the URL or an empty index object will be created if the URL isn’t specified.

It’s possible to create many index objects for different types of datasets. The index objects are saved in the pool. The pool is a hash with the type of dataset as a key and the index object as a value. The pool is a class variable of the Relaton::Index class. The pool is used to find an existing index object by the type of dataset. The Relaton::Index.find_or_create method returns an existing index object if it was created before.

If the URL or filename is specified and has a new value, different from what was used to create the index object before, the index object will be recreated.

require 'relaton/index'

# Create a new index object. The first argument is the type of dataset (ISO, IEC, IHO, etc.) URL and filename are optional.
Relaton::Index.find_or_create :IHO, url: "https://raw.githubusercontent.com/relaton/relaton-data-iho/master/index.zip", file: "index-iho.yaml", id_keys: %i[number part year]

# Find an existing index object (created before).
Relaton::Index.find_or_create :IHO

# If the URL or filename is specified and has a new value (different from what it was before), the index object will be recreated.
Relaton::Index.find_or_create :IHO, url: nil, file: "index.yaml"

# Remove the index from the pool.
Relaton::Index.close :IHO

The aim of this gem is to be used by Relaton libraries in two ways: - indexing documents' files in GitHub repositories - searching a document by a reference when the Relaton library fetches a document

Indexing

In this case, the Relaton library creates an index object and adds documents' files to it. By default, the index object is saved to the index.yaml file in the root of the repository. The filename can be changed using the filename setting.

# Create a new index object or fetch an existing one. The first argument is the type of dataset (ISO, IEC, IHO, etc.) URL should not be specified.
index = Relaton::Index.find_or_create :IHO

# Add a document to the index or update it if it already exists.
index.add_or_update "B-4 2.19.0", "data/b-4_2_19_0.xml"

# Save the index to the `index.yaml` file in the current directory.
index.save

Searching

In this case, the Relaton library should create an index object and search for a document by reference. The gem looks for the .relaton/[TYPE]/index.yaml file in the user’s home directory. The [TYPE] is one of downcased ISO, IEC, IHO, etc. If the file is not found or is older than 24 hours then it will be downloaded from the URL specified in the find_or_create method. The URL can be specified as true if the index should not be fetched from the URL.

# Create a new index object or fetch an existing one. URL should be specified. If the index file is not found or is older than 24 hours, it will be downloaded from the URL. By default, the index is saved as `index.yaml` file to the `/[HOME]/.relaton/iho/` folder. If the URL is specified as `true`, the index won't be fetched from the URL.
index = Relaton::Index.find_or_create :IHO, url: "https://raw.githubusercontent.com/relaton/relaton-data-iho/master/index.zip"


# Search for a document by reference
index.search "B-4"
# => [{ id: "B-4 2.19.0", file: "data/b-4_2_19_0.xml" }]

# Search for a document by reference using a block
index.search do |row|
  # do something with the index row
  row[:id] == "B-4"
end
# => [{ id: "B-4 2.19.0", file: "data/b-4_2_19_0.xml" }]

Remove all index records

This method removes all records from the index object. The index file is not removed.

index.remove_all

Remove index file

This method removes the index file. The index object is cleared and can be used for indexing or loading a file from the specified URL.

index.remove_file

Configuration

The gem can be configured by using the Relaton::Index.config method. The following settings are available:

• filename - the name of the index file. By default, it is index.yaml.

• storage - the storage class or module. By default, it is FileStorage module. It can be any class or module that implements the ctime, read, and write class methods.

• storage_dir - the directory where the .relaton/[TYPE] folder is created. By default, it’s a user’s home dir. This setting works only when the gem is used for searching. When indexing, the index is always saved in a current folder.

Relaton::Index.config do |config|
  config.filename = "index-v1.yaml"
  config.storage = S3Storage
  config.storage_dir = "/"
end

It’s also possible to redefine file name for a specific type of index:

index = Relaton::Index.find_or_create :IHO, file: "index-v2.yaml"

Development

After checking out the repo, run bin/setup to install dependencies. Then, run rake spec to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and the created tag, and push the .gem file to [rubygems.org](https://rubygems.org).

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/relaton/relaton-index.

License

The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).