-
Notifications
You must be signed in to change notification settings - Fork 1
Resource Views
In CKAN 2.3 we are adding a new way to visualize and preview data that is stored within CKAN. Previously this was achieved through Resource Previews. This feature had the following limitations:
- It restricted each resource to a single preview type. This was defined by which plugin was deemed more important for each file format (but this was not clearly defined)
- There was only one preview per resource.
- There was no way of storing a predefined configurations of these previews.
- No uniform way to embed previews in other sites with a particular configuration.
The Resource Views feature gets over these limitations by having a way to persist saved configurations of "Views" of the data. Each View has a type (i.e could be a graph or a map) and those types will be defined by a new extension point IResourceViews. Old Resource Previews, using the IResourcePreview interface, will still work (and show up as the first View in the new interface) but this extension point is not encouraged for use anymore. There was no way we could change the old extension point without breaking backwards compatibility. However, a migration to convert an IResourcePreivew extension to a IResourceViews one should be fairly simple as they share many of the same methods.
The main difference between the new behaviour and the old behaviour is that there is now an explicit model in CKAN to store the state of the Views. Each View has a view_type defined by the extension point that created it.
The model will be as follows:
id text PRIMARY KEY
resource_id text REFERENCES resource (id)
title text
description text
view_type text NOT NULL
“order” integer NOT NULL
config text -- (JSON)
Through the API you only get direct access to the resource_id, title, description and view_type. Ordering is done by CKAN through a special reordering action. Any keys in the config are promoted to the top level json object through the API and these are validated by the schema defined in the extension. i.e if an extension has an extra field "image", a valid json to create a Resource View could be:
{"resource_id": "resid", "view_type": "customviewtype", "title": "My Image", "image": "link to my image"}
As you can see when you put the image key in the main dictionary and this will get converted in and out of the config field automatically when using all the API endpoints.
The API endpoints have all necessary functions to view, create, delete and reorder Resource Views:
resource_view_show
resource_view_list
resource_view_create
resource_view_update
resource_view_reorder
resource_view_delete
These are currently documented in the code and will therefore be in the docs once released.
Examples creating not View types can be found in core extensions. They use the new extension point the same way an external plugin would. So these act as a good place to start when creating your own.
https://github.com/ckan/ckan/blob/1251-resource-view/ckanext/imageview/plugin.py
https://github.com/ckan/ckan/blob/1251-resource-view/ckanext/pdfpreview/plugin.py
https://github.com/ckan/ckan/blob/1251-resource-view/ckanext/reclinepreview/plugin.py
https://github.com/ckan/ckan/blob/1251-resource-view/ckanext/textpreview/plugin.py
https://github.com/ckan/ckan/blob/1251-resource-view/ckanext/webpageview/plugin.py
Also a few external plugins have already been made using this extension point:
https://github.com/okfn/ckanext-vegaview
https://github.com/ckan/ckanext-basiccharts
Resource Views inherit authorization of the Resource they belong to. The resource inherits in turn authorization from the dataset it belongs to. There is no current way to make a resource view separate from a resource. There are plans, nonetheless, to be able to feed custom config to a view in order to embed your own custom version of a view_type in another page (the API for this is not finalized).
There is no way to explicitly mark a resource as the default one shown. However, each Resource View has an order it appears against a Resource. The default one shown is the first in the order, so you can just move the View to being first in the order to make it effectively the default. The only exception to this is if the old Resource Preview extension is used and that preview is always the first View in order to preserve old behaviour.
The changes to the Resource Views are located in the 1251-resource-view branch. To use it, do:
git checkout 1251-resource-view
As new plugins have been registered, you need to run:
python setup.py develop
The old preview plugins (ending with _preview) have been renamed to end with _view. These the ones you'd probably want to test things out:
ckan.plugins = resource_proxy datastore datapusher text_view pdf_view recline_grid_view recline_graph_view recline_map_view
Finally, you will need to upgrade the database, as new tables have been added:
paster db upgrade
At this point none of the existing old previews available (whether core previews like Recline, PDF, JSON, etc or external ones like GeoJSON or custom) will work, as new views need to be created in the database for them.
Note: This will change as part of #1852
To migrate all views from core plugins (recline_grid_view, pdf_view, text_view, Web pages and images) you can run the following paster command:
paster views create all
It will only create views for the view plugins that are defined in your config file. So for example if you only have pdf_preview defined as a plugin in your config, only PDF Views will be created.
Alternatively you can run the command specifying the view types, eg:
paster views create webpage image
Note: The new grid view will only work if the file (CSV or Excel) is loaded into the DataStore. The DataProxy is no longer used as the default fallback.
If you disable a Resource View extension point once it has been used, then it can cause problems. This is because CKAN will not know how to display the Views the disabled extension defined. There is a paster command to delete all views with an unavailable view_type.
paster views clean
This should be run every time you disable a used Resource View extension in your config.
An embed link will be available for all Views in the interface.
The following code is a simplified version of the following view extension (with a lot more comments): https://github.com/ckan/ckan/blob/1251-resource-view/ckanext/imageview/
This plugin lets you specify a JPG image to view. It sets up a custom form so that the user can add an image location. It will also automatically make a new view for any new resource with a format of JPG.
File: plugin.py
class JPGView(p.SingletonPlugin):
'''This extension makes views of JPG Images'''
# All views extension need to implement the following 2 interfaces.
p.implements(p.IConfigurer, inherit=True)
p.implements(p.IResourceView, inherit=True)
# IPackageController is needed if you need to auto generate a view once a resource is created.
p.implements(p.IPackageController, inherit=True)
def update_config(self, config):
# Specify the directory where your custom resource view form and your view template reside.
p.toolkit.add_template_directory(config, 'theme/templates')
def info(self):
return {'name': 'jpg_image',# Name of plugin
'title': 'JPG Image',# Title to be displayed in interface
'icon': 'picture',# Icon used.
'schema': {'image_url': [not_empty, unicode]},# Schema to validate the form.
'iframed': False}
def can_view(self, data_dict):
# Returning True says a that any resource can use this view type.
# It will appear in every resource view dropdown.
return True
def view_template(self, context, data_dict):
# The template used to display the view. See below.
return 'image_view.html'
def form_template(self, context, data_dict):
# The template used to generate the custom form elements. See below.
return 'image_form.html'
def add_default_views(self, context, data_dict):
# This is needed if you want to make a new view whenever a resources is created.
# The get_new_resource toolkit function get out just the new resources that have been created by the user.
resources = p.toolkit.get_new_resources(context, data_dict)
for resource in resources:
# for this example only resources with a format of JPG will have a view automatically created for it.
if resource.get('format') == 'JPG':
# The view data that is the same as what can be sent to the resource_view_create api action API endpoint.
view = {'title': 'Resource Image',
'description': 'View of the Image',
'resource_id': resource['id'],
'view_type': 'image',
'image_url': resource['url']}
## IMPORTANT: Make sure defer_commit: True is set otherwise the views and resources may become out of sync in the database.
p.toolkit.get_action('resource_view_create')(
{'defer_commit': True}, view
)
def after_update(self, context, data_dict):
# This is part of the IPackageController interface and is needed so default views can be created when a package is updated i.e a resource is created.
self.add_default_views(context, data_dict)
def after_create(self, context, data_dict):
# This is needed as well to make sure new views are created when a new package is made.
self.add_default_views(context, data_dict)
This the form needed to add an extra field to the form.
File: theme/templates/image_form.html
{% import 'macros/form.html' as form %}
{{ form.input('image_url', id='field-image_url', label=_('Image url'), placeholder=_('eg. http://example.com/image.jpg'), value=data.image_url, error=errors.image_url, classes=['control-full', 'control-large']) }}
This is the actual template that renders the view. It gets the data from the custom field created.
File: theme/templates/image_view.html
<img style="margin:auto; max-height:100%; display:block" src="{{ resource_view.get('image_url') }}" />
class IResourceView(Interface):
'''Add custom Resource View type for a resource.
'''
def info(self):
'''Return configuration for the view. Info can return
:param name: name of view type
:param title: title of view type (Optional)
:param schema: schema to validate extra view config (Optional)
:param icon: icon from
http://fortawesome.github.io/Font-Awesome/3.2.1/icons/
without the icon- prefix eg. compass (Optional).
:param iframed: should we iframe the view template before rendering.
If the styles or JavaScript clash with the main site theme this
should be set to true. Default is true. (Optional)
:param preview_enabled:
Says if the preview button appears for this resource. Some preview
types have their previews integrated with the form.
Some preview types have their previews integrated with the form.
Default false (Optional)
:param full_page_edit: Says if the edit form is the full page width
of the page. Default false (Optional)
eg:
{'name': 'image',
'title': 'Image',
'schema': {'image_url': [ignore_empty, unicode]},
'icon': 'compass',
'iframed': false,
}
'''
return {'name': self.__class__.__name__}
def can_view(self, data_dict):
'''Return info on whether the plugin can preview the resource.
The ``data_dict`` contains: ``resource`` and ``package``.
return ``True`` or ``False``.
'''
def setup_template_variables(self, context, data_dict):
'''
Add variables to the ``data_dict`` that is passed to the
template being rendered.
Should return a new dict instead of updating the input ``data_dict``.
The ``data_dict`` contains: ``resource_view``, ``resource`` and
``package``.
'''
def view_template(self, context, data_dict):
'''
Returns a string representing the location of the template to be
rendered when the view is rendered.
The ``data_dict`` contains: ``resource_view``, ``resource`` and
``package``.
'''
def form_template(self, context, data_dict):
'''
Returns a string representing the location of the template to be
rendered for the read page.
The ``data_dict`` contains: ``resource_view``, ``resource`` and
``package``.
'''