DevNotes_Processes_PublicationsWebPageInstructions

How to update the publications website

The publications website is linked to a github markdown page. The markdown page is automatically generated and updated using python code in the saspubs repository that connects to a Zotero database.

Adding publications to the list

An automated process is used to edit publications.md and then push to github. Any edits to the markdown page not done by the automated process will be overwritten. The safest way to edit the publications list is to update the Zotero database.

1. Gain access to the Zotero database: Zotero is a free cloud-based database system for cataloging bibliographies. Create a Zotero account if you do not have one, and then request access to the Sasview group. @ajj, @butlerpd, @smk78, and @krzywon are the administrators of the group.
2. Download and configure the Zotero client: Download the client (available for all platforms), and install it. The free version only offers a limited amount of storage space and the default settings will automatically download PDF and image files to the database. To prevent the database from filling up, under Edit > Preferences, on the general tab of file handling, de-select all automatic file attachment settings.
3. Start using Zotero: Zotero has published a quick start guide. To add a publication to the list, the easiest way is to add an item by an ID, which can include a DOI, ISBN, or PubMed ID. Be sure to add the publications to the sasview group, which looks like a folder, not the top-level collection.
4. Sit back and wait: Once an item has been added to Zotero, the automated process will do the rest.

Modifying output of publications.md

The publications.md file is generated by the python-based saspubs repository. A short description of the files used are given here. Ask @ajj and/or @krzywon if you would like access to the repository. Members of the web team are automatically granted access.

• citeproc_to_html.py: Using the JSON formatted list of publications, defines the format of and generates the markdown file that is pushed to sasview.github.io.
• config.py: Defines constants used for the generation of the markdown file as well what items are imported from Zotero for each publication.
• instrument_zotero_feed.py: Fetches information from the Zotero database as defined by config.py and stores the list of publications in JSON format.
• make_pubsjson.py: Stand-alone json generator that uses a narrowly defined set of items pulled from Zotero.
• update_current_list.py: Using data in the json file, checks for and then deletes duplicate DOIs, and compares existing values against the most recent published version of the citation and updates. Both checks directly update the zotero database when changes are needed.
• shell_scripts/push_udpates.sh: Bash script that links all scripts together, finds newly added items, and pushes updates to github, if changes are found.
• shell_scripts/modification_and_duplicate_check.sh: Bash script that checks for duplicates and changes by running update_current_list.py, and then pushes any updates to github.
• static/SasView_linktitle.csl: Defines the citation style.

Zotero usage and notes

• Download the client (available for all platforms) and install it.
• Optionally, install the browser connector.
• Login to your account through the client by going to Edit > Preferences > Sync.
• Configure your client (and connector) to disable all automatic file downloads to prevent filling up the database. Under Edit > Preferences, on the general tab of file handling, de-select all automatic file attachment settings.
• Click the green Sync button on the main screen to download all publications to the client - this might take a few minutes.
• Start adding publications - The easiest method is to add an item by an ID.
• Don't forget to click the green Sync button again once finished to push the new additions to the Zotero cloud server so it can be pushed to the live web page.

Automatic update process

Automatic updates are handled via a shell script that is run on the DANSE server. The shell script runs a series of python scripts that check for changes to the Zotero database.

1. Script repository: All python and shell scripts used in the automated process can be found at https://github.com/SasView/saspubs, administered by @ajj and @krzywon.
2. Catching new additions: The push_updates shell script assumes the location of a github key file is passed as the only argument. If the key file is not valid, the script halts and an error is thrown. If the key file is valid, the latest versions of both the saspubs and sasview.github.io repositories are pulled in, then the instrument_zotero_feed.py script is run. Instrument_zotero_feed checks for any changes to the Zotero database, formats and sorts the bibliography, creates a json file of all publications, and finally using the json file, generates the publications.md file. This file is copied to the local sasview.github.io and any changes are pushed to github.
3. Catching duplicates and citation changes: The modification_and_duplicate_check shell script assumes two arguments are passed; the location of a github key file, and the value of a zotero API key. If the zotero API key is valid, every publication will be checked against the most up-to-date version from the publisher. Any changes to the citation will be pushed to the zotero database. A check for duplicate DOIs is also run with the most recently added item retained and all other duplicates are deleted. If any updates or deletions are made, instrument_zotero_feed is run to capture those changes.
4. Server setup: The DANSE server stores the saspubs and sasview.github.io repositories in parallel paths. github and zotero key files are stored on the server to pull/push directly from github and edit the zotero database. Two cron jobs run periodically, one calling push_updates every 30 minutes, and the other calling modification_and_duplicate_check once per day at 12:05 AM, server time.

Please contact @krzywon with any questions.