SHE feed upgrades

Upgrades and changes in this release:

• Introduced backend static data mappers for all data plugs
• Added a hook to clear database entries from cache
• Updated claim email design and content
• Reduced default storage allowance to 100MB

Introducing claimable HATs

Changes and updates:

• Adds API endpoints to support HAT claim functionality
• Updates password reset tokens to support HAT claim functionality
• Adds new email template to be sent when the HAT is put into "unclaimed" state
• Uses new endpoint to enable frontend action logging
• Fixes Fitbit data timezone bug

Adds Lambda-based near-hat functions for Smart HAT Engine

Merge pull request #47 from Hub-of-all-Things/v2.6.2

V2.6.2

Extending the application and SHE function data models

• Tool (SHE function) information model to include more app-style info and developer details
• App information model to include developer details, other details of trusted publishers
• SHE function showing aggregate number of data updates since most recent viewing
• Email templates styled to look like HAT signup email

the beginnings of the Smart HAT Engine (SHE)

A a major update to HAT:

• revamps Application management and permissions model to unify Apps, DataPlugs and any other tools
• starts mapping data for SHE feed in real-time, making it easier to change data formats when required
  • includes data mappers for all existing plugs to a simple "feed-style" structure: Facebook, Twitter, Fitbit, Google Calendar, Spotify
  • includes data mappers for endpoints currently "draft": Instagram, Monzo, Starling
  • starts adding smarter data processing to SHE: interpolates locations based on time information of a datapoint and locations if existing as reported by a mobile application to attach it to more objects
• adds first version of data "insights" by SHE: counting numbers of data records reported by the various DataPlugs
• improves DataDebit model to include more complete information about the requester of the data
• performance improvements including user information caching to reduce database querying
• starts using path-based api versioning to make it easier to move between major changes
• other dependency updates to stay fresh as well as best security practices improvements

Removed v1 API

This release of HAT focuses on API improvements:

• Removes obsolete V1 APIs based on the notion of data tables and the laborious data structure management
• Updates PHATA APIs to make them easy to customise by changing bundle definition within the configuration
• Tightens default security policy by no longer allowing credentials in CORS headers, redirects to SSL only for GET requests
• General performance and test coverage improvements

Initial SHE release

The initial release of the Smart HAT Engine (SHE)

• Builds a plugin-based architecture for executing functions on incoming data
• Uses the existing data bundling method to define the specifics of what data needs to be sent to a SHE function for execution
• Demonstrates the approach via existing data mapping into a single homogeneous stream from Facebook, Twitter, Fitbit and Google Calendar plugs
• Removes obsolete database tables previously used for "contextualised" data
• Updates project dependencies to newest version of Scala (2.12.4), Play Framework (2.6.6) and Slick (2.2.1) along with all related smaller dependencies

Overhauled data APIs

In this new version, HAT Data Apis have received a full rebuild to significantly simplify all data operations while at the same time providing more flexible operations

New in this version:

• Overhauled APIs to simplify data input and minimise setup required before data can be saved into a HAT
• Support for linking data records together during saving or afterwards
• Any record can now be updated in a single call or deleted using the right permissions
• Data bundling was adapted for the new model of data storage, allowing for:
  • Picking specific data points from within records
  • Mixing data from different sources into the same response
  • Filtering and searching for specific records
• Data Debits adapted for the APIs and build directly on the Bundles model to accurately specify requested data
• Authentication mechanism extended to include a more flexible user roles' model as well as a notion of "namespaces" within which an application can operate

More details in our guides in the brand new developers' portal:

• Raw Data APIs: http://developers.hubofallthings.com/guides/raw_data_io/
• Data Bundling: http://developers.hubofallthings.com/guides/data-bundling/
• Data Debits for consented sharing: http://developers.hubofallthings.com/guides/data-debits/

Want to see your favourite function featured in this list? We would too. Let us know here – http://forum.hatcommunity.org/ or vote for what others proposed http://marketsquare.hubofallthings.com/ideas and we’ll see what we can do.

File storage and merge with Rumpel

What's New in Version 2.3.0

• HAT Access Logged in a table for audit and system apis
• Adds "System Status" APIs for user to query last logins, share of storage used
• Able to store files, including photos
• Merges Rumpel into the core HAT to be served directly
• File management based on AWS S3 for backing storage and scalable I/O

Besides these, there were performance improvements

• Improved PHATA UI - to bring PHATA, and other launch items into Rumpel, to improve UX as just "using HAT" without too many products / services in their mind.
• Adds password strength checking based on (zxcvbn)[https://github.com/dropbox/zxcvbn]
• Adds password management APIs for co-hosted UI to use
• When changing password, an email must be sent to confirm that the password has been changed. In case some of us have forgotten which email was being used.

Want to see your favourite function featured in this list? We would too. Let us know here - http://forum.hatcommunity.org/ or vote for what others proposed http://www.hatdex.org/ideas-page/ and we’ll see what we can do.

Major HAT rebuild for better efficiency

• HAT now able to now serve multiple HAT databases
  • Assumed identity of a HAT is determined by the host address set in the incoming connection
  • Assumed identity configuration retrieved from static configuration file (from development) or the Milliner provisioning service in real deployments
• Migration from Akka Spray HTTP Framework to Play Framework (2.5.4) due to higher relative maturity
• Contextual data APIs not ported over to the new version. v2.2.0 only offers raw data APIs and contextless Data Debit data sharing mechanisms.
• Simplifies running HAT in development mode through configurable self-initialisation support
• hatlogin allows for additional parameters to be passed through to the redirect url
• password resets cleaner and use mail-tokens separate from full authentication tokens
• pretty "HAT not found" page when there is no configuration matching the requested HAT