Skip to content

Commit

Permalink
Migrate to LaTiS v3
Browse files Browse the repository at this point in the history 
We are dropping support for LaTiS v2 and focusing on LaTiS v3. These
changes are to migrate the HAPI service interface to fit in the LaTiS
v3 service interface architecture, add an implementation of the HAPI
algebra for LaTiS v3, and make any tweaks to the HAPI service required
for LaTiS v3 support.

In order to support producing a standalone executable, the SBT build
has been modified to include dependencies on everything required to
make a standalone server. This is a temporary hack to work around the
fact that we don't have artifacts published that LaTiS can pull in.
  • Loading branch information
@lindholc
lindholc committed Dec 6, 2019
1 parent 2ac40df commit 8bdc0f9
Show file tree
Hide file tree
Showing 52 changed files with 669 additions and 1,170 deletions.
94 changes: 40 additions & 54 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,79 +1,65 @@
# <img align="right" src="http://lasp.colorado.edu/home/wp-content/uploads/2012/01/lasp-logo.color_.1line-subtext.transp-bg.med_.png" height="75" width="256"/> LASP HAPI Server

A [HAPI][hapi] server built on [LaTiS][latis].
A [HAPI][hapi] server built on [LaTiS][latis]. This project is based
on LaTiS v3, which is currently under development.

HAPI (Heliophysics API) is an interoperable, REST-style interface to
timeseries data. We implement [version 2.0][hapi2] of the
timeseries data. We implement [version 2.1][hapi2] of the
specification.

LaTiS is a library for describing and manipulating scientific data. By
building a HAPI server on LaTiS, we can

- use existing LaTiS adapters for a variety of data formats (ASCII,
NetCDF, FITS, CDF, relational databases, etc.)
- use existing LaTiS operations (filtering, subsetting, time and
format conversions, etc.)
- aggregate granules (e.g., a collection of files) into virtual
datasets

to make datasets in a wide variety of formats available through a HAPI
interface.
building a HAPI server on LaTiS, we can reuse existing LaTiS
capabilities to make datasets in a wide variety of formats available
through a HAPI interface.

[hapi]: https://hapi-server.github.io/
[hapi2]: https://github.com/hapi-server/data-specification/blob/master/hapi-2.0.0/HAPI-data-access-spec-2.0.0.md
[latis]: http://lasp.colorado.edu/home/mission-ops-data/tools-and-technologies/latis/
[hapi2]: https://github.com/hapi-server/data-specification/blob/master/hapi-2.1.0/HAPI-data-access-spec-2.1.0.md
[latis]: https://github.com/latis-data/latis3

## Configuring Datasets
## Getting Started

See [Configuring Datasets](docs/configuring-datasets.md) for an
example.
First, download the [latest release][releases] of the `latis-hapi`
script.

## Running with Docker
By default, running this script will start a HAPI server running on
port 8080. It will look for FDML files in a directory called
`datasets` in its working directory.

The `docker` SBT task will build a Docker image. The Docker daemon
must be running.
Visit `http://localhost:8080/hapi/catalog` to verify that your
datasets are visible to the server.

Then run the container:
[releases]: https://github.com/lasp/hapi-server/releases

```
$ docker run -p 8080:8080 --mount type=bind,src=<CATALOG DIR>,dst=/srv/hapi <IMAGE ID>
```
## Configuring Datasets

The landing page will be available at `http://localhost:8080/hapi`
with the default configuration.
See [Configuring Datasets](docs/configuring-datasets.md) for an
example of creating and validating FDML files.

## Running with an executable JAR
## Configuration

The `assembly` SBT task will build an executable JAR.
This server will run on port 8080 and serve datasets out of
`./datasets` by default. Configuration is only necessary to change the
port, location of FDML files, or enable additional endpoints.

Then run the JAR:
To configure the server, create a `latis.conf` file and start the
server with the `-J-Dconfig.file=<path to latis.conf>` flag.

```
$ java -jar <JAR>
```

Or with a specific configuration file:
Here is an example `latis.conf`:

```
$ java -Dconfig.file=<CONF> -jar <JAR>
```
# latis.conf
The landing page will be available at `http://localhost:8080/hapi`
with the default configuration.

## Configuration
latis {
# Set the port:
port = 8090
There are two methods for configuring the HAPI server:
# Set the directory containing FDML files:
fdml.dir = <path to FDML files>
1. [HOCON][hocon] configuration file
2. Environment variables

### Configuration Options

| Configuration key | Environment variable | Description | Default |
| ----------------- | -------------------- | ---------------------------- | ---------------- |
| `port` | `HAPI_PORT` | Port to listen on | 8080 |
| `mapping` | `HAPI_MAPPING` | URL segment before `hapi` (e.g., `"/"` → `/hapi`, `"/foo/"` → `/foo/hapi`) | "/" |
| `catalog-dir` | `HAPI_CATALOG` | Catalog directory | `./catalog` (JAR); `/srv/hapi` (Docker) |

[hocon]: https://github.com/lightbend/config/blob/master/HOCON.md
# Enable the DAP2 endpoint:
services = [
{type: "class", mapping: "/hapi", class: "latis.service.hapi.HapiService"},
{type: "class", mapping: "/dap", class: "latis.service.dap2.Dap2Service"}
]
}
```
84 changes: 18 additions & 66 deletions build.sbt
View file
Original file line number Diff line number Diff line change
@@ -1,44 +1,35 @@
ThisBuild / organization := "lasp"
ThisBuild / scalaVersion := "2.12.7"
ThisBuild / organization := "io.latis-data"
ThisBuild / scalaVersion := "2.12.8"

val http4sVersion = "0.18.17"
val pureconfigVersion = "0.9.2"
val http4sVersion = "0.20.13"

lazy val `hapi-server` = (project in file("."))
.enablePlugins(DockerPlugin)
.dependsOn(latis2)
lazy val root = (project in file("."))
.settings(compilerFlags)
.settings(dockerSettings)
.settings(assemblySettings)
.settings(
name := "hapi-server",
libraryDependencies ++= Seq(
"io.circe" %% "circe-generic" % "0.9.3",
"org.http4s" %% "http4s-blaze-server" % http4sVersion,
"org.http4s" %% "http4s-circe" % http4sVersion,
"org.http4s" %% "http4s-dsl" % http4sVersion,
"org.http4s" %% "http4s-scalatags" % http4sVersion,
"org.log4s" %% "log4s" % "1.6.1",
"ch.qos.logback" % "logback-classic" % "1.2.3" % Runtime,
"com.github.pureconfig" %% "pureconfig" % pureconfigVersion,
"com.github.pureconfig" %% "pureconfig-cats-effect" % pureconfigVersion,
"org.scalatest" %% "scalatest" % "3.0.5" % Test
),
Test / logBuffered := false
"io.latis-data" %% "latis3-core" % "0.1.0-SNAPSHOT",
"io.latis-data" %% "latis3-service-interface" % "0.1.0-SNAPSHOT",
"io.latis-data" %% "latis3-server" % "0.1.0-SNAPSHOT",
"io.latis-data" %% "latis3-hapi" % "0.1.0-SNAPSHOT",
"io.latis-data" %% "dap2-service-interface" % "0.1.0-SNAPSHOT",
"org.http4s" %% "http4s-dsl" % http4sVersion % Provided,
"org.http4s" %% "http4s-circe" % http4sVersion,
"org.http4s" %% "http4s-scalatags" % http4sVersion,
"io.circe" %% "circe-generic" % "0.12.3",
// coursier only seems to include compile dependencies when
// building a standalone executable (see coursier/coursier#552)
"ch.qos.logback" % "logback-classic" % "1.2.3"
)
)

lazy val latis2 = ProjectRef(
uri("git://github.com/latis-data/latis.git#78e60aeb5387a047ceb99db56a1bd49786100904"),
"latis"
)

lazy val compilerFlags = Seq(
scalacOptions ++= Seq(
"-deprecation",
"-encoding", "utf-8",
"-feature",
"-language:higherKinds",
"-unchecked",
"-Xfatal-warnings",
"-Xfuture",
"-Xlint:-unused,_",
"-Ypartial-unification",
Expand All @@ -48,45 +39,6 @@ lazy val compilerFlags = Seq(
"-Ywarn-value-discard"
),
Compile / console / scalacOptions --= Seq(
"-Xfatal-warnings",
"-Ywarn-unused"
)
)

lazy val dockerSettings = Seq(
docker / imageNames := {
Seq(ImageName(s"${organization.value}/${name.value}:${version.value}"))
},
docker / dockerfile := {
val jarFile = (Compile / packageBin / sbt.Keys.`package`).value
val classpath = Seq(
(Runtime / managedClasspath).value,
(Runtime / internalDependencyAsJars).value
).flatten
val mainclass = (Compile / packageBin / mainClass).value.getOrElse {
sys.error("Expected exactly one main class")
}
val jarTarget = s"/app/${jarFile.getName}"
val cp = s"$jarTarget:" + classpath.files.map { x =>
s"/app/${x.getName}"
}.mkString(":")

new Dockerfile {
from("openjdk:8-jre-alpine")
copy(classpath.files, "/app/")
copy(jarFile, jarTarget)
expose(8080)
env("HAPI_CATALOG", "/srv/hapi")
entryPoint("java", "-cp", cp, mainclass)
}
}
)

lazy val assemblySettings = Seq(
assembly / assemblyMergeStrategy := {
case "latis.properties" => MergeStrategy.first
case x =>
val strategy = (assemblyMergeStrategy in assembly).value
strategy(x)
}
)
Loading

0 comments on commit 8bdc0f9

Please sign in to comment.