Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More flexible upickle integration #1969

Merged
merged 3 commits into from
Oct 12, 2023
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 17 additions & 2 deletions docs/json.md
Original file line number Diff line number Diff line change
Expand Up @@ -232,12 +232,12 @@ or for ScalaJS (cross build) projects:
"com.softwaremill.sttp.client4" %%% "upickle" % "@VERSION@"
```

To use, add an import: `import sttp.client4.upicklejson._` (or extend `SttpUpickleApi`) and define an implicit `ReadWriter` (or separately `Reader` and `Writer`) for your datatype.
To use, add an import: `import sttp.client4.upicklejson.default._` and define an implicit `ReadWriter` (or separately `Reader` and `Writer`) for your datatype.
Usage example:

```scala mdoc:compile-only
import sttp.client4._
import sttp.client4.upicklejson._
import sttp.client4.upicklejson.default._
import upickle.default._

val backend: SyncBackend = DefaultSyncBackend()
Expand All @@ -254,3 +254,18 @@ basicRequest
.response(asJson[ResponsePayload])
.send(backend)
```

If you have a customised version of upickle, with [custom configuration](https://com-lihaoyi.github.io/upickle/#CustomConfiguration), you'll need to create a dedicated object, which provides the upickle <-> sttp integration. There, you'll need to provide the implementation of `upickle.Api` that you are using. That's needed as the api contains the `read`/`write` methods to serialize/deserialize the JSON; moreover, the `ReadWriter` isn't a top-level type, but path-dependent one, also part of the `upickle.Api` instance.

For example, if you want to use the `legacy` upickle configuration, the integration might look as follows:

```scala mdoc:compile-only
import upickle.legacy._ // get access to ReadWriter type, macroRW derivation, etc.

object legacyUpickle extends sttp.client4.upicklejson.SttpUpickleApi {
override val upickleApi: upickle.legacy.type = upickle.legacy
}
import legacyUpickle._

// use upickle as in the above examples
```
2 changes: 1 addition & 1 deletion docs/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ Your code might then look as follows:

```scala mdoc:compile-only
import sttp.client4._
import sttp.client4.upicklejson._
import sttp.client4.upicklejson.default._
import upickle.default._

val backend = DefaultSyncBackend()
Expand Down
Original file line number Diff line number Diff line change
@@ -1,29 +1,29 @@
package sttp.client4.upicklejson

import upickle.default.{read, write, Reader, Writer}
import sttp.client4._
import sttp.client4.internal.Utf8
import sttp.model.MediaType
import sttp.client4.json._

trait SttpUpickleApi {
val upickleApi: upickle.Api
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: Can't we just have a default implementation = upickle.default here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could but that wouldn't be very helpful. The type of the upickleApi needs to be overriden to the singleton type of the value (I'll extend the docs). Otherwise, things don't work as expected, as the compiler can't lookup the correct Reader/Writer values.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh right, the value type has to be overridden by a concrete type. This dependent type management on the library user side is quite confusing.


implicit def upickleBodySerializer[B](implicit encoder: Writer[B]): BodySerializer[B] =
b => StringBody(write(b), Utf8, MediaType.ApplicationJson)
implicit def upickleBodySerializer[B](implicit encoder: upickleApi.Writer[B]): BodySerializer[B] =
b => StringBody(upickleApi.write(b), Utf8, MediaType.ApplicationJson)

/** If the response is successful (2xx), tries to deserialize the body from a string into JSON. Returns:
* - `Right(b)` if the parsing was successful
* - `Left(HttpError(String))` if the response code was other than 2xx (deserialization is not attempted)
* - `Left(DeserializationException)` if there's an error during deserialization
*/
def asJson[B: Reader: IsOption]: ResponseAs[Either[ResponseException[String, Exception], B]] =
def asJson[B: upickleApi.Reader: IsOption]: ResponseAs[Either[ResponseException[String, Exception], B]] =
asString.mapWithMetadata(ResponseAs.deserializeRightWithError(deserializeJson)).showAsJson

/** Tries to deserialize the body from a string into JSON, regardless of the response code. Returns:
* - `Right(b)` if the parsing was successful
* - `Left(DeserializationException)` if there's an error during deserialization
*/
def asJsonAlways[B: Reader: IsOption]: ResponseAs[Either[DeserializationException[Exception], B]] =
def asJsonAlways[B: upickleApi.Reader: IsOption]: ResponseAs[Either[DeserializationException[Exception], B]] =
asStringAlways.map(ResponseAs.deserializeWithError(deserializeJson)).showAsJsonAlways

/** Tries to deserialize the body from a string into JSON, using different deserializers depending on the status code.
Expand All @@ -32,15 +32,16 @@ trait SttpUpickleApi {
* - `Left(HttpError(E))` if the response was other than 2xx and parsing was successful
* - `Left(DeserializationException)` if there's an error during deserialization
*/
def asJsonEither[E: Reader: IsOption, B: Reader: IsOption]: ResponseAs[Either[ResponseException[E, Exception], B]] =
def asJsonEither[E: upickleApi.Reader: IsOption, B: upickleApi.Reader: IsOption]
: ResponseAs[Either[ResponseException[E, Exception], B]] =
asJson[B].mapLeft {
case HttpError(e, code) => deserializeJson[E].apply(e).fold(DeserializationException(e, _), HttpError(_, code))
case de @ DeserializationException(_, _) => de
}.showAsJsonEither

def deserializeJson[B: Reader: IsOption]: String => Either[Exception, B] = { (s: String) =>
def deserializeJson[B: upickleApi.Reader: IsOption]: String => Either[Exception, B] = { (s: String) =>
try
Right(read[B](JsonInput.sanitize[B].apply(s)))
Right(upickleApi.read[B](JsonInput.sanitize[B].apply(s)))
catch {
case e: Exception => Left(e)
case t: Throwable =>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
package sttp.client4

package object upicklejson extends SttpUpickleApi
package object upicklejson {
object default extends SttpUpickleApi {
override val upickleApi: upickle.default.type = upickle.default
}
}
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package sttp.client4.upicklejson

import upickle.default._
import sttp.client4.upicklejson.default._
import org.scalatest.concurrent.ScalaFutures
import sttp.client4.basicRequest
import sttp.client4.testing.SyncBackendStub
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
package sttp.client4.upicklejson

import upickle.default._
import org.scalatest._
import sttp.client4.internal._
import sttp.client4._
Expand All @@ -11,6 +10,9 @@ import ujson.Obj

class UpickleTests extends AnyFlatSpec with Matchers with EitherValues {
"The upickle module" should "encode arbitrary bodies given an encoder" in {
import UsingDefaultReaderWriters._
import sttp.client4.upicklejson.default._

val body = Outer(Inner(42, true, "horses"), "cats")
val expected = """{"foo":{"a":42,"b":true,"c":"horses"},"bar":"cats"}"""

Expand All @@ -20,6 +22,9 @@ class UpickleTests extends AnyFlatSpec with Matchers with EitherValues {
}

it should "decode arbitrary bodies given a decoder" in {
import UsingDefaultReaderWriters._
import sttp.client4.upicklejson.default._

val body = """{"foo":{"a":42,"b":true,"c":"horses"},"bar":"cats"}"""
val expected = Outer(Inner(42, true, "horses"), "cats")

Expand All @@ -29,30 +34,45 @@ class UpickleTests extends AnyFlatSpec with Matchers with EitherValues {
}

it should "decode None from empty array body" in {
import UsingDefaultReaderWriters._
import sttp.client4.upicklejson.default._

val responseAs = asJson[Option[Inner]]

runJsonResponseAs(responseAs)("[]").right.value shouldBe None
}

it should "decode Left(None) from upickle notation" in {
import UsingDefaultReaderWriters._
import sttp.client4.upicklejson.default._

val responseAs = asJson[Either[Option[Inner], Outer]]

runJsonResponseAs(responseAs)("[0,[]]").right.value shouldBe Left(None)
}

it should "decode Right(None) from upickle notation" in {
import UsingDefaultReaderWriters._
import sttp.client4.upicklejson.default._

val responseAs = asJson[Either[Outer, Option[Inner]]]

runJsonResponseAs(responseAs)("[1,[]]").right.value shouldBe Right(None)
}

it should "fail to decode from empty input" in {
import UsingDefaultReaderWriters._
import sttp.client4.upicklejson.default._

val responseAs = asJson[Inner]

runJsonResponseAs(responseAs)("").left.value should matchPattern { case DeserializationException(_, _) => }
}

it should "fail to decode invalid json" in {
import UsingDefaultReaderWriters._
import sttp.client4.upicklejson.default._

val body = """not valid json"""

val responseAs = asJson[Outer]
Expand All @@ -62,6 +82,9 @@ class UpickleTests extends AnyFlatSpec with Matchers with EitherValues {
}

it should "encode and decode back to the same thing" in {
import UsingDefaultReaderWriters._
import sttp.client4.upicklejson.default._

val outer = Outer(Inner(42, true, "horses"), "cats")

val encoded = extractBody(basicRequest.body(outer))
Expand All @@ -71,6 +94,9 @@ class UpickleTests extends AnyFlatSpec with Matchers with EitherValues {
}

it should "set the content type" in {
import UsingDefaultReaderWriters._
import sttp.client4.upicklejson.default._

val body = Outer(Inner(42, true, "horses"), "cats")
val req = basicRequest.body(body)

Expand All @@ -80,6 +106,9 @@ class UpickleTests extends AnyFlatSpec with Matchers with EitherValues {
}

it should "only set the content type if it was not set earlier" in {
import UsingDefaultReaderWriters._
import sttp.client4.upicklejson.default._

val body = Outer(Inner(42, true, "horses"), "cats")
val req = basicRequest.contentType("horses/cats").body(body)

Expand All @@ -89,6 +118,9 @@ class UpickleTests extends AnyFlatSpec with Matchers with EitherValues {
}

it should "serialize ujson.Obj using implicit upickleBodySerializer" in {
import UsingDefaultReaderWriters._
import sttp.client4.upicklejson.default._

val json: Obj = ujson.Obj(
"location" -> "hometown",
"bio" -> "Scala programmer"
Expand All @@ -105,15 +137,33 @@ class UpickleTests extends AnyFlatSpec with Matchers with EitherValues {
actualContentType should be(expectedContentType)
}

case class Inner(a: Int, b: Boolean, c: String)
it should "encode using a non-default reader/writer" in {
import UsingLegacyReaderWriters._
object legacyUpickle extends SttpUpickleApi {
override val upickleApi: upickle.legacy.type = upickle.legacy
}
import legacyUpickle._

object Inner {
implicit val reader: ReadWriter[Inner] = macroRW[Inner]
val body = Outer(Inner(42, true, "horses"), "cats")
val expected = """{"foo":{"a":42,"b":true,"c":"horses"},"bar":"cats"}"""

val req = basicRequest.body(body)

extractBody(req) shouldBe expected
}

case class Inner(a: Int, b: Boolean, c: String)
case class Outer(foo: Inner, bar: String)

object Outer {
object UsingDefaultReaderWriters {
import upickle.default._
implicit val reader: ReadWriter[Inner] = macroRW[Inner]
implicit val readWriter: ReadWriter[Outer] = macroRW[Outer]
}

object UsingLegacyReaderWriters {
import upickle.legacy._
implicit val reader: ReadWriter[Inner] = macroRW[Inner]
implicit val readWriter: ReadWriter[Outer] = macroRW[Outer]
}

Expand Down