Alternative product formats #93

rditerwich · 2014-03-09T18:19:40Z

The alternative product formats support the following:

Use default values defined on case-classes when fields are missing from json
Omit empty arrays and objects when serializing fields with default values
Allow property renaming
Override formats per field
Allow advanced customizations

The formats can be obtained by using formatN instead of jsonFormatN

It supersedes my previous pull request #56

jdevelop · 2014-04-22T03:30:39Z

Any of this to be merged into master soon?

sirthias · 2014-04-22T08:38:53Z

Ruud, thanks for this, from the looks of it it looks like a useful addition.
However, we currently don't have any capacity for spray-json related work, so this PR might have to sit some time longer.
However, we have scheduled a discussion with the other Typesafe teams (Akka / Play) regarding a general joint strategy for JSON support and expect this to get things moving again...

Sorry for not reacting earlier!

Closes spray#41

…(HashMap) rather than JsObject(ListMap)

sirthias · 2014-09-19T13:31:48Z

@rditerwich I'd like to get that merged and shipped with 1.3.0.
However, you have to accept the Typesafe CLA first (just a few clicks). Would that be possible for you?

sirthias · 2014-09-19T13:32:22Z

Also, the patch doesn't merge cleanly anymore.
Could you rebase on top of the current master?

Add jsonFormat0 for fieldless case classes

rditerwich · 2014-09-27T10:39:54Z

This is great news. I will accept the Typesafe CLA and rebase in the next couple of days.

Conflicts: src/main/scala/spray/json/ProductFormats.scala

fzakaria · 2014-10-24T22:42:43Z

I'd love to see this in!

jamesmorgan · 2014-12-01T16:56:47Z

+1 for getting this in if possible :)

bvolders · 2014-12-18T08:03:06Z

plz pull in master

diversit · 2015-02-05T14:20:13Z

What is the status? Can this be merged?

sirthias · 2015-02-06T13:16:02Z

I guess it can.
We'll merge it when we cut the next minor release, which will hopefully be soon.

mustajarvi · 2015-03-11T14:09:15Z

Any time plan for this one? Would be great to have.

acruise · 2015-04-27T21:28:11Z

I would really, really like this functionality too. I have a lot of custom JsonFormat definitions that are 95% boilerplate, but with a small amount of special handling for a minority of properties.

acruise · 2015-05-11T18:10:03Z

Could this be broken out into a tiny separate library, or does it need to live in the spray.json namespace? :)

fbjon · 2015-05-22T09:10:31Z

There is a problem with the way this is implemented:

If the case class contains "additional fields" (i.e. some inner case class, a lazy val etc.), then the automatic field names don't work.
Normally that would mean using the "jsonFormat" method and specifying fields names manually but I can't find such a method for this alternative product format.

I've fixed it in our project by getting a TypeTag for P in ProductFormatImpl, getting the names of the constructor parameters from that tag and then filtering the declaredFields by name. This should eliminate one reason for the formatN-methods not working:

val tpe = ru.typeOf[P]
def termsOnly(s:ru.Symbol) = if(s.isTerm) Some(s.asTerm) else None

...

// list all case class constructor parameters
val constructor = tpe.members
   .filter(_.isMethod).map(_.asMethod).find(_.isConstructor).toSeq
val constructorParams = constructor
   .flatMap(_.paramss.head).flatMap(termsOnly)

// Filter declared fields based on constructor param names
val fields = runtimeClass.getDeclaredFields
   .filter(field => constructorParams.exists(_.name.toString == field.getName))

There's probably a nice way of doing even more of the work with just the TypeTag, but this works for me for now.

ghost · 2015-06-15T09:10:50Z

Nice improvement, +1 for getting this merged.

ksilin · 2015-07-21T11:01:04Z

+1 for sane optional values

merlijn · 2016-02-10T16:27:23Z

+1, would be really helpfull

aditanase · 2016-05-18T11:31:13Z

Any idea if this is still on the table? It's open for more than 2 years now.

mboggan · 2016-06-22T22:49:47Z

+1, this would be helpful

shlushchanka · 2016-10-07T13:46:43Z

+1, when this will be merged?

ktoso · 2016-10-07T15:17:54Z

Hi there, I recently gained rights to maintain this repository and release etc from @sirthias.
Will try to get on top of issues and PRs soon. This one was waiting a long time, but hope we can get it in perhaps now :)

zhangcbrian · 2016-10-23T21:19:06Z

+1 for merging this in...

mtomas · 2016-12-20T13:11:57Z

+1 for merging

greenhost87 · 2017-01-17T20:13:53Z

please release this 👍

poohsen · 2017-02-01T08:57:52Z

@ktoso I see you started merging in some of the newer PRs. That's great news! Do you have any idea when you will be able to look at this one?

poohsen · 2017-03-06T09:52:08Z

No updates on this?

zzzhy · 2017-03-21T09:37:31Z

any progress on this ?

firehooper · 2017-05-17T18:49:06Z

Can this get merged?

fmeeuw · 2017-07-07T09:13:40Z

+1

saraiva132 · 2017-07-27T12:11:58Z

Please do merge!

craffit · 2017-08-08T16:41:41Z

+1

jyotman · 2017-09-01T09:48:22Z

Please merge!

aednichols · 2017-09-06T14:27:11Z

+1

kittsville · 2017-09-26T16:46:54Z

+1

jrudolph · 2017-10-23T12:19:10Z

README.markdown

@@ -19,28 +20,22 @@ as depicted in this diagram:
 ### Installation

 _spray-json_ is available from the [repo.spray.io] repository.
-The latest release is `1.2.6` and is built against Scala 2.9.3, Scala 2.10.4 and Scala 2.11.0-RC4.
+The latest release is `1.3.0` and is built against Scala 2.10.4 and Scala 2.11.2.


Let's remove patch versions as they shouldn't matter (i.e. "2.10.x" and "2.11.x" and "2.12.x" and "2.13.0-M2"

I changed that in #239

jrudolph

Thanks a lot @rditerwich and sorry for keeping this around for years without doing anything about it. I like the idea a lot but we need to be careful about compatibility.

To make it happen we need:

a subset of this PR which only implements the new ProductFormats without any other changes, especially, binary incompatible ones
ProductFormats2 should be sbt-boilerplate generated
All the extra features need to be documented in the README and with ScalaDoc
We need to decide if jsonFormatN vs formatN is the right way of naming things
Would be nice to see some benchmarks comparing previous and new ProductFormats performance

I think it would be a pretty awesome feature to have so let us know @rditerwich if you would still like to make it happen or if something else should try to get it to the finish line.

jrudolph · 2017-10-23T12:21:18Z

src/main/scala/spray/json/CollectionFormats.scala

-    def write(list: List[T]) = JsArray(list.map(_.toJson))
-    def read(value: JsValue) = value match {
-      case JsArray(elements) => elements.map(_.convertTo[T])
+    def write(list: List[T]) = JsArray(list.map(_.toJson).toVector)


All these changes in here need to be performance verified. It would be easier if they'd be broken out into an extra PR.

jrudolph · 2017-10-23T12:21:53Z

src/main/scala/spray/json/CollectionFormats.scala

-  implicit def immLinearSeqFormat[T :JsonFormat]  = viaList[imm.LinearSeq[T], T](list => imm.LinearSeq(list :_*))
-  implicit def immSetFormat[T :JsonFormat]        = viaList[imm.Set[T], T](list => imm.Set(list :_*))
-  implicit def vectorFormat[T :JsonFormat]        = viaList[Vector[T], T](list => Vector(list :_*))
+  implicit def immIterableFormat[T :JsonFormat]   = viaSeq[imm.Iterable[T], T](seq => imm.Iterable(seq :_*))


We should add explicit return types to each of these (check with mima that they don't change).

jrudolph · 2017-10-23T12:22:16Z

src/main/scala/spray/json/CollectionFormats.scala


  /**
    * A JsonFormat construction helper that creates a JsonFormat for an Iterable type I from a builder function
    * List => I.
   */
-  def viaList[I <: Iterable[T], T :JsonFormat](f: List[T] => I): RootJsonFormat[I] = new RootJsonFormat[I] {


It's a public method so it needs a proper deprecation cycle.

jrudolph · 2017-10-23T12:23:04Z

src/main/scala/spray/json/JsValue.scala

@@ -49,20 +49,18 @@ sealed abstract class JsValue {
 */
 case class JsObject(fields: Map[String, JsValue]) extends JsValue {
  override def asJsObject(errorMsg: String) = this
-  def getFields(fieldNames: String*): Seq[JsValue] = fieldNames.flatMap(fields.get)
+  def getFields(fieldNames: String*): immutable.Seq[JsValue] = fieldNames.flatMap(fields.get)(collection.breakOut)


Might be a binary compatibility problem? If yes, needs deprecated forwarder.

jrudolph · 2017-10-23T12:24:29Z

src/main/scala/spray/json/JsonParser.scala

@@ -72,7 +71,7 @@ class JsonParser(input: ParserInput) {
  // http://tools.ietf.org/html/rfc4627#section-2.2
  private def `object`(): Unit = {
    ws()
-    var map = ListMap.empty[String, JsValue]


Will probably lead to incompatible semantic change.

jrudolph · 2017-10-23T12:24:52Z

src/main/scala/spray/json/JsonParser.scala

          case c => sb.append(c); rec(ix + 1, lineStartIx, lineNr)
        }
      val savedCursor = _cursor
-      _cursor = 0
+      _cursor = -1


Is that a bug fix?

jrudolph · 2017-10-23T12:25:07Z

src/main/scala/spray/json/JsonPrinter.scala

        sb.append(callback).append('(')
        print(x, sb)
-        sb.append(')');
-      }
+        sb.append(')')


jrudolph · 2017-10-23T12:27:14Z

src/main/scala/spray/json/ProductFormats.scala

@@ -23,9 +23,18 @@ import scala.util.control.NonFatal
 * Provides the helpers for constructing custom JsonFormat implementations for types implementing the Product trait
 * (especially case classes)
 */
-trait ProductFormats extends ProductFormatsInstances {
+trait ProductFormats extends ProductFormatsInstances with ProductFormats2 {


I wonder if it's a good idea to extend here? I guess it makes it easier to deprecate the old formats at some point.

jrudolph · 2017-10-23T12:28:08Z

src/main/scala/spray/json/ProductFormats2.scala

+
+import scala.reflect.ClassTag
+
+trait ProductFormats2 {


Let's keep using sbt-boilerplate for code generation.

giorgiovinci · 2018-06-22T09:51:19Z

I'm in need of the feature and I just went through all the history (from 2013) and now I'm arrive at the end of this one and I feel like hanging at the end of one of game of throne book!
So @jrudolph you asked @rditerwich and I can guess he said there are other priorities but maybe you can just let us know if there's any plan about this one?

rditerwich added 3 commits March 9, 2014 18:21

Added alternative product formats

c40e5aa

Added tests for alternative product format

f985f24

typo in readme

e273071

rditerwich mentioned this pull request Mar 9, 2014

Support for default values. #56

Closed

preserve field order in product formats

6934de8

Add jsonFormat0 for fieldless case classes

12d83eb

Closes spray#41

sirthias force-pushed the master branch from 4a8fcac to c4a92c0 Compare September 19, 2014 12:38

sirthias added 2 commits September 19, 2014 15:13

Switch JsArray(List) to JsArray(Vector), make parser produce JsObject…

37af0d4

…(HashMap) rather than JsObject(ListMap)

Improve JsonPrinter to enable printing to custom StringBuilder

f33e846

sirthias and others added 3 commits September 19, 2014 16:42

Fix small problem in JsonParser error reporting

a6aaf51

Merge pull request spray#117 from chris-martin/jsonFormat0

dce0643

Add jsonFormat0 for fieldless case classes

Prepare for release 1.3.0

cddb9a2

rditerwich added 6 commits September 27, 2014 14:36

Added alternative product formats

493a11f

Added tests for alternative product format

74f13db

typo in readme

ac5f073

preserve field order in product formats

373d5e6

Merge branch 'master' of https://github.com/rditerwich/spray-json

f81c864

Conflicts: src/main/scala/spray/json/ProductFormats.scala

Fixed tests for ProductFormats2

cc186e0

ktoso added the Feature label Oct 7, 2016

ktoso self-assigned this Oct 7, 2016

jrudolph reviewed Oct 23, 2017

View reviewed changes

ktoso removed their assignment Oct 24, 2017

shankarshastri mentioned this pull request Jul 3, 2018

How To Provide DefaultValues For Fields In JSON While Marshalling Or Unmarshalling? #257

Open

chetanmeh mentioned this pull request Jan 6, 2020

Simplify manual derivation via reflection pureconfig/pureconfig#673

Closed

Alternative product formats #93

Are you sure you want to change the base?

Alternative product formats #93

Conversation

rditerwich commented Mar 9, 2014

jdevelop commented Apr 22, 2014

sirthias commented Apr 22, 2014

sirthias commented Sep 19, 2014

sirthias commented Sep 19, 2014

rditerwich commented Sep 27, 2014

fzakaria commented Oct 24, 2014

jamesmorgan commented Dec 1, 2014

bvolders commented Dec 18, 2014

diversit commented Feb 5, 2015

sirthias commented Feb 6, 2015

mustajarvi commented Mar 11, 2015

acruise commented Apr 27, 2015

acruise commented May 11, 2015

fbjon commented May 22, 2015

ghost commented Jun 15, 2015

ksilin commented Jul 21, 2015

merlijn commented Feb 10, 2016

aditanase commented May 18, 2016

mboggan commented Jun 22, 2016

shlushchanka commented Oct 7, 2016

ktoso commented Oct 7, 2016

zhangcbrian commented Oct 23, 2016

mtomas commented Dec 20, 2016

greenhost87 commented Jan 17, 2017

poohsen commented Feb 1, 2017

poohsen commented Mar 6, 2017

zzzhy commented Mar 21, 2017

firehooper commented May 17, 2017

fmeeuw commented Jul 7, 2017

saraiva132 commented Jul 27, 2017

craffit commented Aug 8, 2017

jyotman commented Sep 1, 2017

aednichols commented Sep 6, 2017

kittsville commented Sep 26, 2017

jrudolph Oct 23, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jrudolph left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

giorgiovinci commented Jun 22, 2018

jrudolph Oct 23, 2017 •

edited

Loading