Upgrade MongoDB Implementation to Support Document Structure

[iceddante]

JHipster Currently Has Limited Support for NoSQL data modeling

I am currently using JHipster to generate applications for a client that relies heavily on NOSQL datastores. Ideally, you don't want to generate your application to be dependent on a type of database, yet there's no denying that certain patterns emerge when treating your data as a collection of documents vs a completely denormalized table-based system.

Having gone through the process of creating a JDL file, commenting out the relationships, generating the application with JHipster, and then reworking it for Mongo, I don't think it would be that bad to make some changes to work with Mongo. At this point the project would look like this:

1. Modify the JDL syntax to support subdocuments. User @agoncal talked about the embeddable keyword to describe this type of object (Enriching the JDL grammar jhipster-core#141) and I think it would work well here. Any subdocument not represented with an Document Reference Id could use this and be linked by a primary document.

2. POJO generation now supports relationships: it basically works as-is since we support relationships between entities and embeddables, and embeddables and embeddables. The root document and all its subdocuments are serialized to JSON and written to the data store.

3. Integration tests contain assertions for related data types. For complex documents this makes the tests very long but that's why it's good that they're generated. For myself, modifying entities to add these datatypes after-the-fact takes a long time precisely because generating the test data and writing the assertions is cumbersome. Using JSON Pattern matching to do assertions on lists of embedded documents will probably be the hardest thing but I have code in place that does that already so I think it should not be too hard.

This is something that would benefit myself and other developers working on JHipster/Mongo apps so I would like to start working on this change, but wanted to ping the experts first to get your feedback and to see if anyone else was already actively working on this.

[jdubois]

Yes this question comes quite often, supporting embedded documents in MongoDB would be very good - everybody agrees on that, but nobody wants to do it :-(
So if you want to do a PR on this, don't hesitate - just try to have something similar in the REST output as what we have for SQL databases, so that it works out-of-the-box with the front-end (otherwise that's going to be too much work).
Also please don't try to do too much at once: it's far easier for us to merge simple PRs. And if you can add integration tests on our Travis build (look at the travis directory) it's also much better, as this is going to need testing!

[iceddante]

Ah yes- the scaffolded interface. That is the tricky part. My thinking is the rest interfaces would be used to CRUD the entire document, but this definitely would not jive with the existing scaffolded interfaces. I have to give some more thought to that problem. My initial hunch is that Mongo systems would have to have their own unique templates for generating angular which is obviously not ideal. But then again, not supporting relationships on the backend is not good either!

I see what you are saying about taking the iterative approach and I will consider that going forward.

[gmarziou]

Solving the scaffolded interface issue for Mongo and embedded docs could be also very useful for SQL databases, we could have something closer to DDD aggregates which would build nicer APIs.

[iceddante]

Yes, I thought that supporting two types of relationships might work here: OneToMany OneToFew In terms of a back end implementation both would be identical, but for the front end a OneToFew relationship means a data set could be included in, say, an edit modal. Book has a few Authors so when you are editing a book's details authors is included there "inline". When looking at the OneToMany relationship for "soldInStores" where potentially thousands of stores are in play, the interface for managing this relationship is handled separately.

…

On Tue, Dec 26, 2017 at 12:35 PM, Gaël Marziou ***@***.***> wrote: Solving the scaffolded interface issue for Mongo and embedded docs could be also very useful for SQL databases, we could have something closer to DDD aggregates which would build nicer APIs. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#6907 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGhxnWiuY0b9CDjM0big8W3cOeBE5Un7ks5tES5IgaJpZM4RMYgm> .

[jdubois]

We use OneToMany because that means something in Hibernate/JPA, OneToFew doesn't mean anything so I find this confusing.
But sure, we should have guidelines on this - there's the same issue in Hibernate/JPA: you can of course have relationships with much more entities than in MongoDB, but then you need to paginate them, and that's also causing a lot of questions. So we should at least document/guide people here.

[deepu105]

@iceddante if you are gonna do this first make sure the current front end works with this, then we can think of additional use cases as I dont think its worth to maintain 2 sets of templates as MongoDB is not that widely used as SQL and we need to take maintenance cost into account.

[jdubois]

Closing this as this has been stuck for 2 weeks - as many times before, nobody is ready to do a PR on this part... so this just not going forward.
I guess it's not very hard to do by hand, so people don't really need JHipster to scaffold this.