Revenj is a framework which can be used on Mono with Postgres. While it can be used as any other framework, it's rather small (feature wise) and it's best used with DSL Platform compiler. This makes it ideal as a REST service built on DSL models, or within some other framework such as ASP.NET. While Revenj supports Oracle too, due to its native dependencies it only runs on Windows.
- Revenj contains LINQ provider for Postgres, somewhat different from other LINQ providers since it leverages object-oriented features of Postgres. This allows for having a NoSQL documents inside a relational databases.
- Revenj also supports various serializations out of the box, such as: Json, Protobuf and XML.
- Inversion of control is used to bind various services together and Autofac (slightly modified) is the default container.
- Advanced features such as AOP are supported, which means aspects can be registered to utilize various inspections and fixes, without providing alternative service implementations.
- Plugin based architecture allows for easy extensions without code changes or recompilations. Signature based extensions are utilized, so even convention or configuration are not necessary, since services are picked up by their signature, not their name or explicit wiring.
In this tutorial we will use it as a REST service to show off why Revenj/DSL Platform is useful. To get started, we'll need Postgres (9.1 or newer) and Mono (3.2 or newer). DSL Platform is a DSL compiler which converts provided DSL model to target code/SQL.
###DSL introduction
Let's start with a minimal DSL and a simple CRUD operation to get a feeling what's happening.
Minimal useful DSL we can write is:
module Tutorial {
aggregate Example;
}
This code is syntax sugar and it is almost equivalent to a slightly longer DSL:
module Tutorial {
aggregate Example(ID) {
int ID { sequence; }
}
}
We have a module named Tutorial which will be mapped to Tutorial namespace in code, to schema Tutorial in Postgres and an aggregate root Example (think of it as a master entity). ID is automatically generated by the database as a unique key since we utilized sequence concept.
To CRUD it we can use REST-like API available in a plugin DLL provided with Revenj.
Rest plugin is using WCF signature for defining endpoint and is available (by default) via /Crud.svc/Tutorial.Example url.
###Using DSL Platform command line client
Through this tutorial we will use DSL command line client dsl-clc.jar which is used to automate interaction with the DSL Platform. More info about the tool can be found along the source code. In this tutorial we will only build server library to get the basic understanding of the workflow.
To continue with this tutorial you'll need:
- DSL command line client
- Mono 3.2 or newer
- Postgres 9.1 or newer
- Java 1.6 or newer
To start, create a new .props (let's call it my.props) file and fill it with configuration values for your environment. For our example we will use these values:
# use an offline compiler (alternative is to use an online service)
compiler
# Path for generated server model library
revenj=./model/ServerModel.dll
# Path to directory where SQL migration script will be generated
sql=./migrationDirectory
# Path that contains all our .dsl files for the project
dsl=./dsl
# Postgres connection string
db=localhost:5432/ngsdb?user=dbUser&password=dbPass
# Write detailed output
log
# Create SQL migration script
migration
# Apply the SQL migration script to the database
apply
We will use CLC to compile the DSL file(s). It will
- gather sources/migration from DSL compiler
- download all library dependencies that are required for the project (from Github or DSL Platform website)
- compile ServerModel.dll using sources and found dependencies using Mono compiler
- create and run a database migration script
After that we will do few configuration steps. So let's start.
Create a new file inside a directory that is specified by the "dsl" parameter in your .props property file and paste in the DSL used at the beginning of the tutorial. File should have an ddd or dsl extension.
We will now use DSL command line client to run one action that will do several things at once using our DSL file: parse the DSL file for errors, generate SQL script, automatically apply the SQL script to the database, generate target/server library. Also it will download latest version of Revenj. This is all done by this command:
java -jar dsl-clc.jar properties=my.props
Instead of using a my.props file, we could send arguments using command line, such as:
java -jar dsl-clc.jar postgres=localhost:5432/ngsdb?user=dbUser revenj.net=./model/ServerModel.dll migration apply compiler
But for convenience, let's stick with the .props file. Output will be as follows:
Do you wish to download compiler from the Internet? (y/N)
Since we have passed in compiler argument stating that we wish to use the offline compiler it will ask to download it from the Internet. Next:
Revenj dependencies not found in: /var/mono/wwwroot/clc/./revenj
Do you wish to download latest Revenj version from the Internet (y/N): y
Enter y to accept and clc will download Revenj libraries to default (./revenj) folder.
Downloading Revenj.NET from Github...
Unpacked: Revenj.Http.exe. Size: 43kB
... (omitted)
Unpacked: Revenj.Http.exe.config. Size: 2kB
Now that all dependencies are downloaded, CLC starts to process our DSL. First by compiling it and creating Mono .dll file with our model in it, then by saving a database migration script to specified folder and finally by applying it on our Postgres database. If you have an error inside your DSL, they will be shown during the beginning of this part of the process and all activities will be stopped. Output from the whole process can be seen here:
Compiling Revenj.NET library...
Compiled Revenj.NET library to: /var/mono/wwwroot/clc/./model/ServerModel.dll
Creating SQL migration...
Migration saved to /var/mono/wwwroot/clc/./migrationDirectory/sql-migration-1406165252523.sql
--CREATE: Tutorial-Example
New object Example will be created in schema Tutorial
--CREATE: Tutorial-Example-ID
New property ID will be created for Example in Tutorial
New object Example will be created in schema Tutorial
New property ID will be created for Example in Tutorial
Applying migration...
Database migrated and script renamed to: applied-sql-migration-1406165252523.sql
As you can see log informed us that Example object is created in schema Tutorial. This corresponds to changes in our DSL; we are creating new objects. All database modifications and changes can be seen in this part of the log. If we had destructive changes that could cause loss of data (ex. removing a column), you will be prompted for a confirmation to proceed.
To see what our migration did, we can connect to the database using pgAdmin or similar tool. We can see that schema Tutorial really exists with table Example:
DB script which was used to migrate the database can be found in the directory specified in configuration. You can open it and see what it does. At the beginning it will contain comments of database changes, following by a check if database upgrade is allowed. After that script will perform migration from the previous state to the current state. Since this was our initial migration, a helper -NGS- schema will be created, which is used for storing the previous version of applied DSL and some system functions. It should look something like this:
###Starting the http server
Now that CLC has done it's job we are ready to start our HTTP server. For simplicity sake we will reuse dependencies downloaded to ./revenj which are used for compilation. Let's go to Revenj directory and open the configuration file "Revenj.Http.exe.config". For this tutorial edit only the following properties to set it up:
- "ServerAssembly" - path and filename of the ServerModel.dll. Point it to one that was generated by DSL CLC.
- "ConnectionString" - full connection string for the Postgres database
So it looks like this:
<add key="ServerAssembly" value="../model/ServerModel.dll"/>
<add key="ConnectionString" value="server=localhost;port=5432;database=ngsdb;user=dbUser;password=dbPass;encoding=unicode" />
Save the file.
Start the http server using this command:
$ mono Revenj.Http.exe
Output be like this:
Starting server
Server running on:
http://localhost:8999/
Test the http server by fetching from URL http://localhost:8999/Domain.svc/search/Tutorial.Example for example with curl command. This should trigger a search command on our Tutorial.Example aggregate to fetch all records. Since the table in the database is empty it should not return anything but an empty Array of our Example aggregate:
curl http://localhost:8999/Domain.svc/search/Tutorial.Example
<?xml version="1.0" encoding="utf-8"?>
<ArrayOfExample/>
To test CRUD, we can just send a POST request to our Revenj server telling it to create one new Example object and insert it into the database. We can do that using curl command:
curl -i -X POST http://localhost:8999/Crud.svc/Tutorial.Example -H "Content-Type: application/json" --data "{}"
HTTP/1.1 201 Created
X-Duration: 148.15820000000000000000000000
Content-Type: application/xml
Server: Mono-HTTPAPI/1.0
Date: Thu, 24 Jul 2014 01:44:29 GMT
Content-Length: 174
Keep-Alive: timeout=15,max=100
<?xml version="1.0" encoding="utf-8"?>
<Example xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns:d1p1="Tutorial">
<ID>1001</ID>
<URI>1001</URI>
</Example>
The resposne should look something similar as above. Response body contains XML with our newly created object, consisting only of ID and URI. Response is in XML format since we did not specify Accept header. Also for now anyone can send this kind of request towards our server, since by default, config has disabled authorization using NoAuth class specified as custom authorization and override for security permissions.
###First tour through Revenj
So how was this processed by Revenj?
POST request came to the Crud.svc/Tutorial.Example url.
This is handled by the ICrudCommands.cs service contract in REST commands plugin:
During initialization, Revenj Http looked up all WCF service contracts and initialized them.
Revenj Http is actually using Mono HttpListener, but it's passing its requests to matched WCF service contracts, in this case Crud.svc Create method.
Its implementation CrudCommands.cs is just passing it through the Revenj pipeline to the underlying Create class in Revenj.Plugins.Server.Commands project, which:
- performs basic validations
- checks caller permissions
- deserializes object (or casts is from previously deserialized one) to correct type
- calls insert on repository with it
- and returns it to the caller in requested format
This is the basic architecture of processing requests in Revenj. While few interesting things happened in the pipeline, we will not discuss them in this tutorial.
###Modeling NoSQL documents
Now that we have a working setup and understand the basic processing of the request, let's write more interesting model, not just standard ER one.
Let's write our final model for this tutorial:
module Tutorial {
aggregate Example {
timestamp StartedOn;
string CodeName;
Set<string(10)> Tags;
List<Idea> Ideas;
persistence { history; }
}
value Idea {
date? ETA;
Importance Rating;
decimal Probability;
string[] Notes;
}
enum Importance {
NotUseful;
GroundBreaking;
WorldChanging;
}
snowflake<Example> ExampleList {
StartedOn;
CodeName;
order by StartedOn desc;
}
}
to show off various minor and major features which are available with few simple descriptions.
DSL supports various property types, collections, references and basically everything you need to describe a complex domain, while DSL Platform will provide best-practice implementations for such concepts. DSL described above will result in a model which the developer would usually write in code and a lot of boilerplate hidden away which will be used, such as repositories, conversions from C# objects to Postgres objects, various validations, boilerplate code for serialization and various other libraries, useful methods such as Clone, Equals and many others.
Run the CLC again with same command as above and you will get much more information than before; for each newly added property or concept.
[mono@oel6dsl clc]$ java -jar dsl-clc.jar properties=my.props
Compiling DSL...
Calling: https://compiler.dsl-platform.com:8443/platform/Platform.svc/unmanaged/source?targets=CSharpServer
Compiling Revenj.NET library...
Compiled Revenj.NET library to: /var/mono/wwwroot/clc/./revenj/ServerModel.dll
Downloading SQL migration...
Calling: https://compiler.dsl-platform.com:8443/platform/Platform.svc/unmanaged/postgres-migration?version=1.0.2.22572
Migration saved to /var/mono/wwwroot/clc/./migrationDirectory/sql-migration-1406167016944.sql
--CREATE: Tutorial-Example-StartedOn
New property StartedOn will be created for Example in Tutorial
--CREATE: Tutorial-Example-CodeName
New property CodeName will be created for Example in Tutorial
--CREATE: Tutorial-Example-Tags
New property Tags will be created for Example in Tutorial
--CREATE: Tutorial-Example-Ideas
New property Ideas will be created for Example in Tutorial
--CREATE: Tutorial-Idea
New object Idea will be created in schema Tutorial
--CREATE: Tutorial-Idea-ETA
New property ETA will be created for Idea in Tutorial
--CREATE: Tutorial-Idea-Rating
New property Rating will be created for Idea in Tutorial
--CREATE: Tutorial-Idea-Probability
New property Probability will be created for Idea in Tutorial
--CREATE: Tutorial-Idea-Notes
New property Notes will be created for Idea in Tutorial
--CREATE: Tutorial-Importance
New object Importance will be created in schema Tutorial
--CREATE: Tutorial-Importance-NotUseful
New enum label NotUseful will be added to enum object Importance in schema Tutorial
--CREATE: Tutorial-Importance-GroundBreaking
New enum label GroundBreaking will be added to enum object Importance in schema Tutorial
--CREATE: Tutorial-Importance-WorldChanging
New enum label WorldChanging will be added to enum object Importance in schema Tutorial
New property StartedOn will be created for Example in Tutorial
New property CodeName will be created for Example in Tutorial
New property Tags will be created for Example in Tutorial
New property Ideas will be created for Example in Tutorial
New object Idea will be created in schema Tutorial
New property ETA will be created for Idea in Tutorial
New property Rating will be created for Idea in Tutorial
New property Probability will be created for Idea in Tutorial
New property Notes will be created for Idea in Tutorial
New object Importance will be created in schema Tutorial
New enum label NotUseful will be added to enum object Importance in schema Tutorial
New enum label GroundBreaking will be added to enum object Importance in schema Tutorial
New enum label WorldChanging will be added to enum object Importance in schema Tutorial
Applying migration...
Database migrated and script renamed to: applied-sql-migration-1406167016944.sql
Now let's take a look what's inside our generated ServerModel.dll file to get a better picture of what's going on. To easily browse our server side library we will use MonoDevelop. Create a project inside a MonoDevelop and add ServerModel.dll as a reference. Take a look what's inside:
Repositories and various other services have internal modifier, but are available through Revenj interfaces.
So ExampleList repository can be resolved as IQueryableRepository<ExampleList> or by using IDataContext with its Query<ExampleList>() method.
If for some reason custom repository needs to be used, a registration to the container with the new repository will override the default registration.
It's interesting to take a look at the database to see the model.
Advanced object-oriented features of Postgres are utilized for some aspects of the model, such as Set, List and value object:
If you look at the database you will find collection of varchar(10) named Tags, collection of types named Ideas, history table matching exact structure, persist_Example function which accepts arrays as arguments and is optimized for bulk processing.
If necessary, optimized single insert/update function can be created in the database and called from repository.
Data access doesn't actually go through the tables, but through the views, so, if required, DBA can alter objects created by the Platform and report an issue which will then result in a better database object or an additional modeling concept.
Of course, dropping down to SQL when everything else fails can be done through the DSL.
###Conclusion
So while some ORMs can support simple NoSQL models, neither is close to supporting advanced NoSQL-like modeling in the database DSL Platform provides and Revenj utilizes. Of course, nobody is forcing developer to use object oriented features, collections and various other non-relational constructs, but since those can provide various optimizations they are often useful.
Basic premise behind DSL is to have a ubiquitous language not only in the core domain, but everywhere. This greatly improves communication between various developers. If they utilize correct domain terminology, this brings application a lot closer to the domain-expert which can now validate model by reading DSL - a formal documentation.
Let's look at few examples of the boilerplate validations which were created.
Aggregate root has a Set<string(10)> field.
Since neither the field, nor the content of the field is optional, generated code will check for nulls.
Also, since we are using a string(10) type, it will guard against tags longer than 10 chars.
All those checks improve the quality of our data, but are often cumbersome to write.
But what about some advanced features available as a single DSL line, such as history concept?
This concept is translated to various objects in the database, snippets of code during persist, casts between representations, specialized services in the code, basically various boilerplate one would need to write/specify to get such a complex feature. And it's complex since history objects is also typesafe in the database. If we just stored the object as JSON or something similar in a field, it would introduce problems down the line when we tried to change the model. Goal of the DSL Platform is to help you write code which will not turn into legacy. To be able to do that, DSL Platform maintains typesafe models everywhere. If domain is explored during lifetime of the application, deeper insights should happen. They result in fields being moved around, renames, nullability changes and various other small and big changes. To keep up with that, automatic migrations keep model in sync with the database and typesafe compilers warn about hand written code on top of recreated DSL model.
Regarding the history feature, from the developer point of view, he will just use a IRepository<IHistory<Example>>, while the compilers will take care of all the boring work.