Working with microservices

Introduction

The XCoLab has several microservices, which each access their own database tables and expose their objects to clients. Some services provide simple CRUD on the DB, while others provide more advanced functionality (our goal is to move more business logic to the services over time).

Service communication

The services expose a RESTful API, which is consumed by client libraries. Services can also communicate with other services by using their client libraries. For each X-service module, there is a corresponding X-client module that contains the POJOs as well as client classes, which perform HTTP requests to the services.

Client query API

We use a set of classes that allow us to communicate with a pre-defined set of resources in a type-safe manner. You can learn more about the API from the JavaDoc in the org.xcolab.util.http.client package.

How-tos

Creating a new entity

In order to create a new entity and expose it to the clients, several steps must be taken.

In the microservice

1. Create the table in your local database
2. Check your service's pom.xml to make sure the table is included in the jOOQ configuration
3. Run ./mvnw compile to generate jOOQ model classes from the database schema
4. Create DAO for the new entity, e.g. ContestDao
5. Create API endpoints to expose desired functionality, e.g. ContestController
   • @RestController marks the class as a controller in Spring
   • With @Request/Post/Put/GetMapping(<path>) annotated methods are API endpoints with these URLs
   • @PathVariable, @RequestParam, @RequestBody must be used to retrieve the parameters sent by the client's request

In the client

1. Create POJO (and/or DTO), e.g. ContestDto
   • within the POJO/DTO, create a constant for the type provider, e.g. public static final TypeProvider<ContestDto> TYPES = new TypeProvider<>(ContestDto.class, new ParameterizedTypeReference<List<ContestDto>>() {});
2. Create client class or methods
   • create new entry for the resource in the client's ResourceEnum (e.g. ContestResource)
   • create RestResource for the POJO using the ResourceEnum (e.g. private final RestResource1<ContestDto, Long> contestResource = new RestResource1<>(ContestResource.CONTEST, ContestDto.TYPES);
   • use query API to create methods that consume certain endpoints
     • create() - POST
     • update() - PUT
     • get/list() - GET
     • delete() - DELETE

Example queries

Retrieve one element

contestResource.get(456)        // 1
    .queryParam("lang", "en")   // 2
    .execute()                  // 3

The query above retrieves a specific contest by its ID (1) with the query parameter "lang" set to the String "en" (2). When execute() is called (3), the following HTTP request is sent:

GET http://xcolab-contest-service/contests/456?lang=en

The result of this request will be automatically converted to the ContestDto type and returned by the execute() method. The name xcolab-contest-service is provided by the ResourceEnum implementation ContestResource. It is the service name, which is resolved behind the scenes through the Eureka service discovery, e.g. to http://localhost:18085/contests/456?lang=en. The resource name /contests is also defined in the ContestResource enum.

More complex: retrieve a filtered list of elements

public List<Contest> findContests(Boolean active, Boolean featured)
    return contestResource.list()                  // 1
        .optionalQueryParam("active", active)      // 2
        .optionalQueryParam("featured", featured)  // 3
        .withCache(CacheName.CONTEST_LIST)         // 4
        .execute()                                 // 5
}

This method retrieves a list of contests, which can optionally be filtered by the active and featured parameters. Line (1) starts a list query on the contestResource, which is then filtered by optional query parameters (2,3). The optionalQueryParam method, as opposed to the queryParam method, only adds the parameter if the value is non-null. This means that if the method is called as findContests(true, null), only the active parameter will be added. Line (4) specifies that the result should be cashed in the CONTEST_LIST cache (the CacheName determines the cache's properties, such as size, eviction policy and expiration policy), so if the same parameters are queried again no HTTP request is sent and the results are retrieved from the cache instead. Finally, the request is executed and the resulting list is returned (5).

This is the request that is sent if findContests(true, null) is called:

GET http://xcolab-contest-service/contests?active=true

Adding a field to an existing entity

If you add a new field to an existing entity, it's a bit easier.

Note: You can also optionally extend the DAO, controller and client to allow searching by the new field.

In the microservice

1. Create the field in your local database
2. Run ./mvnw compile to update the jOOQ model classes
3. Edit the DAO to include the new field (update, create)

In the client

1. Add a new field to POJO

Database changes

To make sure your database changes are executed in our CI builds and on our productions servers, make sure you modify the xcolab-schema.sql file accordingly and include a migration script in sql/deployments

Security concerns

When creating new entities or adding fields, you probably will need to implement some sort of access control over them. This key concept here is that in a production environment, access to the microservices is restricted to each other so that the end user is only allowed to access the view. Therefore, access control should be done by the view (using clients such as the members one to call other services).

This way, non-view microservices can be developed in a CRUD fashion, allowing for the same endpoint to serve multiple purposes.