Skip to content

Commit

Permalink
Updated doc
Browse files Browse the repository at this point in the history
  • Loading branch information
Aklakan committed Mar 1, 2024
1 parent e0d0572 commit 96d7cdb
Show file tree
Hide file tree
Showing 4 changed files with 8 additions and 28 deletions.
Binary file added docs/images/2024-03-01-mvn-sync-architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 2 additions & 2 deletions docs/sync/change-detection.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
title: Maven Repository Events
layout: home
layout: default
parent: Maven-RDF Sync
nav_order: 10
nav_order: 20
---


Expand Down
4 changes: 2 additions & 2 deletions docs/sync/messaging.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
title: Messaging
layout: home
layout: default
parent: Maven-RDF Sync
nav_order: 20
nav_order: 30
---

# Messaging
Expand Down
28 changes: 4 additions & 24 deletions docs/sync/sync-architecture.md
Original file line number Diff line number Diff line change
@@ -1,41 +1,21 @@
---
title: Maven-RDF Sync
title: Mvn-Sync Architecture
layout: default
nav_order: 60
has_children: true
parent: Maven-RDF Sync
nav_order: 10
---

# Synchronizing a Triple Store with Maven Repository Data
# Mvn Sync Architecture

## Synopsis

* This chapter presents a lightweight trigger-based approach to realize "build actions" (or "bots") over local maven repositories. A local maven repository is simply a certain directory structure.
These bots can be used to automatically create new maven projects for producing RDF metadata.

## Purpose

* Metadata artifacts are just plain maven artifacts whose content describes another artifact.
* A large part of RDF metadata generation is agnostic of the content of a dataset and can be fully automated. In those cases, a user should not need to manually set up metadata projects.

## Abstract Approach

<img src="../images/2024-03-01-mvn-sync-architecture.png" width="900px" />

The `mvn-rdf-sync` approach comprises two separate processes:


1. A *file system watch* on a (local) maven repository notifies raises events whenever the repository content changes.
2. The event is transmitted to an appropriate receiver.
3. A *filter* discards irrelevant events, such as changes to non-dataset artifacts.
4. If a dataset is deployed (which is not a metadata dataset), then automatically create an instance of a template maven project, and run the build.

TODO architecture chart with the addition of prepublish and triple store loading


### Concrete Approach

1. A simple way to watch a directory recursively for changes is `inotifywait`. This commands even works from within a docker container on a mounted host folder.
2. A relatively straight forward way to transmit events is with Apache Kafka. The advantage of a messaging protocol is that it is possible to send custom events besides the `inotifyway` source.
This is useful when one wants to trigger recreation of metadata after changing the routine that produced the matdata.


0 comments on commit 96d7cdb

Please sign in to comment.