Skip to content

Commit

Permalink
Update to match b37a6fd by repository_dispatch 298
Browse files Browse the repository at this point in the history
  • Loading branch information
jbigot committed Aug 27, 2024
1 parent 67cafbb commit 47fc1c2
Show file tree
Hide file tree
Showing 251 changed files with 37,924 additions and 1,052 deletions.
117 changes: 117 additions & 0 deletions 1.7/Concepts.html
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<meta name="generator" content="Doxygen 1.9.6"/>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<title>PDI: Core Concepts</title>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<script type="text/javascript" src="navtreedata.js"></script>
<script type="text/javascript" src="navtree.js"></script>
<link href="style.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div>
<header id="titlearea">
<div id="projectlogo"><a href="index.html"><img alt="Logo" src="logo.png"/></a></div>
<h1 id="projectname">PDI&nbsp;<span id="projectnumber">1.7.1</span></h1>
<h2 id="projectbrief">the <span class="acronym">P</span>DI <span class="acronym">d</span>ata <span class="acronym">i</span>nterface</h2>
</header>
<nav id="tabs"><ul>
<li><a href="index.html">About</a>
<li><a href="Installation.html">Installation</a>
<li><a href="PDI_usage.html">PDI usage</a>
<li><a href="Concepts.html">Core concepts</a>
<li><a href="First_steps.html">First steps</a>
<li><a href="Hands_on.html">Tutorial</a>
<li><a href="PDI_example.html">PDI example</a>
<li><a href="modules.html">C API reference</a>
<li><a href="Specification_tree_ref.html">Specification tree</a>
<li><a href="Plugins.html">Plugins</a>
<li id="gitlab-ribbon"><a rel="noopener" href="https://gitlab.pdi.dev" target="_blank">Contribute on Gitlab</a></li>
</ul></nav>
<!-- Generated by Doxygen 1.9.6 -->
</div><!-- top -->
<div id="side-nav" class="ui-resizable side-nav-resizable">
<div id="nav-tree">
<div id="nav-tree-contents">
<div id="nav-sync" class="sync"></div>
</div>
</div>
<div id="splitbar" style="-moz-user-select:none;"
class="ui-resizable-handle">
</div>
</div>
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:d3d9a9a6595521f9666a5e94cc830dab83b65699&amp;dn=expat.txt MIT */
$(document).ready(function(){initNavTree('Concepts.html',''); initResizable(); });
/* @license-end */
</script>
<div id="doc-content">
<div><div class="header">
<div class="headertitle"><div class="title">Core Concepts </div></div>
</div><!--header-->
<div class="contents">
<div class="textblock"><p><a class="anchor" id="md_Concepts"></a> </p>
<h1><a class="anchor" id="pdi_philosophy"></a>
Philosophy of PDI</h1>
<p>Similarly to aspect-oriented programming, <a class="el" href="namespacePDI.html">PDI</a> distinguishes between core concerns and cross-cutting concerns.</p>
<p>Core concerns are the aspects of the code that fulfill its main goal. We consider as core concerns of a simulation code the aspects handled in the main loop that generate elements needed as input of the next iteration of the loop.</p>
<p>Cross-cutting concerns are the aspect of the code that are not core concerns. We consider as cross-cutting concerns of a simulation code:</p>
<p>the aspects handled before the simulation main loop such as parameters reading, data initialization, etc, the aspects handled after the simulation main loop, such as result post-processing, storage to disk, visualization, etc, the aspects handled during the simulation main loop, but whose execution is not required for the next iteration of the loop, such as fault tolerance, logging, etc. The simulation could run with none of the cross-cutting concerns implemented (even if this would be completely useless with no result ever saved). The deactivation of a core concern on the other hand would lead to a failure of the simulation.</p>
<p><a class="el" href="namespacePDI.html">PDI</a> supports calling libraries from the specification tree instead of from the code. This is well suited for cross-cutting concerns but means that the code has no control over what and how these aspects are handled which does not fit the needs of core concerns.</p>
<h1><a class="anchor" id="PDI_description"></a>
PDI description</h1>
<p>PDI offers to exchange data between the application code and various external data handlers, such as for example the file-system for I/O or another code for code-coupling.</p>
<p>Before using PDI, it is however a good idea to understand the three core concepts that make this possible.</p>
<p>Each data exchange is a two-step process. The PDI-enabled application makes its data available through the <a class="el" href="Concepts.html#Data_store">data store</a>. The <a class="el" href="Concepts.html#Event_subsystem">event subsystem</a> then notifies interested external data handlers about this change in the store. Once notified, each data handler (implemented as a plugin) can look in the store and use the data accessible from there.</p>
<p>The orchestration of these exchanges, the description of what data can be put in the store, what each handler should do with it, etc. is described in the <a class="el" href="Concepts.html#Specification_tree">specification tree</a>.</p>
<div class="image">
<img src="PDI_schema.jpg" alt=""/>
<div class="caption">
PDI structure schema</div></div>
<h1><a class="anchor" id="Data_store"></a>
Data store</h1>
<p>The data store is the mechanism provided by PDI to handle <em>data transfer</em> between the application code and external data handlers. Data transfer is the action of making data available to another part of the code. For example, in a function call the list of parameters determines data transfer.</p>
<p>PDI data store is somewhat similar to a file-system or a document store with some specific properties:</p><ul>
<li>unlike in traditional file-systems, data stored in PDI store is typed, one can differentiate between a record with named members and an array for example;</li>
<li>each process contains a distinct instance of the store, inter-process communications might be offered by data handlers accessing objects in the store and exchanging them between process boundaries, but not by PDI itself;</li>
<li>storing an object in PDI store is cheap as it does not triggers any copy, instead the store holds a reference to the exact same object in memory as that manipulated by the code;</li>
<li>to prevent invalid concurrent accesses to the objects, the stores offers a mutual exclusion mechanism where only one handler can access the object for write but concurrent read access is possible.</li>
</ul>
<p>This approach makes it possible to exchange data between very weakly coupled code modules. Each module can add or access objects in the store and does not need to know which other module created it or how.</p>
<p>In summary, the store hold a set of object references, each identified by a name and that contains:</p><ul>
<li>the address of the buffer storing the data (a pointer),</li>
<li>the description of the type of the data (memory layout and interpretation),</li>
<li>a RW-lock to ensure exclusive access.</li>
</ul>
<h1><a class="anchor" id="Event_subsystem"></a>
Event subsystem</h1>
<p>While the <a class="el" href="Concepts.html#Data_store">data store</a> handles data transfer between the application code and external data handlers, the event subsystem handles <em>control</em> transfer. Control transfer is the action of passing the CPU control to another part of the code. For example, calling a function is a way to transfer control, creating a thread is another way.</p>
<p>The event subsystem makes it possible to observe the store and to be notified when it is accessed; thus complementing the store with a way for data handler to implement their expected behavior. Notifications are emitted when:</p><ul>
<li>a reference becomes available in the store,</li>
<li>a reference ceases to be available in the store,</li>
<li>someone accesses a reference that is not in the store,</li>
<li>a <a class="el" href="group__annotation.html#gab21d59fd8d6532f6b8d7a4ac69a2388b">named event</a> is emitted.</li>
</ul>
<p>To ensure minimal overhead, the PDI event subsystem is synchronous by default (like a function call). Plugins can implement other behaviors on top of that. For example, a plugin could create a thread for asynchronous execution.</p>
<p>This approach makes it possible to exchange data between very weakly coupled code modules. Each module can execute specific code when the required data becomes available and does not need to know which other module created it or how.</p>
<h1><a class="anchor" id="Specification_tree"></a>
Specification tree</h1>
<p>The combination of the data transfer provided by the <a class="el" href="Concepts.html#Data_store">data store</a> and the control transfer provided by the <a class="el" href="Concepts.html#Event_subsystem">event subsystem</a> offers the basis for weak coupling of multiple independent modules. The specification tree builds on this basis and orchestrates the interaction between the multiple modules used in an execution.</p>
<p>The specification tree is specified in a file written in the <a href="https://en.wikipedia.org/wiki/YAML">YAML</a> format and provided to PDI at <a class="el" href="group__init__final.html#ga3dc660be40c93c169337e3d2692b2ed0">initialization</a>. This makes it possible to change the list of modules to load and their interactions without having to recompile any code.</p>
<p>The specification tree structure is described in details in its <a class="el" href="Specification_tree_ref.html">reference documentation</a>. It contains two main subparts:</p><ul>
<li>the data types description,</li>
<li>the plugin list configuration.</li>
</ul>
<p>The data types description defines the type of data expected in the store. This is most useful for non-reflexive statically typed languages (C, C++, Fortran) for which this information can not be automatically extracted at execution. The types can be expressed in function of the value of other data through <code>$-expressions</code>. For example, a given object might be described as an array of <code>N</code> <code>doubles</code> where <code>N</code> is an integer available elsewhere in the store.</p>
<p>The plugin list configuration defines the list of plugins to load and a configuration for each of them. Each plugin can accept a different way of specifying the configuration, but in any case, this is where the orchestration of interactions is specified. The <a class="el" href="user_code_plugin.html">user-code plugin</a> is somewhat specific in that instead of providing a service itself, it enables the application to react to events and implement specific code handle data from the store.</p>
<h1><a class="anchor" id="autotoc_md3"></a>
Conclusion</h1>
<p>To summarize, interactions between weakly coupled modules in PDI go through the <a class="el" href="Concepts.html#Data_store">data store</a> that acts somewhat like a file-system. One can share data buffers in this store to make them available to other modules. When a buffer becomes available in the store, the <a class="el" href="Concepts.html#Event_subsystem">event subsystem</a> notifies interested modules so that they can use the data. The plugins loaded and configured through the <a class="el" href="Concepts.html#Specification_tree">specification tree</a> offer various reusable services through that mechanism such as data write to disk, fault tolerance, code coupling, etc. </p>
</div></div><!-- contents -->
</div><!-- PageDoc -->
</div><!-- doc-content -->
</body>
</html>
Loading

0 comments on commit 47fc1c2

Please sign in to comment.