22.12.1

Update SSD version.

22.12

Main notes

The new version of DeepPhysX brings a lot of changes with the data storage and the data flow between components.
Instead of using Numpy tensors referenced as input and output of neural networks, data is now streamed on a SQL Database that can contain any data field.
This allows to easily design new data flows and to interface with more complex network architectures and learning algorithms.

The SQL Database features are implemented in an external Python library - SimulationSimpleDatabase - that must be installed to use this release.
--> quick install with pip install SimulationSimpleDatabase

Changes

Dataset

• the project now uses SimulationSimpleDatabase to store the Dataset;
• any data field can be added to the Database;
• you can convert your previouly generated datasets to match the new release using DeepPhysX.Core.Utils.converter.DatasetConverter tools;
• some API updates for DatabaseConfig (see examples/tutorial for further details).

  from DeepPhysX.Core.Utils.converter import DatabaseConverter
  
  # Considering that your dataset is in the working session 'my_session/dataset/<dataset_partitions>'
  converter = DatabaseConverter(session_path='my_session')
  converter.numpy_to_database(batch_size=16, max_file_size=3, normalize=True)

Environment

• the project now uses SimulationSimpleDatabase for the visualization tool, see the dedicated documentation to use the Factory;
• any data field can be added to the Database, they must be initialized in the new init_database method, using the set_training_data and set_additional_data methods;
• training data and additional data must be defined using the set_training_data and set_additional_data methods respectively, using the fields defined at initialization;
• the apply_prediction method now receives a dictionary that contains all the fields produced by the Network (these data fields are defined in the Network);
• some API updates for BaseEnvironment and BaseEnvironmentConfig (see examples/tutorial for further details).

  from DeepPhysX.Core.Environment.BaseEnvironment import BaseEnvironment
  
  class MyEnvironment(BaseEnvironment):
  
      def __init__(self, as_tcp_ip_client=True, instance_id=1, instance_nb=1, **kwargs):
          BaseEnvironment.__init__(self, as_tcp_ip_client=as_tcp_ip_client, instance_id=instance_id, instance_nb=instance_nb)
  
      def init_database(self):
          self.define_training_fields(fields=[('input', ndarray), ('ground_truth', ndarray)])
          self.define_additional_fields(fields=[('value', int)])
  
      def init_visualization(self):
          self.factory.add_mesh(positions=..., cells=...)
  
      def step(self):
          self.set_training_data(input=..., ground_truth=...)
          self.set_additional_data(value=...)
  
      def apply_prediction(self, prediction):
          network_output = prediction['prediction'] # Depends on the fields defined in the Network

Network

• the Network object now has three variables that must be set to define the data field names: net_fields for the input data fields (should match the data field names defined in the Environment), opt_fields for the data used for the optimization process, pred_fields for the data that will be returned to the Environment;
• some API updates for BaseNetwork and BaseNetworkConfig (see examples/tutorial for further details).

  from DeepPhysX.Core.Network.BaseNetwork import BaseNetwork
  
  class MyNetwork(BaseNetwork):
    
        def __init__(self, config):
            BaseNetwork.__init__(self, config)
            self.net_fields = ['input']
            self.opt_fields = ['ground_truth']
            self.pred_fields = ['prediction']

22.06

Main notes

The DPX project provides easy solutions to interface AI with numerical simulations.
The project provides a Core package with additional compatibility layers for external AI and simulations frameworks.
The project includes three main AI pipelines:

• Generate a dataset with synthetic data from numerical simulations;
• Train an artificial neural network with a synthetic dataset;
• Use the prediction of trained networks in a numerical simulation.

This version is the first stable release of the project.
It provides a Core package with 2 corresponding SOFA & PyTorch compatible layers. It also provides a documentation page and examples with shared training data.

Features

Dataset

• Automatic training dataset storage and loading with multiple files management;
• Dataset shuffle and normalization;
• Multiple dataset modes: Training, Validation, Prediction;
• Customizable dataset fields.

Simulation

• Data generation achieved by several simulations running in multiprocessing with a client-server architecture;
• Operation with internal data, from the dataset or from the neural network;
• Increased interactions with other components (dataset, neural network, visualizer);
• Check the validity of the training data;
• A visualization Factory to init, update and render the simulated objects (written with Vedo).

Network

• Automatic storage and loading of networks during training;
• Customizable data transformations at each step (forward pass, optimization, prediction apply);
• Customizable optimization process with training data;
• An analysis of the evolution of the training session (written with Tensorboard);
• Already implemented architectures: FC, UNet.