Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for Unstructured Grid Exchange Between Computing and I/O Nodes Using ADIOS2 #4460

Open
AbooooodA opened this issue Feb 11, 2025 · 0 comments

Comments

@AbooooodA
Copy link

AbooooodA commented Feb 11, 2025

I am working on a distributed simulation where N computing nodes exchange unstructured grid data using MPI. Currently, at each iteration, the updated data is written to a common output file in HDF5.

Now, I would like to introduce M staging nodes dedicated to I/O and analytics. The goal is to have the N computing nodes communicate their unstructured grid data to a middleware, which then redistributes it to the M staging nodes. At each iteration, computing nodes should update the staging nodes with their new data, which is then further redistributed to the M analytics nodes.

The unstructured grid consists of both cell-centered and point-centered data, which can be scalar or vectorial. I would like to know if ADIOS2 currently supports this workflow, or if there are existing examples demonstrating a similar use case.

Use Case and Requirements

  • When simulation starts, the computing nodes share a global unstructured mesh, which is partitioned across them using our own library (or potentially via Scotch).
    -Each computing node needs to send its partial graph description (mesh connectivity, points, and cells) to ADIOS2.
  • ADIOS2 should reconstruct the graph and redistribute it to M analytics nodes.
  • During each iteration, computing nodes send their updated cell and point data (including scalar and vector fields) to ADIOS2.
  • ADIOS2 then redistributes the required data to the analytics nodes.
  1. Does the current release of ADIOS2 support this workflow?
  2. If yes, are there existing examples in the repository demonstrating a similar implementation?
  3. I have come across implementations that use VTK.xml to store graph and mesh information for visualization. (https://adios2.readthedocs.io/en/latest/ecosystem/visualization.html)
    -Can this approach be used to facilitate graph exchange, reconstruction and redistribution in ADIOS2?
    -Are there any available examples of such an implementation?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant