Skip to content

Commit

Permalink
Use generated git version in configure.ac. More slight cleanups to do…
Browse files Browse the repository at this point in the history
…cs and examples.
  • Loading branch information
tskisner committed Aug 1, 2017
1 parent 2dda878 commit 0081639
Show file tree
Hide file tree
Showing 5 changed files with 48 additions and 40 deletions.
2 changes: 1 addition & 1 deletion configure.ac
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ dnl
dnl +------------------------
dnl | Initialize package info
dnl +------------------------
AC_INIT([TIDAS], [0.1.0], [[email protected]], [tidas], [https://github.com/hpc4cmb/tidas])
AC_INIT([TIDAS], m4_esyscmd_s([m4/git-version.sh]), [[email protected]], [tidas], [https://github.com/hpc4cmb/tidas])
AC_CONFIG_SRCDIR([Makefile.am])
AM_INIT_AUTOMAKE([foreign])
AC_CONFIG_HEADERS(config.h)
Expand Down
11 changes: 6 additions & 5 deletions docs/sphinx/group.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,16 @@ Groups

A "group" contains a collection of timestreams with consistent sampling. In other words, each timestream in the group has the same number of samples. Each timestream has a name, a data type, and a string describing the units. This metadata for one timestream is called a "field". The collection of fields used in a group is called the "schema" of the group. A group's schema is fixed at creation time, but the number of samples may be changed as needed. A group is associated with a parent block. Read and write operations on the data in a group is passed off to the "backend" class in use. A group may also have a dictionary of scalar properties associated with it.

C++
~~~~~~~~~

.. doxygenclass:: tidas::group
:members:

Python
~~~~~~~~~

.. autoclass:: tidas.Group
:members:


C++
~~~~~~~~~

.. doxygenclass:: tidas::group
:members:
18 changes: 10 additions & 8 deletions docs/sphinx/intervals.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,6 @@ Intervals

When dealing with time domain data, it is very common to specify ranges of time that have some meaning. These might be spans of good or bad data, or time ranges that enclose some important event or feature. This use case is so common that TIDAS has a built in object for this. An "intervals" object represents a list of time ranges. Each span can also optionally have a range of sample indices, which is useful if the intervals are associated with a particular group.

C++
~~~~~~~~~

.. doxygenclass:: tidas::intrvl
:members:

.. doxygenclass:: tidas::intervals
:members:

Python
~~~~~~~~~
Expand All @@ -25,3 +17,13 @@ Python
:members:


C++
~~~~~~~~~

.. doxygenclass:: tidas::intrvl
:members:

.. doxygenclass:: tidas::intervals
:members:


47 changes: 27 additions & 20 deletions docs/sphinx/volume.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,20 @@ Volumes

A volume is the top-level TIDAS object. It defines the "backend" data format and holds a metadata database for fast(er) data selection and query operations. It also contains the root of a hierarchy (a tree) of "blocks".


Planning the Layout of a Volume
--------------------------------------

An important feature of TIDAS volumes is the ability to open / copy / link only a subset of the data. This selection operation is performed with a regular expression match on the tree of block names. This means that it is critical to plan out the organization of blocks to make it possible to do any desired data selections.

Most data collected from monitoring systems, experimental apparatuses, etc, have natural ways that the data might be split up into pieces. For example, if the data collection is started fresh each day, you might have a block for each day. If there are different sorts of data within one day which you might want to frequently split up, then you could have sub-blocks in each day for the different data types. Going the other direction up the hierarchy, you might organize the days into blocks for each month, each quarter, each year, etc.


Interface
---------------

C++
~~~~~~~~~
Here is a basic reference for the volume interface.

.. doxygenclass:: tidas::volume
:members:
:no-link:

Python
~~~~~~~~~
Expand All @@ -23,17 +28,19 @@ Python
:members:


C++
~~~~~~~~~

.. doxygenclass:: tidas::volume
:members:
:no-link:


Blocks
----------------

A "block" is simply a logical collection of data which has a "name" (a string) associated with it. A block can contain data itself, other blocks, or both. A volume contains a special "root" block, which is the top of the tree / hierarchy of blocks in the volume.

C++
~~~~~~~~~

.. doxygenclass:: tidas::block
:members:

Python
~~~~~~~~~
Expand All @@ -42,12 +49,11 @@ Python
:members:


Planning the Layout of a Volume
--------------------------------------

An important feature of TIDAS volumes is the ability to open / copy / link only a subset of the data. This selection operation is performed with a regular expression match on the tree of block names. This means that it is critical to plan out the organization of blocks to make it possible to do any desired data selections.
C++
~~~~~~~~~

Most data collected from monitoring systems, experimental apparatuses, etc, have natural ways that the data might be split up into pieces. For example, if the data collection is started fresh each day, you might have a block for each day. If there are different sorts of data within one day which you might want to frequently split up, then you could have sub-blocks in each day for the different data types. Going the other direction up the hierarchy, you might organize the days into blocks for each month, each quarter, each year, etc.
.. doxygenclass:: tidas::block
:members:


MPI Volumes
Expand All @@ -64,16 +70,17 @@ see those changes until the synchronization method is called.
Multiple processes writing to the same group or intervals object is not supported, and may corrupt the data files for those objects!


C++
Python
~~~~~~~~~

.. doxygenclass:: tidas::mpi_volume
.. autoclass:: tidas.mpi_volume.MPIVolume
:members:
:no-link:

Python

C++
~~~~~~~~~

.. autoclass:: tidas.mpi_volume.MPIVolume
.. doxygenclass:: tidas::mpi_volume
:members:
:no-link:

10 changes: 4 additions & 6 deletions examples/demo_weather.py
Original file line number Diff line number Diff line change
Expand Up @@ -74,8 +74,8 @@
# Add a block for this year
yb = root.block_add(year)

for monthnum in range(1, 13):
# Add a block for the month
for monthnum in range(1, 2):
# Add a block for the month. Just use January for now.
month = calendar.month_abbr[monthnum]
mb = yb.block_add(month)

Expand Down Expand Up @@ -155,8 +155,6 @@
# interested in the speed, not the direction. Let's extract just the wind
# speed data for the month of January, 2019.

print("Small demo")

file = "demo_weather_small"
if os.path.isdir(file):
shutil.rmtree(file)
Expand All @@ -167,6 +165,7 @@

# Take a quick peek at the small data volume:

print("\nSmall volume with just wind speed:\n")
with tidas.Volume(file) as vol:
vol.info()

Expand All @@ -176,8 +175,6 @@
# new derived data products. We can make a volume which links to the original
# data and then add new groups to this.

print("Link demo")

file = "demo_weather_link"
if os.path.isdir(file):
shutil.rmtree(file)
Expand Down Expand Up @@ -222,5 +219,6 @@
# in this volume. However, note that trying to write to these would fail
# if the filesystem permissions were read-only.

print("\nVolume linked to original with new local Intervals:\n")
with tidas.Volume(file) as vol:
vol.info()

0 comments on commit 0081639

Please sign in to comment.