Initialize NXazint contribution #3

woutdenolf · 2024-06-28T19:10:04Z

Current NXazint Status

The docs are build daily from the main branch: NXDL --> HTML

First time contributors

Clone the project locally and install all python packages needed to build the docs

python -m venv azintenv
source azintenv/bin/activate

git clone [email protected]:nexuscontributions/nxazint.git
cd nxazint

make install

Build the docs locally

source azintenv/bin/activate
make local
firefox build/manual/build/html/index.html

Existing application definitions that are related

NXmonopd: Monochromatic Neutron and X-Ray Powder diffractometer.
NXcanSAS: Implementation of the canSAS standard to store reduced small-angle scattering data of any dimension.

Initialize NXazint contribution

fork official repo https://github.com/nexusformat/definitions
add at least one MAXIV member as owners to the fork
initialize NXazint (initialize the NXazint contribution #1 )
setup NeXus documentation rendering for the fork here
create PR of main to official NeXus repo (NXazint contribution nexusformat/definitions#1395)
convert the MAXIV proposal to the first NXazint version (Parse MaX IV proposal #2 )
host MAXIV proposal example here

The text was updated successfully, but these errors were encountered:

woutdenolf · 2024-06-28T19:42:17Z

Ping @meghdadyazdi Please accept the invitation and invite other MAXIV members to the nxazint team on github.

kif · 2024-07-02T07:47:02Z

FYI, pyFAI is already compatible with:

fgjorup · 2024-07-10T10:28:00Z

Feel free to add me when you get the chance @meghdadyazdi

woutdenolf · 2024-07-17T22:12:37Z

Seems like +1 length for edges is actually allowed in NXdata but got lost over time in the docs: nexusformat#1396 (comment). So good news for this MR.

zdemat · 2024-08-05T13:12:43Z

Feel free to add me here @woutdenolf , my GitHub user name is @zdemat.

zdemat · 2024-09-27T08:50:10Z

@woutdenolf just a status report on this:

we would like to refactor this contributions at MAX IV
- we likely want to split 1D and 2D case
- we want to make NXazit rather a base class that beside data keep data about azint procedure parameters and sw
- we would like to add Application classes that refer to the NXazint
we would like to discuss this more with ESRF beamline staff
we ... refers here to a MAX IV working group of more than 5 people

fgjorup · 2024-09-27T14:03:27Z

Just to elaborate a bit on Zdenek's comment:
We suggest a base class, for now called NXazint, and a series of application classes. I don't know whether all of those application classes needs to be strictly defined in NeXus, or if they should just be suggestions to data structuring. The idea would be to have one or more application classes as NXsubentries in one unified NXentry, which also contains the common instrument information (monochromator, source, etc.), which the subentries soft link to.

Unfortunately I could not attach the example.h5 here on github, but here is a rough summary tree:

entry/ - (NXentry)
entry/instrument/ - (NXinstrument)
    entry/instrument/azint1d/ - (NXazint)
        entry/instrument/azint1d/data - (NX_FLOAT)
        entry/instrument/azint1d/process/ - (NXprocess)
            entry/instrument/azint1d/process/date - (NX_DATE_TIME)
            entry/instrument/azint1d/process/note - (NX_CHAR)
            entry/instrument/azint1d/process/parameters/ - (NXparameters)
                entry/instrument/azint1d/process/parameters/error_model - (NX_CHAR)
                entry/instrument/azint1d/process/parameters/mask_file - (NX_CHAR)
                entry/instrument/azint1d/process/parameters/num_of_azi_bins - (NX_INT)
                entry/instrument/azint1d/process/parameters/num_of_radial_bins - (NX_INT)
                entry/instrument/azint1d/process/parameters/pixel_splitting - (NX_INT)
                entry/instrument/azint1d/process/parameters/polarization_factor - (NX_FLOAT)
                entry/instrument/azint1d/process/parameters/poni - (NX_CHAR)
                entry/instrument/azint1d/process/parameters/solid_angle_correction - (NX_BOOLEAN)
                entry/instrument/azint1d/process/parameters/unit - (NX_CHAR)
                entry/instrument/azint1d/process/parameters/wavelength - (NX_FLOAT)
            entry/instrument/azint1d/process/program - (NX_CHAR)
            entry/instrument/azint1d/process/reference - (NX_CHAR)
            entry/instrument/azint1d/process/version - (NX_CHAR)
        entry/instrument/azint1d/radial_axis - (NX_FLOAT)
    entry/instrument/azint2d/ - (NXazint)
        entry/instrument/azint2d/azimuthal_axis - (NX_FLOAT)
        entry/instrument/azint2d/data - (NX_FLOAT)
        entry/instrument/azint2d/data_errors - (NX_FLOAT)
        entry/instrument/azint2d/process/ - (NXprocess)
            entry/instrument/azint2d/process/date - (NX_DATE_TIME)
            entry/instrument/azint2d/process/note - (NX_CHAR)
            entry/instrument/azint2d/process/parameters/ - (NXparameters)
                entry/instrument/azint2d/process/parameters/error_model - (NX_CHAR)
                entry/instrument/azint2d/process/parameters/mask_file - (NX_CHAR)
                entry/instrument/azint2d/process/parameters/num_of_azi_bins - (NX_INT)
                entry/instrument/azint2d/process/parameters/num_of_radial_bins - (NX_INT)
                entry/instrument/azint2d/process/parameters/pixel_splitting - (NX_INT)
                entry/instrument/azint2d/process/parameters/polarization_factor - (NX_FLOAT)
                entry/instrument/azint2d/process/parameters/poni - (NX_CHAR)
                entry/instrument/azint2d/process/parameters/solid_angle_correction - (NX_BOOLEAN)
                entry/instrument/azint2d/process/parameters/unit - (NX_CHAR)
                entry/instrument/azint2d/process/parameters/wavelength - (NX_FLOAT)
            entry/instrument/azint2d/process/program - (NX_CHAR)
            entry/instrument/azint2d/process/reference - (NX_CHAR)
            entry/instrument/azint2d/process/version - (NX_CHAR)
        entry/instrument/azint2d/radial_axis - (NX_FLOAT)
    entry/instrument/monochromator/ - (NXmonochromator)
        entry/instrument/monochromator/energy - (NX_FLOAT)
        entry/instrument/monochromator/wavelength - (NX_FLOAT)
    entry/instrument/name - (NX_CHAR)
    entry/instrument/source/ - (NXsource)
        entry/instrument/source/name - (NX_CHAR)
        entry/instrument/source/probe - (NX_CHAR)
        entry/instrument/source/type - (NX_CHAR)
entry/integration1d/ - (NXsubentry)
    entry/integration1d/data/ - (NXdata)
    entry/integration1d/definition
    entry/integration1d/instrument/ - (NXinstrument)
entry/integration2d/ - (NXsubentry)
    entry/integration2d/data/ - (NXdata)
    entry/integration2d/definition
    entry/integration2d/instrument/ - (NXinstrument)
entry/xrdmap/ - (NXsubentry)
    entry/xrdmap/data/ - (NXdata)
        entry/xrdmap/data/I - (NX_FLOAT)
    entry/xrdmap/definition

zdemat · 2024-09-30T12:20:10Z

There is NXiqproc. It is an application class and there are more classes like this. I think the advantage of the application class is that we may be more strict in requirements on the present fields and their names.

@SAnsell can NXiqproc somehow cover the QxQy case?

woutdenolf · 2024-09-30T13:54:49Z

Note that we can now use single class inheritance in NeXus.

zdemat · 2024-09-30T14:41:53Z

That is a good point but NXiqproc is really quite an Application class and we do not want most of its fields (instrument, sample etc.) and specialized fields like qx etc. The idea of NXazint (not relevant if it is base-class or application-class) is that the AZINT program, script, pipeline can write it independently of other info, as e.g. detector data.

fgjorup · 2024-10-24T13:19:00Z

Summary of meeting 24/10-2024

Scrap nxazint base class and move data and process directly to the application classes
Define 5 application classes: azint1d, azint2d, azintmap1d, azintmap2d, azintQxQy. (names are subject to change)
Include definitions of allowed units and long names (degrees, 1/angstrom, angstrom, etc.)

zdemat · 2024-10-24T13:58:22Z

Hi,

I agree with the 3 points
note that it was effectively already done with this commit on the separate-2d branch. There is now just single NXazint class that covers only "azint1d". I think we just need to rename it and include the long names.
once the 1d class is polished, we can build the 2d class ... it may happen that they will have much in common and we can use inheritance, but I prefer to do it step by step

zdemat · 2024-10-24T14:03:05Z

@woutdenolf is there any CI to build & update the "web-representation" and publish it? I think the initial one at ESRF is too old and most of the activities are happening at MAX IV now. However for some of us it can be complicated to understand the source definition. We can build CI at MAX IV but we are behind firewall and it will not be visible outside.

woutdenolf · 2024-10-25T06:56:50Z

is there any CI to build & update the "web-representation" and publish it?

Unfortunately not. For the main branch of this repo I have a CI running every day on our gitlab (https://gitlab.esrf.fr/hdf5/nexus/nxazint) which publishes here: https://hdf5.gitlab-pages.esrf.fr/nexus/nxazint/classes/contributed_definitions/NXazint.html.

The CI of the main NeXus repo builds documentation which I'm guessing you can download somehow? https://github.com/nexusformat/definitions/actions/workflows/ci.yaml

Edit: we have it too but I don't know where to download the HTML files: https://github.com/nexuscontributions/nxazint/actions/workflows/ci.yaml.

woutdenolf · 2024-10-25T07:05:00Z

I think the initial one at ESRF is too old

It builds the main branch of this repo every day.

woutdenolf · 2024-10-25T07:06:20Z

I just rebased to main branch of this repo on the main branch of the official repo.

zdemat · 2024-10-25T07:11:17Z

note that the current NXazint is in the separate-2d branch. It is the current 1D case, so it should be renamed.
a CI build at ESRF once per day is I think OK for us

pjb671 · 2024-12-06T14:36:46Z

In an attempt to help my overloaded colleagues here, I made the separate 2d class to complement the 1d case that Zdenek made in the "separate-2d" branch. Should I push this to that branch, and I think I need adding to this project to be able to do so. What I have done though also does not mirror exactly Zdenek's 1d case, as we have in the mean time implemented something at DanMAX. So there can be something to discuss when I push the 2d application definition.

pjb671 · 2024-12-06T14:38:01Z

Also, how does one see the html rendering now? I can see my changes locally of course, but is there a public page being generated by the CI and if so what is the address? This page linked above is not found https://nxazint-hdf5-nexus-3229ecbd09ba8a773fbbd8beb72cace6216dfd5063e1.gitlab-pages.esrf.fr/classes/contributed_definitions/NXazint.html

zdemat · 2024-12-06T15:23:50Z

@pjb671 if you push here (in seprate-2d) the rendering will be visible at ESRF link within a day.

woutdenolf · 2024-12-06T17:48:56Z

To render:

make install   # this installs packages in your current python environment
make local
firefox build/manual/build/html/index.html

woutdenolf · 2024-12-06T17:54:57Z

I think I need adding to this project to be able to do so

I invited you. Your colleagues should have the rights to do that as well.

a CI build at ESRF once per day is I think OK for us

To ESRF CI only builds the main branch. If you want me to change that let me know.

pjb671 · 2024-12-09T10:52:00Z

Hi, thanks for adding me. I have pushed something to the separate-2d branch. If we could build the html for this branch it would help Mads @mrvj and @fgjorup be able to review, as I suppose only Zdenek, Meghdad and now me will build locally. You should see now an azint1d, which corresponds still to the last commit of Zdenek, and an azint2d which corresponds to what I believe was the last discussion with Wout and Fredrik and to what we have implemented at DanMAX (only). I suspect I am still mixing up NeXuS vs choice of implementation in a file, as the multimodal files from DanMAX contain both 1d and 2d application definitions, with soft links to avoid duplication of the instrument (etc) but perhaps this is not what we need to document here.

pjb671 · 2024-12-09T10:57:42Z

I have not looked too closely at long names etc (which were discussed and fixed in the 1d case) as the general format needs understanding first.

pjb671 · 2025-01-09T15:26:15Z

Hi @woutdenolf , returning to this after the holidays ... "ESRF CI only builds the main branch. If you want me to change that let me know." ... could you build the "separate-2d " branch so we can all see the html?

woutdenolf · 2025-01-09T18:42:04Z

HTML rendering of NXazint1d and NXazint2d

woutdenolf self-assigned this Jun 28, 2024

woutdenolf changed the title ~~Initialize azint contribution~~ Initialize NXazint contribution Jun 28, 2024

woutdenolf mentioned this issue Jun 28, 2024

NXazint contribution nexusformat/definitions#1395

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Initialize NXazint contribution #3

Initialize NXazint contribution #3

woutdenolf commented Jun 28, 2024 •

edited

Loading

woutdenolf commented Jun 28, 2024 •

edited

Loading

kif commented Jul 2, 2024 •

edited

Loading

fgjorup commented Jul 10, 2024

woutdenolf commented Jul 17, 2024

zdemat commented Aug 5, 2024

zdemat commented Sep 27, 2024 •

edited

Loading

fgjorup commented Sep 27, 2024

zdemat commented Sep 30, 2024

woutdenolf commented Sep 30, 2024

zdemat commented Sep 30, 2024

fgjorup commented Oct 24, 2024

zdemat commented Oct 24, 2024

zdemat commented Oct 24, 2024

woutdenolf commented Oct 25, 2024 •

edited

Loading

woutdenolf commented Oct 25, 2024

woutdenolf commented Oct 25, 2024

zdemat commented Oct 25, 2024

pjb671 commented Dec 6, 2024 •

edited

Loading

pjb671 commented Dec 6, 2024 •

edited

Loading

zdemat commented Dec 6, 2024

woutdenolf commented Dec 6, 2024 •

edited

Loading

woutdenolf commented Dec 6, 2024

pjb671 commented Dec 9, 2024 •

edited

Loading

pjb671 commented Dec 9, 2024

pjb671 commented Jan 9, 2025

woutdenolf commented Jan 9, 2025

Initialize NXazint contribution #3

Initialize NXazint contribution #3

Comments

woutdenolf commented Jun 28, 2024 • edited Loading

Current NXazint Status

First time contributors

Existing application definitions that are related

Initialize NXazint contribution

woutdenolf commented Jun 28, 2024 • edited Loading

kif commented Jul 2, 2024 • edited Loading

fgjorup commented Jul 10, 2024

woutdenolf commented Jul 17, 2024

zdemat commented Aug 5, 2024

zdemat commented Sep 27, 2024 • edited Loading

fgjorup commented Sep 27, 2024

zdemat commented Sep 30, 2024

woutdenolf commented Sep 30, 2024

zdemat commented Sep 30, 2024

fgjorup commented Oct 24, 2024

zdemat commented Oct 24, 2024

zdemat commented Oct 24, 2024

woutdenolf commented Oct 25, 2024 • edited Loading

woutdenolf commented Oct 25, 2024

woutdenolf commented Oct 25, 2024

zdemat commented Oct 25, 2024

pjb671 commented Dec 6, 2024 • edited Loading

pjb671 commented Dec 6, 2024 • edited Loading

zdemat commented Dec 6, 2024

woutdenolf commented Dec 6, 2024 • edited Loading

woutdenolf commented Dec 6, 2024

pjb671 commented Dec 9, 2024 • edited Loading

pjb671 commented Dec 9, 2024

pjb671 commented Jan 9, 2025

woutdenolf commented Jan 9, 2025

woutdenolf commented Jun 28, 2024 •

edited

Loading

woutdenolf commented Jun 28, 2024 •

edited

Loading

kif commented Jul 2, 2024 •

edited

Loading

zdemat commented Sep 27, 2024 •

edited

Loading

woutdenolf commented Oct 25, 2024 •

edited

Loading

pjb671 commented Dec 6, 2024 •

edited

Loading

pjb671 commented Dec 6, 2024 •

edited

Loading

woutdenolf commented Dec 6, 2024 •

edited

Loading

pjb671 commented Dec 9, 2024 •

edited

Loading