unpack schema sample data into folders, produce mef files at build time #7457

jodygarnett · 2023-10-26T21:07:48Z

Following example of HNAP plugin metadata101/iso19139.ca.HNAP#350 I had offered to do the same change for core-geonetwork.

plugin sample data folders are unzipped, so we can manage changes in pull-requests
pom.xml packages up sample data into target/sample-data mef files.
assembly modified to exclude sample-data folders contents, and include generated mef files

The iso19115-3.2018 sample data for service record was ... odd:

was included in the plugin as a zip file, not a mef file.
contents of zip had similar files as the mef format, but arranged in a different folder structure

unzip -l 0dbac056-cd5a-4c9c-bbec-564e47c4507c.zip

0beee14d-aa34-46ba-9f43-de5cf2308b53/
0beee14d-aa34-46ba-9f43-de5cf2308b53/metadata/
0beee14d-aa34-46ba-9f43-de5cf2308b53/public/
0beee14d-aa34-46ba-9f43-de5cf2308b53/private/
info.xml
metadata.xml

This build change is prepared to update sample data for #7363

Checklist

jodygarnett · 2023-10-26T21:24:24Z

@ianwallen you expressed interest in this activity

jodygarnett · 2023-10-26T21:58:04Z

Testing these sample records, appear to work as before - however the records content themselves are outdated.

jodygarnett · 2023-10-27T17:37:47Z

@ianwallen do not have sufficient permissions to request your review. If anyone else is interested this is a straightforward change to test locally.

jodygarnett · 2023-12-06T16:52:43Z

@fxprunayre can this be merged now? I wish to have an updated sample records to document online-help functionality.

fxprunayre · 2023-12-06T16:59:56Z

@fxprunayre can this be merged now? I wish to have an updated sample records to document online-help functionality.

No objection here, but I'm a bit short in time to check it. If you've tested it on projects with @josegar74 or @ianwallen don't hesitate to go ahead.

schemas/dublin-core/pom.xml

schemas/iso19115-3.2018/pom.xml

schemas/iso19139/pom.xml

josegar74 · 2023-12-07T07:14:15Z

@jodygarnett about your comments in the PR:

The iso19115-3.2018 sample data for service record was ... odd
was included in the plugin as a zip file, not a mef file.

MEF is just a regular zip with a folder structure.

contents of zip had similar files as the mef format, but arranged in a different folder structure

It seems a MEF 2 format, that allows multiple metadata included. MEF 1 format only allows 1 metadata.

schemas/iso19139/src/main/plugin/iso19139/sample-data/photographic_session/metadata.xml

jodygarnett · 2023-12-08T05:19:18Z

This should be good to merge and backport

jodygarnett · 2023-12-15T18:05:30Z

feedback has been addressed; I have also rebased against main to ensure the changes are clean.

ianwallen · 2023-12-17T11:38:10Z

@jodygarnett

I did a sample build on my system and I looked for sample-data folder in the target zip file that was created.

I was able to find the sample-data folder in iso19139 and dublin-core
I was not able to find the sample-data folder in schemas/iso19115-3.2018/target/gn-schema-iso19115-3.2018-4.4.2-SNAPSHOT.zip

Can you double check iso19115-3.2018 to ensure that it is properly creating the sample-data folder

jodygarnett · 2024-01-12T23:41:13Z

Can you double check iso19115-3.2018 to ensure that it is properly creating the sample-data folder

I concur, it did not show up as expected - let me fix that.

Checking dublin core the empty folders are not included:

unzip -l gn-schema-dublin-core-4.4.2-SNAPSHOT.zip | grep sample

        0  01-12-2024 15:11   dublin-core/sample-data/
     1201  01-12-2024 15:11   dublin-core/sample-data/natural_polar_ecosystems.mef

Comparing with main branch which includes empty folders:

unzip -l dublin-core.mef

      485  01-12-2011 16:58   info.xml
     1097  03-12-2007 22:02   metadata.xml
        0  03-12-2007 22:02   private/
        0  03-12-2007 22:02   public/

Update: I was able to fix this with a change to the ant zip task : whenempty="create"

jodygarnett · 2024-01-13T01:14:40Z

iso19115-3.2018 service_record.mef now reports back:

unzip -l target/sample-data/service_record.mef

        0  12-15-2023 10:10   metadata/
        0  12-15-2023 10:10   private/
        0  12-15-2023 10:10   public/
      741  12-15-2023 10:10   info.xml
    22322  12-15-2023 10:10   metadata.xml

And this is included in the plugin:

unzip -l target/gn-schema-iso19115-3.2018-4.4.2-SNAPSHOT.zip | grep sample-data

        0  01-12-2024 17:13   iso19115-3.2018/sample-data/
     4602  01-12-2024 17:13   iso19115-3.2018/sample-data/service_record.mef

jodygarnett · 2024-01-13T01:15:31Z

With feedback addressed I am going to rebase, and am in position to merge on Monday.

@fxprunayre you are still listed as a requested reviewer, let me know if you still wish to review this change.

Update: Rebased done for a clean change history suitable for backport

Signed-off-by: Jody Garnett <[email protected]>

josegar74

Works fine, @jodygarnett as the 4.4.2 release is next week, I moved the target to 4.4.3.

jodygarnett · 2024-01-17T16:44:15Z

Works fine, @jodygarnett as the 4.4.2 release is next week, I moved the target to 4.4.3.

Why not merge now?

…ld time (#7457) * unpack iso19139 sample data, create mef file at build time Signed-off-by: Jody Garnett <[email protected]> * unpack iso19155-3.2018 sample data, create mef file at build time Signed-off-by: Jody Garnett <[email protected]> * unpack dublin-core sample data, create mef file at build time Signed-off-by: Jody Garnett <[email protected]> --------- Signed-off-by: Jody Garnett <[email protected]>

jodygarnett force-pushed the sample-data branch from 08ccdee to 90f27d3 Compare October 26, 2023 21:21

josegar74 added this to the 4.4.1 milestone Nov 1, 2023

josegar74 requested review from josegar74 and fxprunayre November 1, 2023 15:39

fxprunayre modified the milestones: 4.4.1, 4.4.2 Nov 22, 2023

josegar74 reviewed Dec 7, 2023

View reviewed changes

schemas/dublin-core/pom.xml Outdated Show resolved Hide resolved

schemas/iso19115-3.2018/pom.xml Outdated Show resolved Hide resolved

schemas/iso19139/pom.xml Outdated Show resolved Hide resolved

josegar74 reviewed Dec 7, 2023

View reviewed changes

schemas/iso19139/src/main/plugin/iso19139/sample-data/photographic_session/metadata.xml Outdated Show resolved Hide resolved

ianwallen added the backport 4.2.x label Dec 8, 2023

jodygarnett force-pushed the sample-data branch from cdea5e1 to 8ce7e25 Compare December 15, 2023 18:10

jodygarnett force-pushed the sample-data branch from 3ee1f38 to 0de59c9 Compare January 13, 2024 01:21

jodygarnett added 3 commits January 12, 2024 17:28

unpack iso19139 sample data, create mef file at build time

d8e6fe3

Signed-off-by: Jody Garnett <[email protected]>

unpack iso19155-3.2018 sample data, create mef file at build time

63d6beb

Signed-off-by: Jody Garnett <[email protected]>

unpack dublin-core sample data, create mef file at build time

43a5369

Signed-off-by: Jody Garnett <[email protected]>

jodygarnett force-pushed the sample-data branch from 0de59c9 to 43a5369 Compare January 13, 2024 01:29

josegar74 modified the milestones: 4.4.2, 4.4.3 Jan 17, 2024

josegar74 approved these changes Jan 17, 2024

View reviewed changes

josegar74 modified the milestones: 4.4.3, 4.4.2 Jan 18, 2024

josegar74 merged commit 22cbf18 into geonetwork:main Jan 18, 2024
6 checks passed

ticheler mentioned this pull request Jan 18, 2024

[Backport 4.2.x] unpack schema sample data into folders, produce mef files at build time #7634

Closed

jodygarnett deleted the sample-data branch January 19, 2024 03:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

unpack schema sample data into folders, produce mef files at build time #7457

unpack schema sample data into folders, produce mef files at build time #7457

jodygarnett commented Oct 26, 2023 •

edited

Loading

jodygarnett commented Oct 26, 2023

jodygarnett commented Oct 26, 2023

jodygarnett commented Oct 27, 2023

jodygarnett commented Dec 6, 2023

fxprunayre commented Dec 6, 2023

josegar74 commented Dec 7, 2023 •

edited

Loading

jodygarnett commented Dec 8, 2023

jodygarnett commented Dec 15, 2023 •

edited

Loading

ianwallen commented Dec 17, 2023 •

edited

Loading

jodygarnett commented Jan 12, 2024 •

edited

Loading

jodygarnett commented Jan 13, 2024

jodygarnett commented Jan 13, 2024 •

edited

Loading

josegar74 left a comment

jodygarnett commented Jan 17, 2024

unpack schema sample data into folders, produce mef files at build time #7457

unpack schema sample data into folders, produce mef files at build time #7457

Conversation

jodygarnett commented Oct 26, 2023 • edited Loading

Checklist

jodygarnett commented Oct 26, 2023

jodygarnett commented Oct 26, 2023

jodygarnett commented Oct 27, 2023

jodygarnett commented Dec 6, 2023

fxprunayre commented Dec 6, 2023

josegar74 commented Dec 7, 2023 • edited Loading

jodygarnett commented Dec 8, 2023

jodygarnett commented Dec 15, 2023 • edited Loading

ianwallen commented Dec 17, 2023 • edited Loading

jodygarnett commented Jan 12, 2024 • edited Loading

jodygarnett commented Jan 13, 2024

jodygarnett commented Jan 13, 2024 • edited Loading

josegar74 left a comment

Choose a reason for hiding this comment

jodygarnett commented Jan 17, 2024

jodygarnett commented Oct 26, 2023 •

edited

Loading

josegar74 commented Dec 7, 2023 •

edited

Loading

jodygarnett commented Dec 15, 2023 •

edited

Loading

ianwallen commented Dec 17, 2023 •

edited

Loading

jodygarnett commented Jan 12, 2024 •

edited

Loading

jodygarnett commented Jan 13, 2024 •

edited

Loading