IATI organisation file support #959

siemvaessen · 2018-01-31T14:33:46Z

We are looking into CoVe for converting IATI organisation file types, next to the activity file.

We don't see support for this in CoVe (yet?) so we are wondering what your thinking is on implementing such a feature and if you do not have short term plans to do so, we'd be happy to talk about how we could contribute to this.

Bjwebb · 2018-02-01T13:08:01Z

Yes, support for this doesn't exist yet, and it makes sense to implement this in CoVE.
Our conversion library flatten-tool nearly supports this, it's just missing a way to name the root XML element (iati-organisations). There's an issue for this OpenDataServices/flatten-tool#150

There's also some discussion in this issue - #732 - about how CoVE might auto detect organisation spreadsheets from the Meta tab.

We have no short terms plans to do anything on this, so would by happy to see you contribute.

siemvaessen · 2018-02-02T08:20:51Z

Thanks @Bjwebb. On the library flatten-tool, any ETA on delivery?

And, where should we look for relevant code in CoVe on the org file detection yet?

Bjwebb · 2018-02-07T11:17:48Z

Sorry for the delay on this, I had a long weekend away at FOSDEM.

We don't have an ETA on that flatten-tool issue atm, but it should be fairly straightforward, so something we can slot into our backlog. How quickly are you hoping to get this functionality?

For the CoVE the changes need to happen in a few places. I'm thinking a call might be useful to talk over these?

siemvaessen · 2018-03-12T11:22:56Z

Hi @Bjwebb - delay in answering. Will CoVE support this in Q2 this year? We'd be happy to contibute, but I'd first like to find out where CoVE now stands with org.file support. We could on the base what is out there continue building.

Bjwebb · 2018-03-19T11:07:25Z

We don't currently have this on our roadmap, but we'd definitely be happy to support you to do this.

siemvaessen · 2018-03-20T14:39:36Z

check. Closing this ticket then.

andylolz · 2018-04-18T12:42:24Z

Any chance of reopening this issue? I’d also find org file support useful. Thanks!

andylolz · 2018-04-19T11:28:50Z

Thanks for reopening, @Bjwebb!

we'd definitely be happy to support you to do this.

I don’t know how far this invitation extends?! But OpenDataServices/flatten-tool#201 is my first blocker. Can I get a sense of whether / when it will be reviewed / merged? Thanks!

Bjwebb · 2018-04-19T12:07:18Z

Thanks, I'll try and take a look at that this week.

siemvaessen · 2018-06-20T11:09:14Z

Bumping this up. @Bjwebb any updates on this? We'd be happy to assist, seeing we need it :-) so let me know, also happy to setup a brief call for specs so we can continue.

andylolz · 2018-06-20T11:20:11Z

I started making progress on this (with OpenDataServices/flatten-tool#201), but it transpired that to do it properly was a bigger job than I’d originally anticipated.

It turns out it’s sort of helpful to get roundtripping working correctly first, and then to remove hard-coded assumptions that we’re dealing with an activity file.

~~Next up on my list is: OpenDataServices/flatten-tool#176~~

siemvaessen · 2018-06-21T07:53:32Z

@andylolz - how can we contribute. Got a plan of action so we can distribute tasks?

So PR Nr. 201 has been accepted, so it's just #180 ?

andylolz · 2018-06-24T22:41:41Z

@siemvaessen really good point, and thanks for pushing this ticket forward.

I know back in April I had a plan for this, and I regret not detailing it in a ticket (in fact I can see I have a local branch and a bunch of unstaged work on this, too.)

I’ll try to retrace my steps, and will break it down here ~~next~~ this week.

Bjwebb · 2018-06-25T09:48:25Z

I've not had chance to look into what's still needed for organisation files in CoVE. @andylolz if you do write up a plan, that would be helpful. I'm happy to check it.

andylolz · 2018-06-26T12:20:11Z

if you do write up a plan, that would be helpful.

Sure, why not.

I'm happy to check it.

Thanks!

Bjwebb · 2018-08-13T12:00:32Z

I will be doing some work on IATI Organisation file support this week.

andylolz · 2018-08-13T12:05:07Z

I will be doing some work on IATI Organisation file support this week.

Haha, kk. So as mentioned in #993 (comment), I think this was blocked by OpenDataServices/flatten-tool#196, which is now sorted. But by all means.

#959 (this is instead of specifying root_list_path which takes precedence) This approach is intead of changing order of precedence in flatten-tool OpenDataServices/flatten-tool#214

Bjwebb · 2018-08-15T12:13:29Z

Here's a dev deploy of the work so far: http://959-org-file.dev.cove.opendataservices.coop/
Here's a spreadsheet file to test with it: https://github.com/OpenDataServices/cove/raw/959-org-file/cove_iati/fixtures/basic_iati_org_valid.xlsx

What works:

Spreadsheet to valid IATI organisation XML (including correct sorting) for simple data
CoVE validates against the IATI organisation schema
Here's a list of all possible headings https://gist.github.com/Bjwebb/e32f5139e62ce78ba923cdfcbfdf6cbd generated by this code Add limited XML create-template support flatten-tool#215

Still TODO:

One to many relationships, because we need to use organisation-identifier as the key instead of iati-identifier
CoVE doesn't tell you that you've uploaded an organisation file, or that it is validating against the organisation schema
More work on tests
Multilingual template generation
Add an issue for IATI organisation XML -> spreadsheet (includes meta tab roundtripping)

Additionally IATI organisation XML -> spreadsheet doesn't work, you just get a blank spreadsheet. I am unlikely to get time to fix this.

andylolz · 2018-08-15T16:39:51Z

COOL!

Additionally IATI organisation XML -> spreadsheet doesn't work, you just get a blank spreadsheet. I am unlikely to get time to fix this.

^^ Right yeah, I guess this bit is about meta tab roundtripping.

Requires an as yet unmerged flatten-tool commit OpenDataServices/flatten-tool#219

State that the file is an organisation file, and has been validated against the organisation schema. Disable the rulesets section, as this doesn't support organisation files properly atm.

Bjwebb · 2018-08-16T15:37:11Z

Okay, so I think we're done in terms of CoVE feature development for this PR. Still some to do work on tests and template generation.

Many thanks to @andylolz for having done a lot of the work on this!

Dev deploy: http://959-org-file.dev.cove.opendataservices.coop/
Here's a spreadsheet file to test with it, that's been updated to have multiple tabs with one to many relationships: https://github.com/OpenDataServices/cove/raw/959-org-file/cove_iati/fixtures/basic_iati_org_valid.xlsx

#959 (this is instead of specifying root_list_path which takes precedence) This approach is intead of changing order of precedence in flatten-tool OpenDataServices/flatten-tool#214

Requires an as yet unmerged flatten-tool commit OpenDataServices/flatten-tool#219

State that the file is an organisation file, and has been validated against the organisation schema. Disable the rulesets section, as this doesn't support organisation files properly atm.

Lathrisk · 2019-09-10T11:13:40Z

#959 (comment)

Organisation XML -> Spreadsheet is now implemented (pending deployment), but round trip is still not possible as the spreadsheet is still missing the Meta-tab.

Multilingual support is still missing also.

See: #1219

stevieflow · 2019-12-13T20:11:04Z

What's blocking us getting this to deployed @Bjwebb @Lathrisk ?

Lathrisk · 2019-12-16T11:30:18Z

@stevieflow I'm just updating the PR and refreshing my memory just now, but it will need re-reviewed (@Bjwebb had previously requested changes). I'm not sure if the meta-tab work that Ben has been doing will tie in with this as well? I will mention it at stand-up tomorrow morning.

Bjwebb added the IATI label Feb 23, 2018

andylolz mentioned this issue Mar 20, 2018

Validate against IATI org schema #993

Merged

siemvaessen closed this as completed Mar 20, 2018

Bjwebb reopened this Apr 18, 2018

Bjwebb self-assigned this Aug 13, 2018

Bjwebb added a commit that referenced this issue Aug 15, 2018

[#959] Also pass IATI organisation schema to flatten-tool for sorting

ab80159

Bjwebb added a commit to OpenDataServices/flatten-tool that referenced this issue Aug 16, 2018

[OpenDataServices/cove#959] Add a config var for id_name

fbb362d

Bjwebb mentioned this issue Aug 16, 2018

Add a config var for id_name OpenDataServices/flatten-tool#219

Merged

Bjwebb added a commit to OpenDataServices/flatten-tool that referenced this issue Aug 16, 2018

[OpenDataServices/cove#959] Add a config var for id_name

347393d

Bjwebb added a commit that referenced this issue Aug 16, 2018

[#959] Put IDName in default_configuration to allow spreadsheet override

97c77de

Requires an as yet unmerged flatten-tool commit OpenDataServices/flatten-tool#219

Bjwebb added a commit that referenced this issue Aug 16, 2018

[#959] Put IDName in default_configuration to allow spreadsheet override

0a12e59

Requires an as yet unmerged flatten-tool commit OpenDataServices/flatten-tool#219

Bjwebb added a commit to OpenDataServices/flatten-tool that referenced this issue Aug 21, 2018

[OpenDataServices/cove#959] Add test of IDName configuration command

4fe532d

Bjwebb added a commit that referenced this issue Aug 21, 2018

[#959] Split IATI Organisation file validation into its own test

009c82d

andylolz mentioned this issue Oct 18, 2018

Merge question #1055

Closed

rhiaro pushed a commit that referenced this issue Oct 28, 2018

[#959] Also pass IATI organisation schema to flatten-tool for sorting

5afd629

rhiaro pushed a commit that referenced this issue Oct 28, 2018

[#959] Put IDName in default_configuration to allow spreadsheet override

543de56

Requires an as yet unmerged flatten-tool commit OpenDataServices/flatten-tool#219

rhiaro pushed a commit that referenced this issue Oct 28, 2018

[#959] Split IATI Organisation file validation into its own test

40cc3dc

rhiaro pushed a commit that referenced this issue Oct 28, 2018

[#959] Test that IATI org xlsx file converts

802968d

stevieflow mentioned this issue Aug 17, 2019

[IATI] Org file XML > spreadsheet output #1208

Open

stevieflow assigned Lathrisk Dec 13, 2019

stevieflow added the roundtripping Working between formats label Oct 21, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

IATI organisation file support #959

IATI organisation file support #959

siemvaessen commented Jan 31, 2018

Bjwebb commented Feb 1, 2018

siemvaessen commented Feb 2, 2018

Bjwebb commented Feb 7, 2018

siemvaessen commented Mar 12, 2018

Bjwebb commented Mar 19, 2018

siemvaessen commented Mar 20, 2018

andylolz commented Apr 18, 2018 •

edited

Loading

andylolz commented Apr 19, 2018

Bjwebb commented Apr 19, 2018

siemvaessen commented Jun 20, 2018

andylolz commented Jun 20, 2018 •

edited

Loading

siemvaessen commented Jun 21, 2018

andylolz commented Jun 24, 2018 •

edited

Loading

Bjwebb commented Jun 25, 2018

andylolz commented Jun 26, 2018

Bjwebb commented Aug 13, 2018

andylolz commented Aug 13, 2018

Bjwebb commented Aug 15, 2018 •

edited

Loading

andylolz commented Aug 15, 2018

Bjwebb commented Aug 16, 2018

Lathrisk commented Sep 10, 2019

stevieflow commented Dec 13, 2019

Lathrisk commented Dec 16, 2019

IATI organisation file support #959

IATI organisation file support #959

Comments

siemvaessen commented Jan 31, 2018

Bjwebb commented Feb 1, 2018

siemvaessen commented Feb 2, 2018

Bjwebb commented Feb 7, 2018

siemvaessen commented Mar 12, 2018

Bjwebb commented Mar 19, 2018

siemvaessen commented Mar 20, 2018

andylolz commented Apr 18, 2018 • edited Loading

andylolz commented Apr 19, 2018

Bjwebb commented Apr 19, 2018

siemvaessen commented Jun 20, 2018

andylolz commented Jun 20, 2018 • edited Loading

siemvaessen commented Jun 21, 2018

andylolz commented Jun 24, 2018 • edited Loading

Bjwebb commented Jun 25, 2018

andylolz commented Jun 26, 2018

Bjwebb commented Aug 13, 2018

andylolz commented Aug 13, 2018

Bjwebb commented Aug 15, 2018 • edited Loading

andylolz commented Aug 15, 2018

Bjwebb commented Aug 16, 2018

Lathrisk commented Sep 10, 2019

stevieflow commented Dec 13, 2019

Lathrisk commented Dec 16, 2019

andylolz commented Apr 18, 2018 •

edited

Loading

andylolz commented Jun 20, 2018 •

edited

Loading

andylolz commented Jun 24, 2018 •

edited

Loading

Bjwebb commented Aug 15, 2018 •

edited

Loading