Are validation of network services to strict ?

[MetaGIS]

We se in validation results that we get many errors related to exception-handling.
These cases are not that the servers return wrong results in case when a proper request is done.
It's the case that when client send wrong or too few arguments.
Then according to OGC the service shall behave well and generate a proper error message.
In general I agree to this rule. But it seems now that some implementation of these servers are done to be more "user-friendly"
Eg if you add service= when calling getCapabilities the servers are automatically adding eg service=WFS instead of returning an error.

We get quite large number of these messages from standard platforms like ArcGIS Server and Geoserver.
So these servers are maybe not by default fulfilling all OGC requirements.

I got some second hand info that Geoserver has not runned through a complete OGC-test for many years.
If so there is a major problem.

So what is the reason for our observation ?
Is the case that the most used platforms do not support complete OGC-requirements
OR
Are the servers themselves wrongly deployed ?

If the first observation is true I think the Reference validator should not have such strict rules.

When developing/selecting the rules for reference-validator for Network services what servers have this been tested on?
In Sweden Geoserver, MapServer, ArcGIS Server and Carmenta covers 100 % of all servers used.
All these have these issues

Kind regards
Michael Ostling, Lantmäteriet (SE)

[iuriemaxim]

@MetaGIS We faced similar problems when implementing INSPIRE. As far as I know, currently there is no out of the box solution, simple to configure, to pass the INSPIRE (and OGC) tests. However it is possible with certain effort to adapt the existing solutions to be conform to INSPIRE TGs and OGC standards and the pass the INSPIRE validations.
Here https://www.ogc.org/resource/products/stats you may find the list of products that are passing the OGC tests.
If you need some links to test a Geoserver implementation that is passing the tests, let me know.
The tests are not too strict and if they will be made less stricter, then interoperability cant be ensured. Standards are written having in mind interoperability and harmonization. Less rules or less stricter rules means no standard. Producers should make their products according to the standards, as the standards should not be changed in order to be inline with some software. This is because first are produced the standards and then producers are implementing them. It is true that there exist also OGC tests that have some issues, and in certain specific circumstances errors are triggered even if no errors exists. But these are few exceptions (one or two bugs). It is true also that standards can be ambiguous and producers would interpret them differently. But that's why standards have versions.

[dahermansson]

OGC lists Carmenta Server 4.6 https://www.ogc.org/resource/products/details/?pid=1555 but our WMS setup with Carmenta Server 4.6 fails in the Inspire Validator on at05 GetCapabilities request parameters witch is specified under iso-19128 (same document as WMS 1.3) OpenGIS Web Map Service (WMS) Implementation Specification.
Section 7.2.2 GetCapabilities request overview specifies the mandatory and optional parameters in a GetCapabilities-Request like this:

Request parameter	Mandatory/optional
VERSION	O
SERVICE	M
REQUES	M
FORMAT	O
UPDATESEQUENCE	O

I can't find in the OGC specification that an exception is requierd in case of a missing mandatory parameter and OGC don't seams to have that in there test suit. https://cite.opengeospatial.org/te2/about/wms13/1.3.0/site/wms-1_3_0-ats.html. If they hade, Carmenta who is not responding with an exception in case the GetCapabilities-request is missing Service=WMS would not be listed as compliant.

Why is Inspire validating this cases when OGC them self is not doing it?

[iuriemaxim]

@dahermansson

For testing a WMS according to INSPIRE TG for View services (network service) the user is asked to provide the URL to the GetCapabailities document. However this does not mean that the validator is checking only the GetCapabilities request according to section 7.2.2. at page 24.

I think that the errors triggered by the validator that you are referring to are related to GetMap request.

In section 7.3.2. at page 33 in the WMS 1.3.0 OGC standard at https://www.ogc.org/standards/wms are the GetMap requests optional and mandatory parameters.

But indeed the OGC Test suite for WMS is not used by INSPIRE validator because in order to test against the OGC Test suite a specific dataset from OGC need to be used while performing the test. See https://cite.opengeospatial.org/teamengine/about/wms13/1.3.0/site/

Therefore the INSPIRE view services cant be tested against the OGC WMS ATS.

You are right that the INSPIRE validator is stricter than the OCG WMS validator.
As an example correct implementation of the request for transparent parameter is checked by INSPIRE validator while this is not checked by OGC validator.
At https://cite.opengeospatial.org/teamengine/about/wms-client/1.3.0/site/ is illustrated what it is not tested by OGC:

This does not mean that these should not be tested in INSPIRE, as OGC could implement these tests in any further release of their validator.

But indeed even if a WMS is compliant according to OGC, it does not mean that it will be compliant according to INSPIRE TG for view services, tested by the INSPIRE validator.

To your question: Why is Inspire validating this cases when OGC them self is not doing it?:

• OGC could at any time make a new version of the OCG test in order to test what they are not currently testing even if the standard has a requirement and
• validator is checking the view services against the TG for view services, not (only) against the OGC WMS standard
• OGC test is using a specific sample dataset, so INSPIRE view services cant be tested with the OCG test suite.

For investigating, you may indicate the URL to the WMS GetCapabilities service and the exact error that you think is not correctly triggered.

[dahermansson]

This is the test that is failing.

It's quite clear that i does at GetCapabilities-request to the WMS-service without the parameter Service=WMS.
Carmenta Server is using different endpoints for WMS and WFS so the service-parameter is not mandatory do decide what Capabilities should be returned.
The Inspire Validator is complaining that it gets at complet GetCapabilities-respons instead off at OWS exception.

My opinion is that that is quiet usles to validate on that specific thing for Inspire when OGC don't do it.

If you validate against our WMS at tho moment you will get more errors than this one but those are thing we can handle and update in near time.

[MetaGIS]

We need to raise this issue again.
I cant see any official response from JRC on this.

It seems we are not alone in Sweden having these issues.
Also from Norway, Denmark and Finland we get reports on service validations not going through completely.
The Inspire Geoportal does accept these services but the validator still reports many errors.

We think the case is that some OGC server providers like Geoserver, ESRI-ArcGIS Server and Carmenta server by default do not pass these rules related to exception handling. But they have indeed passed OGC tests.
Since the first two are among the largest providers we have (90 % of Swedish Inspire agencies are running these servers) it does not make sense validating things they do not support!
Or does it exist special plugins for these servers that also handle exceptions.

We think its important that
Either
A) JRC and the comission argument why these rules are important and how come the major OGC-platforms do not support it. Of course working according to standards is important but it seems Inspire is using a different level of acceptance.
We think this would be an important step towards simplification of Inspire. It is very hard to get acceptance for Inspire services if there are requirements that are not fullfilled in the base software.

Or
B) these rules should be removed from the validator

[matsoo]

The problem with fixing the service parameter issue in the server products is that not only does the OGC test not test the service parameters as mandatory, it is actually impossible to pass the OGC compliance tests for any WMS version if you throw an exception when the service parameter is missing. If a missing service parameter causes an exception a very large number of tests will fail in these test suites.

It is of course possible to add a "strict mode" to allow a server to pass this test in the INSPIRE Validator, but in doing so you break the OGC tests, and also compatibility with a number of clients that does not provide the service parameter, since this has never been enforced by either the OGC or any major vendor.

You should probably either remove this test or raise an issue with the OGC so the test suites at least can be compatible with each other.