Installation of reference validator on own computer using Tomcat

[heidivanparys]

How can the reference validator installed locally (as in, on a Windows desktop computer), if using Docker is not an option, and if anything that needs administrator rights should be avoided? According to the deployment options in the training material, not using Docker is still a valid (and supported?) option. The purpose is mainly to validate metadata quickly and locally.

What I have done so far:

1. Installed Java 8 (version specified on https://docs.etf-validator.net/v2.0/Admin_manuals/index.html#_prerequisites_on_windows ; I used AdoptJDK instead of Oracle, that works fine)
2. Installed Tomcat 8.5.59 (as a stand-alone application, not as a service; version specified on https://docs.etf-validator.net/v2.0/Admin_manuals/index.html#_prerequisites_on_windows).
3. Copied the validator.war that is inside https://github.com/inspire-eu-validation/community/releases/download/v2020.3/inspire-validator-2020.3.zip to the webapps folder in the Tomcat installation.
4. Created the ETF data directory, including all subfolders, see also https://docs.etf-validator.net/v2.0/Admin_manuals/index.html#_etf_data_directory_structure
5. Put etf-config.properties in the config folder of the ETF data directory.
   a. Set: etf.webapp.base.url = http://localhost:8080/validator
   a. Set (uncomment and modify): etf.testobject.max.size = 104857600 (100MB instead of 5 GB, I just need to test metadata and my computer has limited disk space)
6. Put folder ets-repository-2020.3 and its contents in the projects folder of the ETF data directory (retrieved from https://github.com/inspire-eu-validation/community/releases/download/v2020.3/ets-repository-2020.3.zip).
7. Made sure that I have at least 5 GB free on my computer, otherwise I won't be able to run any tests ("The system has currently insufficient resources to process this request.".
8. Set environment variables JAVA_HOME, JAVA_OPTS (-Dfile.encoding=UTF8 -Djavax.servlet.request.encoding=UTF-8), CATALINA_HOME and ETF_WEBAPP_PROPERTIES_FILE.
9. Started Tomcat (call bin\startup.bat).
10. Checked the status of the validator using curl -X GET "http://localhost:8080/validator/v2/status" (when status is MAJOR no tests can be started, occurs when less than 5GB is free on the computer)

So far, so good, I can start a test checking a metadata record. However, then I run into

System error in the Executable Test Suite. Please contact a system administrator. Error information:
Code list http://inspire.ec.europa.eu/theme cannot be accessed.

Which is related to #311, #319, #401 and perhaps more issues.

As far as I can see, a "(reverse?) proxy" has to be set up, to deal with requests to inspire.ec.europa.eu

So now my questions:

1. Is it possible to do that in Apache Tomcat itself?
2. If not, what would be the absolute simplest way to do that on a local computer, avoiding having to install anything with administrator rights?

[heidivanparys]

If anybody has a workaround, that would be fine too. Would it help, to update the test for the metadata locally? Changing e.g. http://inspire.ec.europa.eu to https://inspire.ec.europa.eu ?

[carlospzurita]

Dear @heidivanparys

Thank you for opening this issue and sharing your feedback.

As it has been commented in previous issues, the redirection of INSPIRE codelists from HTTP to HTTPS is not supported in the WAR file directly, as it contains Java source code that should not handle this situations to keep the tool as generic as possible.

To help the community with this, we included a redirection mechanism in the Docker image, that would setup a proxy server along with the validator, to intercept any request to HTTP INSPIRE domain and send it to the HTTPS endopoint.

Anyone that wants to deploy the validator in Tomcat (or any other servlet like Jetty or Wildfly), needs to establish the redirection system in their infrastructure. This can be achieved using virtual domains in Apache server responding to http://inspire.ec.europa.eu/ and redirecting to https://inspire.ec.europa.eu/, as it is done in the Docker image; but is up the system administrators to decide what is the best option. You can check the contents in the latest release to see how we achieved this inside the docker-entrypoint.sh file.

We are going to try to come up with a comprehensive guide to establish a proxy mechanism and solve the redirection issue when deploying the WAR file directly.

Of course, a workaround can be to modify the references in the metadata from HTTP to HTTPS, and avoid the 302 redirection code when requesting the codelists.

[ghost]

Dear @carlospzurita ,

I have one question concering this one:

Of course, a workaround can be to modify the references in the metadata from HTTP to HTTPS, and avoid the 302 redirection code when requesting the codelists.

Do you mean in the metadata or/and ETS for metadata?

In my opinion if one modifies the references in the metadata 302 redirection won't be automatically avoided in all cases and it is also not conformant to TG 2.0 in some cases, because this is only correct for the other INSPIRE resource (https://inspire.ec.europa.eu/) and not for INSPIRE registry resources (https://inspire.ec.europa.eu/registry). This a bit complicated because many INSPIRE Registry ressources have URL like this one: http://inspire.ec.europa.eu/theme.

metadata ETS (I assume in this case HTTPS redirect will also occur)

• see https://github.com/inspire-eu-validation/ets-repository/blob/680a0506e479839d9fb2b109383d00249dce81aa/metadata/2.0/datasets-and-series/ets-md-monitoring-bsxets.xml#L79

And I am also not sure if your statement is according to this statement:

• see https://github.com/inspire-eu-validation/community/issues/319#issuecomment-635220388.

Could somebody clarify this?

Thanks a lot!

[carlospzurita]

Dear @DeordD

You are right that my explanation can result confusing. The workaround I was referring was to modify the references on the loaded ETS and any URI that needs to be requested to the registry from HTTP to HTTPS present on the metadata record.

This is of course, and I want to stress it, a temporal workaround and it doesn't constitute in any way an official validation. The only way to get an official validation is using the released ETS as they appear in the production instance.

[ghost]

Dear @carlospzurita ,

thanks a lot for the clarification, nice support and efforts concerning INSPIRE validator!

[fabiovinci]

Dear @heidivanparys,

were you able to apply the workaround suggested by carlospzurita?

[heidivanparys]

Dear @heidivanparys,

were you able to apply the workaround suggested by carlospzurita?

I don't remember all the details, but we managed to get a local installation up and running.

[fabiovinci]

Thank you for your feedback.
I will close this issue by converting it into a discussion and leaving it open to people who might have the same problems.