Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more container-based 3rd party application tests #172

Open
20 of 34 tasks
smlambert opened this issue Nov 15, 2017 · 8 comments
Open
20 of 34 tasks

Add more container-based 3rd party application tests #172

smlambert opened this issue Nov 15, 2017 · 8 comments
Labels
enhancement type:external 3rd party application tests (invoked by the make target 'external')

Comments

@smlambert
Copy link
Contributor

smlambert commented Nov 15, 2017

As an example of this type of testing, we have added scala tests: https://github.com/AdoptOpenJDK/openjdk-tests/tree/master/thirdparty_containers/scala

We have derby and solr_lucene tests running in the same "dockerized" way, so we should also add those test sub-folders.

Once we have those 3 examples of 3rd party application tests added, we should consider the list of other applications we would like to add and prioritize them (eclipse, cassandra, etc). It will also be good to engage with these communities so that they are aware that we will be running their community test suites against the different flavours of JVM at AdoptOpenJDK (openjdk8, openjdk9, openjdk+openj9...).

Here is a list of potential ones to add, not in any priority order:

Also, this Testing Java link and this Quality Outreach link relates to this effort (engaging with Java projects, unblocking them if they have issues)...

Consider new tests via external_custom test target.
Consider startup tests for quick smoke test of each application.

@suman-mitra
Copy link
Contributor

Add Elasticsearch under thirdparty container tests
#360

@tellison
Copy link
Contributor

In additional to a broad set of application tests to prove the quality of the binaries, it would be useful to have a suite of MicroProfile tests to demonstrate that our binaries are a good foundation for MicroProfile.

Microprofile doesn't have a reference implementation, rather there are a number of popular implementations that cover different proportions of the specification.

The following projects will provide a good test environment for our binaries:

The tests would be taken from the MicroProfile TCKs, e.g. fault tolerance APIs, metrics APIs, etc. Note that not all the implementations implement every part of the MicroProfile spec, so we'd need to define the appropriate set of TCKs to run against each implementation.

As the MicroProfile projects are open communities, I expect we can get help in putting together the tests and running them on our test farm.

@kwsutter
Copy link

kwsutter commented May 2, 2018

This is a little bit longer term, but you should also consider the use of the Jakarta EE test suite (CTS) when that becomes available under Eclipse licensing. I know from personal experience that running the CTS can be complicated and time-consuming. So, maybe we can work with the various vendors and developers that have or will be running the Jakarta EE CTS to also run with AdoptOpenJDK binaries. Not as a requirement, but as a "friend of open source". We're still a few months away before these test suites will be available and usable, but something to keep in your back pocket for future testing efforts.

@smlambert
Copy link
Contributor Author

Thanks @tellison and @kwsutter - we are in process of updating our test doc to state our goals around running these external tests and engaging with the open-source communities (Summarized mission: us running their tests, them using adopt SDKs, attempting to make both aspects of testing drop-dead easy).

Both your suggestions fit in with the overall mission. 👍 Link to updated mission statement to come shortly.

@smlambert
Copy link
Contributor Author

related: adoptium/adoptium-support#30

@joeyleeeeeee97
Copy link
Contributor

Does mybatis worth adding if we are considering hibernate?

benny-coy added a commit to benny-coy/openjdk-tests that referenced this issue Apr 15, 2021
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#      https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
-->
# External Payara-mp-tck Tests
Payara Platform is cloud-native, optimized for production environments.
payara-mp-tck test(payara MicroProfile-tck) tests are part of the third-party application tests that helps verify that the Adoptium binaries are good by running a variety of Java applications inside of Docker containers. AdoptOpenJDK/openjdk-tests/Issue adoptium#172 lists the applications that we have initially targeted to best exercise the Adoptium binaries. For each application, we choose to run a selection of their functional tests.payara-mp-tck tests are functional tests pulled from the [payara/MicroProfile-TCK-Runners](https://github.com/payara/MicroProfile-TCK-Runners.git) repository.

## Running  External Payara-mp-tck tests locally
To run any AQA tests locally, you follow the same pattern:

0. Ensure your test machine is set up with [test prereqs](https://github.com/AdoptOpenJDK/openjdk-tests/blob/master/doc/Prerequisites.md).  For external tests, you do need Docker installed.

1. Download/unpack the SDK that you want to test to your test machine.
2. `export TEST_JDK_HOME=</pathToWhereYouInstalledSDK>`
3. `git clone https://github.com/AdoptOpenJDK/openjdk-tests.git`
4. `cd openjdk-tests`
5. `./get.sh`
6. `cd TKG`
7. Export required environment variables, BUILD_LIST and EXTRA_DOCKER_ARGS (example: `export BUILD_LIST=external/payara-mp-tck` and `export EXTRA_DOCKER_ARGS="-v $TEST_JDK_HOME:/opt/java/openjdk"`
8. `make compile`              (This fetches test material and compiles it, based on build.xml files in the test directories).
9. `make _payara-mp-tck_test`   (When you defined BUILD_LIST to point to a directory in [openjdk-tests/external](https://github.com/AdoptOpenJDK/openjdk-tests/tree/master/external), then this is a testCaseName from the playlist.xml file within the directory you chose).


When [running these from the command-line](https://github.com/AdoptOpenJDK/openjdk-tests/blob/master/doc/userGuide.md#local-testing-via-make-targets-on-the-commandline), these tests are grouped under a make target called 'external', so 'make external' would run the entire set of tests found in the openjdk-tests/external directory.  This is unadvisable!  Limit what you compile and run, BUILD_LIST=external/`<someSubDirectory>`, and TARGET=`<testCaseNameFromSubdirPlaylist>`

These tests run regularly and results can be found in [TRSS Third Party Application view](https://trss.adoptopenjdk.net/ThirdPartyAppView).

Signed-off-by: Benny-coy <[email protected]>
@llxia
Copy link
Contributor

llxia commented Sep 16, 2021

We may also consider adding jython test suite: https://github.com/jython/jython/tree/master/tests
Related openj9 issue: eclipse-openj9/openj9#12238

@joeyleeeeeee97
Copy link
Contributor

Maybe consider adding rocketmq
https://github.com/apache/rocketmq

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement type:external 3rd party application tests (invoked by the make target 'external')
Projects
Status: Todo
Development

No branches or pull requests

6 participants