Skip to content
This repository has been archived by the owner on Nov 1, 2018. It is now read-only.

Issue12 #3

Open
wants to merge 91 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
91 commits
Select commit Hold shift + click to select a range
27d7d8d
#1 refactored to common jargon parent pom
Jan 25, 2017
436796c
#1 refactor
Jan 26, 2017
3b8f132
#1 cleanup
Jan 26, 2017
5bf06a3
#1 big refactor
Jan 26, 2017
ea91ce3
#1 add cruft
Jan 26, 2017
a637a3c
#2 hold open space for abstract api for pid and event
Jan 26, 2017
ed809f8
#1 add pid and event code
Jan 30, 2017
4d3665f
#2 more refactoring
Feb 7, 2017
f714835
#2 factor out handle stuff
Feb 7, 2017
777d8e9
#2 cleaned up refs to handle and es, commented out with fixme
Feb 7, 2017
bc30798
#2 more cleanup
Feb 8, 2017
9fd4f90
#2 refactoring of utils
Feb 8, 2017
5e5c17d
#2 merge
Feb 8, 2017
01c6bc8
#2 adding spring boot cruft
Feb 8, 2017
9c5e995
#2 spring boot will now start
Feb 8, 2017
0437ad4
#2 spring boot will now start
Feb 8, 2017
2cd56be
#2 cleanup though handle stuff wont build yet
Feb 17, 2017
1a7ff99
#2 ran some cleanups and updated codebase for newer jargon
Feb 17, 2017
1a010d8
#6 factor out event service
Feb 20, 2017
7f2591a
#2 factor out publication context
Feb 21, 2017
95daabe
#2 adding reflection based plugin loader infrastructure adapted from …
Feb 21, 2017
fa45027
#2 adding abstract pid and event service with configs
Feb 24, 2017
ae14af6
#2 working on first test for plugin event service
Feb 24, 2017
9a66893
#1 some cleanups and merge of dev
Feb 28, 2017
a0b3e84
#1 looking for missing abstractdataoneplugin
Feb 28, 2017
979873e
#6 introduce abstract plugin
Feb 28, 2017
e264f22
#6 adding config to test boot and load of event dummy impl
Feb 28, 2017
7ab0623
#3 clean up log4j cyclic dependency
Mar 1, 2017
4e97da1
#3 discovery works
Mar 1, 2017
2c9b320
#3 added test of plugin event
Mar 1, 2017
3b44bfd
#3 discovery of pid plugin
Mar 10, 2017
e7663f5
#3 add repo service api
Mar 10, 2017
5d2012a
#1 align version and clean build
Mar 20, 2017
b632325
#1 clean and merge
Mar 20, 2017
4d9110c
#1 cleanup after merge
Mar 20, 2017
e62fc1c
#1 adding repo service and continue to refactor, currently breaks build
Mar 20, 2017
c72ad02
#2 merge
michael-conway May 16, 2017
0b604d7
#2 wip
michael-conway May 18, 2017
57cdde4
#2 reworking plugin factory, currently breaks
michael-conway May 18, 2017
591ce58
#2 adding factories and aligning api for lookups externally
michael-conway May 18, 2017
ffe9b32
#3 add dummy factories
michael-conway May 18, 2017
027aa40
#3 factory methods for create
michael-conway May 18, 2017
4dcafff
#3 successful factory load tests for all plugins
michael-conway May 18, 2017
746ae69
#3 temp break build, substituting plugin calls in impl services
michael-conway May 19, 2017
f53744a
#3 refactor drafted complete
michael-conway May 22, 2017
655ffea
#3 migrate props, add addl logging
michael-conway May 22, 2017
59d9853
#3 adding docker
michael-conway May 22, 2017
fc64294
#3 loads and pings!
michael-conway May 22, 2017
9e4c587
#3 add external lib
michael-conway May 24, 2017
a97f206
#3 cleanup lib
michael-conway May 24, 2017
4efb773
#8 mirror props in PublicationContext
michael-conway May 25, 2017
9685e9b
#7 refactor metadata things out of repo
michael-conway May 25, 2017
6c38191
#9 add default event service
michael-conway Jun 2, 2017
0444aa5
#9 break out event service
michael-conway Jun 2, 2017
2ca9854
#9 init table schema
michael-conway Jun 2, 2017
89fd826
#9 adding dao
michael-conway Jun 2, 2017
22b1c94
#9 adding dao
michael-conway Jun 2, 2017
131878e
#9 basics of indexing service
michael-conway Jun 2, 2017
1a0ddc2
#9 connects in indexer to database
michael-conway Jun 5, 2017
d0258b6
#9 building up to connnect to queue
michael-conway Jun 6, 2017
e46d202
#9 working on transform
michael-conway Jun 6, 2017
d444bbf
#9 json convert
michael-conway Jun 7, 2017
48c5c24
added a missing dependency in default-event-service-persist
Jun 7, 2017
c81215b
#9 working on persistence
michael-conway Jun 7, 2017
55bf167
Merge branch 'master' of https://github.com/DFC-Incubator/dfc-dataone
michael-conway Jun 7, 2017
ea10f2d
#9 adding persistence
michael-conway Jun 7, 2017
9ea4aa1
#9 at least it starts
michael-conway Jun 7, 2017
377e4d0
#9 working persist
michael-conway Jun 7, 2017
8253318
#9 session factory?
michael-conway Jun 7, 2017
9830dca
#9 working on hibernate
michael-conway Jun 7, 2017
839e02c
#9 working on hibernate
michael-conway Jun 7, 2017
c7cfef9
#9 fix spring wiring
michael-conway Jun 7, 2017
225134b
#9 col names
michael-conway Jun 7, 2017
b493f1a
make the Hibernate dialect configurable
Jun 7, 2017
6c8b4eb
allow enumerated types to work with a Postgres enumeration
Jun 7, 2017
76de9be
converted the access log entry ID to a UUID and provided an implement…
Jun 8, 2017
7347acd
#9 externalize properties
michael-conway Jun 8, 2017
485afb8
#9 bean wiring for event servcie, probably wrong and duplicative
michael-conway Jun 8, 2017
26086b5
#9 fix hibernate pom stuff, boot of default servie
michael-conway Jun 9, 2017
1349e98
modified the indexer code to accept timestamps in AMQP messages
Jun 8, 2017
5beda19
use the EventsEnum class rather than the custom EventType enumeration…
Jun 9, 2017
7a619cc
added the query for the /logs endpoint
Jun 9, 2017
8c22f82
#9 fix for missing jta
michael-conway Jun 9, 2017
ab7e25f
added the event service factory for the default event service API imp…
Jun 9, 2017
9fde4bc
changed the "accessor" field in the data object open AMQP message to …
Jun 13, 2017
a15e5b4
#10 factory now boots up and creates instance
michael-conway Jun 20, 2017
ed91270
#10 updated docker to add extra libs, factories now discovered in doc…
michael-conway Jun 20, 2017
d948945
#10 fix d1client.properties to load from /etc
michael-conway Jun 20, 2017
6a5c537
#10 testing classloading
michael-conway Jun 21, 2017
24573e9
#12 fix deps issue temporarily
michael-conway Jun 21, 2017
8a60c8e
#12 fix repo service load
michael-conway Jun 21, 2017
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 26 additions & 35 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,38 +1,29 @@
/target
*.pydevproject
**/.classpath
**/.DS_Store
**/nb-configuration.xml
**/.springBeans
**/testing.properties
*.iml
.project
.idea
.metadata
.springBeans
bin/**
tmp/**
tmp/**/*
*.tmp
*.bak
*.swp
*~.nib
local.properties
.classpath
.settings/
.loadpath
.DS_Store
# External tool builders
.externalToolBuilders/

# Locally stored "Eclipse launch configurations"
*.launch

# CDT-specific
.cproject

# PDT-specific
.buildpath

#Version file
src/**/RestVersion.java

#removed files
test-certs/public.pem
test-certs/private.pem
test-certs/cakey.pem
test-certs/cacert.pem
src/main/resources/admpriv.bin
Servers
.settings
.classpath
target
mvn-repo
bin
*.war
*.log
*/nbactions.xml
*.eml
nbactions.xml
testing.properties
*/test-output
JargonVersion.java
**/${test.option.mount.basedir}
*.*~
**/.project
**/.settings
**/target
28 changes: 28 additions & 0 deletions LICENSE.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
iRODS license terms and copyright info from the irods site at: https://www.irods.org/index.php/License

License
iRODS Copyright and Licensing

iRODS is open source software released under a BSD License, see license text in "iRODS License Terms and Conditions" below.
The BSD license has been described in very general terms as allowing you to do whatever you want to with the software and
source code as long as you acknowledge who wrote it and that, as with any open source software, there is no warranty and you're using the code "as is."
In the spirit of collaborative open source software, the iRODS community encourages you to communicate with us, letting us know what features you like,
features that would be useful, problems, bugs, suggestions, etc., and to perhaps contribute source code.
The iRODS community has formed the Data Intensive Cyberinfrastructure Foundation, a 501(c)(3) nonprofit corporation established to serve
as the home of the iRODS open source community over the long term. If you choose to contribute new code, you'll receive full acknowledgment. All you do is complete the Contributor's Agreement, under which you retain copyright ownership
in your code but give a free license to the iRODS nonprofit foundation, allowing your code to be integrated into iRODS and in turn released under the BSD license.
Note: The above text is an educational overview of iRODS open source licensing, and not intended as legal advice nor is it part of the iRODS license agreement, which is below. As always, for legal advice consult an attorney.

iRODS License Terms and Conditions Notice

Copyright (c) 2005-2011, Regents of the University of California, the University of North Carolina, and the Data Intensive Cyberinfrastructure Foundation
All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the University of California, San Diego (UCSD) nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT
NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
29 changes: 17 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,23 @@
# dfc-dataone
DataONE Member Node Implementation for iRODS
============================================

Supports DataONE Member Node Tier 1 Implementaion v2.0
# Project: DataONE Member Node Service
#### Date:
#### Release Version:
#### git tag:
#### Developer: Lisa Stillwell - RENCI,Mike Conway - DICE

Uses: Jargon, Spring, RestEasy
DataONE Member Node software and supporting libraries

Projects using this code for an iRODS based D1 member node implementation must customize:

1. resources/d1_client.properties
1. implementation of events/EventLogAO interface
1. implementation of id/UniqueIdAO interface
=======

To Do:
------
Develop Tests
Please go to [[https://github.com/DFC-Incubator/dfc-dataone]] for the latest news and info.


## Requirements

* Jargon depends on Java 1.8+
* Jargon is built using Apache Maven2, see POM for dependencies
* Jargon supports iRODS 3.0 through iRODS 3.3.1 community, as well as iRODS 4.2.0 consortium

## Libraries

18 changes: 18 additions & 0 deletions dataone-node/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
FROM tomcat:jre8-alpine
LABEL organization="RENCI"
LABEL maintainer="[email protected]"
LABEL description="DFC Member Node"
ADD target/dataone-node.war /usr/local/tomcat/webapps/
ADD runit.sh /
ADD context.xml /usr/local/tomcat/conf
CMD ["/runit.sh"]
#CMD ["sh"]
# build: docker build -t diceunc/dataone:4.2.1.0-SNAPSHOT .


# run: docker run -i -p 8080:8080 -v /etc/irods-ext:/etc/irods-ext -v /etc/irods-ext/d1plugins:/etc/irods-ext/d1plugins -v /home/mcc/webdavcert:/tmp/cert --add-host irods420.irodslocal:172.16.250.101 diceunc/dataone:4.2.1.0-SNAPSHOT

# to add addl libs -v /etc/irods-ext/d1libs:/tmp/lib


# docker run -i -p 8080:8080 -v /home/mcc/external:/tmp/lib -v /etc/irods-ext:/etc/irods-ext -v /home/mcc/d1plugin:/etc/irods-ext/d1plugins -v /home/mcc/webdavcert:/tmp/cert --add-host irods420.irodslocal:172.16.250.101 diceunc/dataone:4.2.1.0-SNAPSHOT
24 changes: 24 additions & 0 deletions dataone-node/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@

# Project: DataONE Node
#### Date:
#### Release Version:
#### git tag:
#### Developer: Lisa Stillwell - RENCI, Mike Conway - DFC/RENCI

## News


## Requirements

* Depends on Java 1.8+
* Built using Apache Maven2, see POM for dependencies
* Supports iRODS 3.0 through iRODS 3.3.1 community, as well as iRODS 4.2.0 consortium

## Running

```
mvn spring-boot:run

```

## Changes
53 changes: 53 additions & 0 deletions dataone-node/TESTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Testing setup notes

### maven profile

add the following to your maven settings.xml profile in your home/.m2 directory

```xml

<profile>
<id>dataone</id>
<properties>
<test.plugin.jar.location>/home/mconway/temp/dataonejar</test.plugin.jar.location>
</properties>
</profile>



```

That .jar directory will be the place to put the .jar files for your pid and event service plugins. These are discovered by the PluginDiscoveryService.

### build and place your plugin jars in the test plugin jar location

Dummy plugins are being built, including the dummy-event-service-impl included in this repo. These dummy jar files can be copied out of the target directory of the dummy subproject
in order to run the unit and integration tests that depend on successful lookup

### sample unit test

Check out the PluginDiscoveryServiceTest unit test for examples of testing the loading of a plugin, as in this simple case:

```java

@Test
public void testInstanceEventService() throws Exception {
IRODSAccessObjectFactory aof = Mockito
.mock(IRODSAccessObjectFactory.class);
PublicationContext publicationContext = new PublicationContext();
RestConfiguration restConfiguration = new RestConfiguration();
restConfiguration.setPluginJarLocation(dataOneTestHelper
.getPluginJarLocation(dataOneProperties));
publicationContext.setRestConfiguration(restConfiguration);
publicationContext.setIrodsAccessObjectFactory(aof);
PluginDiscoveryService pluginDiscoveryService = new PluginDiscoveryService();
pluginDiscoveryService.setPublicationContext(publicationContext);
pluginDiscoveryService.init();

EventServiceAO eventServiceAO = pluginDiscoveryService
.instanceEventService();
Assert.assertNotNull("no eventServiceAO", eventServiceAO);
}


```
36 changes: 36 additions & 0 deletions dataone-node/context.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
<?xml version='1.0' encoding='utf-8'?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<!-- The contents of this file will be loaded for each web application -->
<Context>
<JarScanner scanManifest="false"/>
<!-- Default set of monitored resources. If one of these changes, the -->
<!-- web application will be reloaded. -->
<WatchedResource>WEB-INF/web.xml</WatchedResource>
<WatchedResource>${catalina.base}/conf/web.xml</WatchedResource>

<!-- Uncomment this to disable session persistence across Tomcat restarts -->
<!--
<Manager pathname="" />
-->

<!-- Uncomment this to enable Comet connection tacking (provides events
on session expiration as well as webapp lifecycle) -->
<!--
<Valve className="org.apache.catalina.valves.CometConnectionManagerValve" />
-->
</Context>
Original file line number Diff line number Diff line change
Expand Up @@ -57,3 +57,23 @@ irods.dataone.events.elasticsearch.searchindex=databook
irods.dataone.events.elasticsearch.searchtype=entity
irods.dataone.events.elasticsearch.cluster.name=databookIndexer

# REST Props
irods.host=localhost
irods.port=1247
irods.zone=tempZone
# user and password for dataone user
irods.username=test1
irods.password=test
# STANDARD | PAM AUTH
auth.type=STANDARD
default.storage.resource=
# web.interface.url is currently optional and not fully implemented, it can be left blank.
# sets jargon ssl negotiation policy for the client. Leaving to DONT_CARE defers to the server, and is recommended
# NO_NEGOTIATION, CS_NEG_REFUSE, CS_NEG_REQUIRE, CS_NEG_DONT_CARE
ssl.negotiation.policy=CS_NEG_DONT_CARE
# location for discovered plug-ins
plugin.jar.location=/etc/irods-ext/d1plugins

plugin.factory.pid=
plugin.factory.event=org.irods.jargon.dataone.events.defdb.DefaultEventServiceFactory
plugin.factory.repo=
Loading