SSL and PKI client authentication added, switched from tomcatnegoad to

tomcat built in

README.md

@@ -7,11 +7,11 @@ elasticsearch-security-plugin
 This plugin adds http/rest security functionality to Elasticsearch in kind of separate modules.
 Instead of Netty a embedded Tomcat 7 is used to process http/rest requests. 
 
-Currently for user based authentication and authorization 
-Kerberos and NTLM are supported through 3rd party library waffle (only on windows servers). 
-For UNIX servers Kerberos is supported through 3rd party library tomcatspnegoad (Works with any kerberos implementation. For authorization either Active Directory and generic LDAP is supported).
+Currently for user based authentication and authorization Kerberos/SPNEGO and NTLM are supported through 3rd party library waffle (only on windows servers). 
+For UNIX servers Kerberos/SPNEGO is supported through tomcat build in SPNEGO Valve (Works with any Kerberos implementation. For authorization either Active Directory and generic LDAP is supported).
+PKI/SSL client certificate authentication is also supported (CLIENT-CERT method). SSL/TLS is also supported without client authentication.
 
-You can use this plugin also without Kerberos/NTLM but then only host based authentication is available.
+You can use this plugin also without Kerberos/NTLM/PKI but then only host based authentication is available.
 <!--
 [![Build Status](https://travis-ci.org/salyh/elasticsearch-security-plugin.png?branch=master)](https://travis-ci.org/salyh/elasticsearch-security-plugin)-->
 
@@ -20,7 +20,7 @@ As of now two security modules are implemented:
 * Document level security (dls): Restrict actions on document level like who is allowed to query for which fields within a document
 
 <h3>Installation</h3> 
-(Until the first release is out you have to build this plugin yourself with maven)
+(Until the first release is out you have to build this plugin yourself with maven or download from the github release page and install manually)
 
 Build yourself:
 * Install maven
@@ -42,21 +42,35 @@ Enable the security plugin
 * ``script.disable_dynamic: true`` Dynamic scripts are unsafe and can potentially tamper this plugin
 * ``http.port: 9200`` Define exactly one port, Port ranges are not permitted
 
-Setup kerberos
-* ``security.kerberosimpl: waffle|spnegoad|none`` Kerberos implementation
+Setup Kerberos
+* ``security.kerberos.mode: waffle|spnegoad|none`` Kerberos implementation (spnegoad is tomcat-built in Kerberos/SPNEGO support)
 
 If you use spnegoad then you must provide the following configuration parameters:
-* ``security.spnegoad.ldapurls: ldap://myldaphost:389`` Ldap Server
-* ``security.spnegoad.isactivedirectory: true`` true if your ldap server is AD, false otherwise
-* ``security.spnegoad.login.conf.path: c:\path\to\login.conf`` JAAS login modules configuration
-* ``security.spnegoad.krb5.conf.path: /path/to/krb5.conf`` Kerberos configuration file
+* ``security.authorization.ldap.ldapurls: ldap://myldaphost:389`` Ldap Server
+* ``security.kerberos.login.conf.path: c:\path\to\login.conf`` JAAS login modules configuration
+* ``security.kerberos.krb5.conf.path: /path/to/krb5.conf`` Kerberos configuration file
+* ``security.authorization.ldap.connectionname: uid=admin,ou=system`` Low priv login to ldap server (Omit for anonymous authentication).
+* ``security.authorization.ldap.connectionpassword: secret`` Password for low priv login to ldap server (Omit for anonymous authentication). No encryption here, this is plaintext!
 
 If you use spnegoad and not Active Directory you may want configure your LDAP layout
-* ``security.spnegoad.ldap.usersearchbase: ""`` (Default is Root DSE)
-* ``security.spnegoad.ldap.usersearchpattern: Default is (&(objectClass=krb5principal) (krb5PrincipalName={0}))`` 0=fullqualified kerberos principal, 1=short kerberos principal
-* ``security.spnegoad.ldap.groupssearchbase: ""`` (Default is Root DSE)
-* ``security.spnegoad.ldap.groupssearchpattern: Default is (&(objectClass=groupofnames)(member={0}))`` 0=DN of user, 1=fullqualified kerberos principal, 2=short kerberos principal
-* ``security.spnegoad.ldap.rolenameattribute: cn`` (Default is cn)
+* ``security.authorization.ldap.userbase: ""`` (Default is Root DSE)
+* ``security.authorization.ldap.usersearch: (sAMAccountName={0})`` Default is (sAMAccountName={0})
+* ``security.authorization.ldap.rolebase: ""`` (Default is Root DSE)
+* ``security.authorization.ldap.rolesearch: (member={0})`` Default is (member={0})
+* ``security.authorization.ldap.rolename: cn`` (Default is cn)
+
+Optionally enable SSL/TLS
+* ``security.ssl.enabled: true`` Enable SSL
+* ``security.ssl.keystorefile: /path/to/keystore`` Keystore for private and public server certificates
+* ``security.ssl.keystorepass: changeit`` Password for the keystore
+* ``security.ssl.keystoretype: JKS`` Keystoretype (either JKS or PKCS12)
+
+If SSL is enabled you can use PKI/Client certificates for authentication
+* ``security.ssl.clientauth.enabled: true`` Enable PKI/Client certificates for authentication
+* ``security.ssl.clientauth.truststorefile: /path/to/truststore`` Keystore (truststore) for public client certificates which the server should trust
+* ``security.ssl.clientauth.truststorepass: changeit`` Password for the truststore
+* ``security.ssl.clientauth.truststoretype: JKS`` (either JKS or PKCS12)
+* ``security.ssl.userattribute: CN`` Name of the attribute from the client certificate user name which denotes the username for further authentication/authorization
 
 Optionally enable XFF 
 * ``security.http.xforwardedfor.header: X-Forwarded-For`` Enable XFF
@@ -380,8 +394,7 @@ and uses (documents of types t1 or t2 or both)<br>
 TODO<br>
 * http://tomcat.apache.org/tomcat-7.0-doc/api/org/apache/catalina/valves/RemoteIpValve.html
 * Check restict highlighting http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-highlighting.html
-* enforce script.disable_dynamic: true
+* Enforce script.disable_dynamic: true
 * Check restrict bulk requests and responses
-* User Tomcat JNDIRealm code for finding users/roles when not using AD
 * Add "at least authenticated" user rule
 * Provide rest api endpoint for displaying current security rules/status

pom.xml

@@ -4,7 +4,7 @@
 
 	<groupId>com.github.salyh.elasticsearch</groupId>
 	<artifactId>elasticsearch-security-plugin</artifactId>
-	<version>0.0.2.Beta2</version>
+	<version>0.0.2.Beta3</version>
 	<packaging>jar</packaging>
 
 	<name>elasticsearch-security-plugin</name>
@@ -162,23 +162,23 @@
 			<artifactId>waffle-tomcat7</artifactId>
 			<version>1.5</version>
 		</dependency>
-
+		<!-- 
 		<dependency>
   <groupId>net.sf.michael-o.tomcat</groupId>
   <artifactId>tomcat-authnz-spnego-ad-tc7</artifactId>
   <version>0.9.1</version>
-</dependency>
+</dependency>-->
 
 		<dependency>
     <groupId>com.jayway.jsonpath</groupId>
     <artifactId>json-path</artifactId>
-    <version>0.9.0</version>
+    <version>0.9.1</version>
 </dependency>
 
 <dependency>
     <groupId>com.jayway.jsonpath</groupId>
     <artifactId>json-path-assert</artifactId>
-    <version>0.9.0</version>
+    <version>0.9.1</version>
     <scope>test</scope>
 </dependency>
 
@@ -231,11 +231,25 @@
 			</testResource>
 			<testResource>
 				<directory>${basedir}/src/test/resources</directory>
-				<filtering>true</filtering>
+				<filtering>false</filtering>
 				<includes>
 					<include>**/*.*</include>
 				</includes>
+				<excludes>
+				<exclude>login.conf</exclude>
+				</excludes>
+			</testResource>
+
+			<testResource>
+				<directory>${basedir}/src/test/resources</directory>
+				<filtering>true</filtering>
+				<includes>
+					<include>login.conf</include>
+				</includes>
+
 			</testResource>
+
+
 		</testResources>
 
 		<plugins>

src/main/java/org/elasticsearch/plugins/security/SecurityPlugin.java

@@ -14,7 +14,7 @@ public class SecurityPlugin extends AbstractPlugin {
 	private final ESLogger log = Loggers.getLogger(this.getClass());
 
 	public SecurityPlugin() {
-		this.log.debug("Starting Security Plugin");
+		log.debug("Starting Security Plugin");
 	}
 
 	@SuppressWarnings("rawtypes")

src/main/java/org/elasticsearch/plugins/security/filter/ActionPathFilter.java

@@ -25,33 +25,33 @@ public void processSecure(final TomcatHttpServerRestRequest request,
 
 		if (SecurityUtil.stringContainsItemFromListAsTypeOrIndex(
 				request.path(), SecurityUtil.BUILT_IN_ADMIN_COMMANDS)) {
-			this.log.warn("Index- or Typename should not contains admin commands like "
+			log.warn("Index- or Typename should not contains admin commands like "
 					+ Arrays.toString(SecurityUtil.BUILT_IN_ADMIN_COMMANDS));
 		}
 
 		if (SecurityUtil.stringContainsItemFromListAsTypeOrIndex(
 				request.path(), SecurityUtil.BUILT_IN_READ_COMMANDS)) {
-			this.log.warn("Index- or Typename should not contains search commands like "
+			log.warn("Index- or Typename should not contains search commands like "
 					+ Arrays.toString(SecurityUtil.BUILT_IN_READ_COMMANDS));
 		}
 
 		if (SecurityUtil.stringContainsItemFromListAsTypeOrIndex(
 				request.path(), SecurityUtil.BUILT_IN_WRITE_COMMANDS)) {
-			this.log.warn("Index- or Typename should not contains write commands like "
+			log.warn("Index- or Typename should not contains write commands like "
 					+ Arrays.toString(SecurityUtil.BUILT_IN_WRITE_COMMANDS));
 		}
 
 		try {
 
 			final PermLevel permLevel = new PermLevelEvaluator(
-					this.securityService.getXContentSecurityConfiguration(
-							this.getType(), this.getId()))
-					.evaluatePerm(
-							SecurityUtil.getIndices(request),
-							SecurityUtil.getTypes(request),
-							this.getClientHostAddress(request),
-							new TomcatUserRoleCallback(request
-									.getHttpServletRequest()));
+					securityService.getXContentSecurityConfiguration(
+							getType(), getId()))
+			.evaluatePerm(
+					SecurityUtil.getIndices(request),
+					SecurityUtil.getTypes(request),
+					getClientHostAddress(request),
+					new TomcatUserRoleCallback(request
+							.getHttpServletRequest(),securityService.getSettings().get("security.ssl.userattribute")));
 
 			if (permLevel == PermLevel.NONE) {
 				SecurityUtil.send(request, channel, RestStatus.FORBIDDEN,
@@ -83,14 +83,14 @@ public void processSecure(final TomcatHttpServerRestRequest request,
 			filterChain.continueProcessing(request, channel);
 			return;
 		} catch (final MalformedConfigurationException e) {
-			this.log.error("Cannot parse security configuration ", e);
+			log.error("Cannot parse security configuration ", e);
 			SecurityUtil.send(request, channel,
 					RestStatus.INTERNAL_SERVER_ERROR,
 					"Cannot parse security configuration");
 
 			return;
 		} catch (final Exception e) {
-			this.log.error("Generic error: ", e);
+			log.error("Generic error: ", e);
 			SecurityUtil.send(request, channel,
 					RestStatus.INTERNAL_SERVER_ERROR,
 					"Generic error, see log for details");

src/main/java/org/elasticsearch/plugins/security/filter/DlsWriteFilter.java

@@ -50,14 +50,14 @@ public void processSecure(final TomcatHttpServerRestRequest request,
 			}
 
 			final List<String> dlsTokens = new PermDlsEvaluator(
-					this.securityService.getXContentSecurityConfiguration(
-							this.getType(), this.getId()))
-					.evaluatePerm(
-							SecurityUtil.getIndices(request),
-							SecurityUtil.getTypes(request),
-							this.getClientHostAddress(request),
-							new TomcatUserRoleCallback(request
-									.getHttpServletRequest()));
+					securityService.getXContentSecurityConfiguration(
+							getType(), getId()))
+			.evaluatePerm(
+					SecurityUtil.getIndices(request),
+					SecurityUtil.getTypes(request),
+					getClientHostAddress(request),
+					new TomcatUserRoleCallback(request
+							.getHttpServletRequest(),securityService.getSettings().get("security.ssl.userattribute")));
 
 			final String json = XContentHelper.convertToJson(request.content(),
 					true);
@@ -67,25 +67,25 @@ public void processSecure(final TomcatHttpServerRestRequest request,
 			// final XContentParser parser = XContentHelper.createParser(request
 			// .content());
 
-			this.log.debug("fieldlevelpermfilter orig: " + json);
+			log.debug("fieldlevelpermfilter orig: " + json);
 
-			this.log.debug("dls tokens: " + dlsTokens);
+			log.debug("dls tokens: " + dlsTokens);
 
 			final String id = SecurityUtil.getId(request);
 
 			try {
-				final GetResponse res = this.securityService
+				final GetResponse res = securityService
 						.getClient()
 						.get(new GetRequest(SecurityUtil.getIndices(request)
 								.get(0), SecurityUtil.getTypes(request).get(0),
 								id)).actionGet();
 
-				this.log.debug("document with id found: " + res.getId());
+				log.debug("document with id found: " + res.getId());
 
-				final List<DlsPermission> perms = this.securityService
+				final List<DlsPermission> perms = securityService
 						.parseDlsPermissions(res.getSourceAsBytesRef());
 
-				this.log.debug("perms " + perms);
+				log.debug("perms " + perms);
 
 				final List<String> fields = new ArrayList<String>();
 
@@ -98,7 +98,7 @@ public void processSecure(final TomcatHttpServerRestRequest request,
 					}
 
 				}
-				this.log.debug("ffields " + fields);
+				log.debug("ffields " + fields);
 
 				final Tuple<XContentType, Map<String, Object>> mapTuple = XContentHelper
 						.convertToMap(request.content(), true);
@@ -107,7 +107,7 @@ public void processSecure(final TomcatHttpServerRestRequest request,
 						.filter(mapTuple.v2(), fields.toArray(new String[0]),
 								new String[] { "*" });
 
-				this.log.debug("filteredSource " + filteredSource);
+				log.debug("filteredSource " + filteredSource);
 
 				final XContentBuilder sourceToBeReturned = XContentFactory
 						.contentBuilder(mapTuple.v1()).map(filteredSource);
@@ -121,20 +121,20 @@ public void processSecure(final TomcatHttpServerRestRequest request,
 			} catch (final Exception e) {
 				// TODO Auto-generated catch block
 				// e.printStackTrace();
-				this.log.debug("no document with id found: " + e.getMessage());
+				log.debug("no document with id found: " + e.getMessage());
 			}
 
 			filterChain.continueProcessing(request, channel);
 			return;
 		} catch (final MalformedConfigurationException e) {
-			this.log.error("Cannot parse security configuration ", e);
+			log.error("Cannot parse security configuration ", e);
 			SecurityUtil.send(request, channel,
 					RestStatus.INTERNAL_SERVER_ERROR,
 					"Cannot parse security configuration");
 
 			return;
 		} catch (final Exception e) {
-			this.log.error("Generic error: ", e);
+			log.error("Generic error: ", e);
 			SecurityUtil.send(request, channel,
 					RestStatus.INTERNAL_SERVER_ERROR,
 					"Generic error, see log for details");