Skip to content

Commit

Permalink
1.5.1 (#97)
Browse files Browse the repository at this point in the history
* removed common part (#82)
* fixed exposed password bug (#83)
* preparations for join pushdown
  • Loading branch information
snehlsen authored Feb 27, 2019
1 parent 5609e18 commit fd8bffc
Show file tree
Hide file tree
Showing 141 changed files with 3,843 additions and 5,644 deletions.
41 changes: 37 additions & 4 deletions doc/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
- [Schema Metadata](#schema-metadata)
- [Expressions](#expressions)
- [Table](#table)
- [Join](#join)
- [Column Lookup](#column-lookup)
- [Literal](#literal)
- [Predicates](#predicates)
Expand Down Expand Up @@ -401,7 +402,7 @@ will produce the following Request, assuming that the Adapter has all required c

Notes
* `pushdownRequest`: Specification what needs to be pushed down. You can think of it like a parsed SQL statement.
* `from`: The requested from clause. Currently only tables are supported, joins might be supported in future.
* `from`: The requested from clause. This can be a table or a join.
* `selectList`: The requested select list elements, a list of expression. The order of the selectlist elements matters. If the select list is an empty list, we request at least a single column/expression, which could also be constant TRUE.
* `selectList.columnNr`: Position of the column in the virtual table, starting with 0
* `filter`: The requested filter (`where` clause), a single expression.
Expand Down Expand Up @@ -749,24 +750,56 @@ This element currently only occurs in from clause
```json
{
"type": "table",
"name": "CLICKS"
"name": "CLICKS",
"alias": "A"
}
```

Notes
* **alias**: This is an optional property and is added if the table has an alias in the original query.

### Join

This element currently only occurs in from clause

```json
{
"type": "join",
"join_type": "inner",
"left": {
...
},
"right" : {
...
},
"condition" : {
...
}
}
```

Notes
* **join_type**: Can be `inner`, `left_outer`, `right_outer` or `full_outer`.
* **left**: This can be a `table` or a `join`.
* **right**: This can be a `table` or a `join`.
* **condition**: This can be an arbitrary expression.

### Column Lookup

A column lookup is a reference to a table column. It can reference the table directly or via an alias.

```json
{
"type": "column",
"tableName": "T",
"tablePosFromClause": 0,
"tableAlias": "A",
"columnNr": 0,
"name": "ID"
}
```

Notes
* **tablePosFromClause**: Position of the table in the from clause, starting with 0. Required for joins where same table occurs several times.
* **tableAlias**: This is an optional property and is added if the referenced table has an alias.
* **columnNr**: Column number in the virtual table, starting with 0.

### Literal
Expand Down
1 change: 1 addition & 0 deletions jdbc-adapter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,7 @@ Property | Value
**ORA_CONNECTION_NAME** | Name of the connection to an Oracle database created with `CREATE CONNECTION`. Used by `IMPORT FROM ORA`.
**IS_LOCAL** | Only relevant if your data source is the same Exasol database where you create the virtual schema. Either `TRUE` or `FALSE` (default). If true, you are connecting to the local Exasol database (e.g. for testing purposes). In this case, the adapter can avoid the `IMPORT FROM JDBC` overhead.
**EXCEPTION_HANDLING** | Activates or deactivates different exception handling modes. Supported values: `IGNORE_INVALID_VIEWS` and `NONE` (default). Currently this property only affects the Teradata dialect.
**EXCLUDED_CAPABILITIES** | A comma-separated list of capabilities that you want to deactivate (although the adapter might support them).
**IGNORE_ERRORS** | Is used to ignore errors thrown by the adapter. Supported values: 'POSTGRESQL_UPPERCASE_TABLES' (see PostgreSQL dialect documentation).


Expand Down
8 changes: 4 additions & 4 deletions jdbc-adapter/doc/deploying_the_virtual_schema_adapter.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ cd virtual-schemas/jdbc-adapter/
mvn clean -DskipTests package
```

The resulting fat JAR is stored in `virtualschema-jdbc-adapter-dist/target/virtualschema-jdbc-adapter-dist-1.4.1.jar`.
The resulting fat JAR is stored in `virtualschema-jdbc-adapter-dist/target/virtualschema-jdbc-adapter-dist-1.5.1.jar`.

## Uploading the Adapter JAR Archive

Expand All @@ -42,8 +42,8 @@ Following steps are required to upload a file to a bucket:
1. Now upload the file into this bucket, e.g. using curl (adapt the hostname, BucketFS port, bucket name and bucket write password).

```bash
curl -X PUT -T virtualschema-jdbc-adapter-dist/target/virtualschema-jdbc-adapter-dist-1.4.1.jar \
http://w:[email protected]:2580/bucket1/virtualschema-jdbc-adapter-dist-1.4.1.jar
curl -X PUT -T virtualschema-jdbc-adapter-dist/target/virtualschema-jdbc-adapter-dist-1.5.1.jar \
http://w:[email protected]:2580/bucket1/virtualschema-jdbc-adapter-dist-1.5.1.jar
```

See chapter 3.6.4. "The synchronous cluster file system BucketFS" in the EXASolution User Manual for more details about BucketFS.
Expand Down Expand Up @@ -75,7 +75,7 @@ CREATE JAVA ADAPTER SCRIPT adapter.jdbc_adapter AS

// This will add the adapter jar to the classpath so that it can be used inside the adapter script
// Replace the names of the bucketfs and the bucket with the ones you used.
%jar /buckets/your-bucket-fs/your-bucket/virtualschema-jdbc-adapter-dist-1.4.1.jar;
%jar /buckets/your-bucket-fs/your-bucket/virtualschema-jdbc-adapter-dist-1.5.1.jar;

// You have to add all files of the data source jdbc driver here (e.g. Hive JDBC driver files)
%jar /buckets/your-bucket-fs/your-bucket/name-of-data-source-jdbc-driver.jar;
Expand Down
2 changes: 1 addition & 1 deletion jdbc-adapter/doc/developing_an_sql_dialect.md
Original file line number Diff line number Diff line change
Expand Up @@ -292,7 +292,7 @@ CREATE OR REPLACE JAVA ADAPTER SCRIPT adapter.jdbc_adapter

// This will add the adapter jar to the classpath so that it can be used inside the adapter script
// Replace the names of the bucketfs and the bucket with the ones you used.
%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.4.1.jar;
%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.5.1.jar;

// You have to add all files of the data source jdbc driver here (e.g. MySQL or Hive)

Expand Down
2 changes: 1 addition & 1 deletion jdbc-adapter/doc/sql_dialects/db2.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ CREATE or replace JAVA ADAPTER SCRIPT adapter.jdbc_adapter AS

// This will add the adapter jar to the classpath so that it can be used inside the adapter script
// Replace the names of the bucketfs and the bucket with the ones you used.
%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.4.1.jar;
%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.5.1.jar;

// DB2 Driver files
%jar /buckets/bucketfs1/bucket1/db2jcc4.jar;
Expand Down
2 changes: 1 addition & 1 deletion jdbc-adapter/doc/sql_dialects/exasol.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ After uploading the adapter jar, the adapter script can be created as follows:
CREATE SCHEMA adapter;
CREATE JAVA ADAPTER SCRIPT adapter.jdbc_adapter AS
%scriptclass com.exasol.adapter.jdbc.JdbcAdapter;
%jar /buckets/your-bucket-fs/your-bucket/virtualschema-jdbc-adapter-dist-1.4.1.jar;
%jar /buckets/your-bucket-fs/your-bucket/virtualschema-jdbc-adapter-dist-1.5.1.jar;
/
```

Expand Down
2 changes: 1 addition & 1 deletion jdbc-adapter/doc/sql_dialects/hive.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ CREATE SCHEMA adapter;
CREATE JAVA ADAPTER SCRIPT jdbc_adapter AS
%scriptclass com.exasol.adapter.jdbc.JdbcAdapter;

%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.4.1.jar;
%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.5.1.jar;

%jar /buckets/bucketfs1/bucket1/hive_metastore.jar;
%jar /buckets/bucketfs1/bucket1/hive_service.jar;
Expand Down
2 changes: 1 addition & 1 deletion jdbc-adapter/doc/sql_dialects/impala.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ CREATE SCHEMA adapter;
CREATE JAVA ADAPTER SCRIPT jdbc_adapter AS
%scriptclass com.exasol.adapter.jdbc.JdbcAdapter;

%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.4.1.jar;
%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.5.1.jar;

%jar /buckets/bucketfs1/bucket1/hive_metastore.jar;
%jar /buckets/bucketfs1/bucket1/hive_service.jar;
Expand Down
2 changes: 1 addition & 1 deletion jdbc-adapter/doc/sql_dialects/oracle.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ CREATE JAVA ADAPTER SCRIPT adapter.jdbc_oracle AS

// You need to replace `your-bucket-fs` and `your-bucket` to match the actual location
// of the adapter jar.
%jar /buckets/your-bucket-fs/your-bucket/virtualschema-jdbc-adapter-dist-1.4.1.jar;
%jar /buckets/your-bucket-fs/your-bucket/virtualschema-jdbc-adapter-dist-1.5.1.jar;

// Add the oracle jdbc driver to the classpath
%jar /buckets/bucketfs1/bucket1/ojdbc7-12.1.0.2.jar
Expand Down
2 changes: 1 addition & 1 deletion jdbc-adapter/doc/sql_dialects/postgresql.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ CREATE OR REPLACE JAVA ADAPTER SCRIPT adapter.jdbc_adapter

// This will add the adapter jar to the classpath so that it can be used inside the adapter script
// Replace the names of the bucketfs and the bucket with the ones you used.
%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.4.1.jar;
%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.5.1.jar;

// You have to add all files of the data source jdbc driver here (e.g. MySQL or Hive)
%jar /buckets/bucketfs1/bucket1/postgresql-42.0.0.jar;
Expand Down
2 changes: 1 addition & 1 deletion jdbc-adapter/doc/sql_dialects/redshift.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ CREATE OR REPLACE JAVA ADAPTER SCRIPT adapter.jdbc_adapter

// This will add the adapter jar to the classpath so that it can be used inside the adapter script
// Replace the names of the bucketfs and the bucket with the ones you used.
%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.4.1.jar;
%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.5.1.jar;

// You have to add all files of the data source jdbc driver here (e.g. MySQL or Hive)

Expand Down
2 changes: 1 addition & 1 deletion jdbc-adapter/doc/sql_dialects/sql_server.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ CREATE OR REPLACE JAVA ADAPTER SCRIPT adapter.sql_server_jdbc_adapter

// This will add the adapter jar to the classpath so that it can be used inside the adapter script
// Replace the names of the bucketfs and the bucket with the ones you used.
%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.4.1.jar;
%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.5.1.jar;

// You have to add all files of the data source jdbc driver here
%jar /buckets/bucketfs1/bucket1/jtds.jar;
Expand Down
2 changes: 1 addition & 1 deletion jdbc-adapter/doc/sql_dialects/sybase.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ CREATE OR REPLACE JAVA ADAPTER SCRIPT adapter.jdbc_adapter
AS

%scriptclass com.exasol.adapter.jdbc.JdbcAdapter;
%jar /buckets/bucketfs1/virtualschema/virtualschema-jdbc-adapter-dist-1.4.1.jar;
%jar /buckets/bucketfs1/virtualschema/virtualschema-jdbc-adapter-dist-1.5.1.jar;
%jar /buckets/bucketfs1/virtualschema/jtds-1.3.1.jar;
/
```
Expand Down
2 changes: 1 addition & 1 deletion jdbc-adapter/doc/sql_dialects/teradata.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ CREATE OR REPLACE JAVA ADAPTER SCRIPT adapter.jdbc_adapter

// This will add the adapter jar to the classpath so that it can be used inside the adapter script
// Replace the names of the bucketfs and the bucket with the ones you used.
%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.4.1.jar;
%jar /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.5.1.jar;

// You have to add all files of the data source jdbc driver here (e.g. MySQL or Hive)
%jar /buckets/bucketfs1/bucket1/terajdbc4.jar;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ general:
debugAddress: '192.168.0.12:3000' # Address which will be defined as DEBUG_ADDRESS in the virtual schemas
bucketFsUrl: http://exasol-host:2580/bucket1
bucketFsPassword: bucket1
jdbcAdapterPath: /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.4.1.jar
jdbcAdapterPath: /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.5.1.jar

exasol:
runIntegrationTests: true
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ general:
debugAddress: '192.168.0.12:3000' # Address which will be defined as DEBUG_ADDRESS in the virtual schemas
bucketFsUrl: http://exasol-host:2580/bucket1
bucketFsPassword: bucket1
jdbcAdapterPath: /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.4.1.jar
jdbcAdapterPath: /buckets/bucketfs1/bucket1/virtualschema-jdbc-adapter-dist-1.5.1.jar

exasol:
runIntegrationTests: true
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ general:
debugAddress: ''
bucketFsUrl: http://127.0.0.1:6594/default
bucketFsPassword: write
jdbcAdapterPath: /buckets/bfsdefault/default/virtualschema-jdbc-adapter-dist-1.4.1.jar
jdbcAdapterPath: /buckets/bfsdefault/default/virtualschema-jdbc-adapter-dist-1.5.1.jar

exasol:
runIntegrationTests: true
Expand Down
2 changes: 1 addition & 1 deletion jdbc-adapter/local/integration-test-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ general:
debugAddress: '10.44.1.228:3000' # Address which will be defined as DEBUG_ADDRESS in the virtual schemas
bucketFsUrl: http://localhost:2580/jars
bucketFsPassword: public
jdbcAdapterPath: /buckets/bfsdefault/jars/virtualschema-jdbc-adapter-dist-1.4.1.jar
jdbcAdapterPath: /buckets/bfsdefault/jars/virtualschema-jdbc-adapter-dist-1.5.1.jar

exasol:
runIntegrationTests: true
Expand Down
3 changes: 1 addition & 2 deletions jdbc-adapter/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,12 @@
<packaging>pom</packaging>

<modules>
<module>virtualschema-common</module>
<module>virtualschema-jdbc-adapter</module>
<module>virtualschema-jdbc-adapter-dist</module>
</modules>

<properties>
<product.version>1.4.1</product.version>
<product.version>1.5.1</product.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
Expand Down
27 changes: 0 additions & 27 deletions jdbc-adapter/virtualschema-common/pom.xml

This file was deleted.

This file was deleted.

This file was deleted.

Loading

0 comments on commit fd8bffc

Please sign in to comment.