Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3x version2 merge to master branch #115

Merged
merged 66 commits into from
Nov 17, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
6743c77
Adding clone feature for can clone table and create or replace clone
Sachin-Thakur Oct 25, 2023
e344da3
adding clone
Sachin-Thakur Oct 25, 2023
9beaf28
Adding
Sachin-Thakur Oct 25, 2023
2010aed
Adding
Sachin-Thakur Oct 25, 2023
ea90879
Adding
Sachin-Thakur Oct 25, 2023
1496e65
Adding
Sachin-Thakur Oct 25, 2023
779cc4e
adding clone
Sachin-Thakur Oct 26, 2023
899c332
Adding
Sachin-Thakur Oct 31, 2023
1f7d955
Adding
Sachin-Thakur Oct 31, 2023
0ea13d7
Adding
Sachin-Thakur Oct 31, 2023
28429ae
Adding
Sachin-Thakur Oct 31, 2023
5d679bd
Adding
Sachin-Thakur Nov 2, 2023
8bdf4f6
Adding
Sachin-Thakur Nov 2, 2023
f0fddab
adding
Sachin-Thakur Nov 2, 2023
019f146
Adding
Sachin-Thakur Nov 3, 2023
b6cd0fb
Adding
Sachin-Thakur Nov 3, 2023
825e18a
Adding
Sachin-Thakur Nov 6, 2023
dbb2512
Adding
Sachin-Thakur Nov 6, 2023
ddfceca
Adding
Sachin-Thakur Nov 6, 2023
3010f20
Adding
Sachin-Thakur Nov 6, 2023
fe39fc9
Adding
Sachin-Thakur Nov 6, 2023
a264913
Adding
Sachin-Thakur Nov 6, 2023
73f7763
Adding
Sachin-Thakur Nov 6, 2023
c36e2ad
Adding
Sachin-Thakur Nov 6, 2023
641d965
adding
Sachin-Thakur Nov 7, 2023
736829c
adding
Sachin-Thakur Nov 7, 2023
33e57fa
adding
Sachin-Thakur Nov 7, 2023
d53dac0
Adding
Sachin-Thakur Nov 7, 2023
7c03ce9
Adding
Sachin-Thakur Nov 7, 2023
9127f30
Adding
Sachin-Thakur Nov 7, 2023
539dca7
Adding
Sachin-Thakur Nov 7, 2023
4d0093d
Adding
Sachin-Thakur Nov 7, 2023
600bdfc
Adding
Sachin-Thakur Nov 7, 2023
80d4caa
Adding
Sachin-Thakur Nov 7, 2023
dbb8492
Adding
Sachin-Thakur Nov 7, 2023
3c7015c
Adding
Sachin-Thakur Nov 7, 2023
4525157
Adding
Sachin-Thakur Nov 8, 2023
00b0f2c
adding test dbt full refresh
Sachin-Thakur Nov 8, 2023
45b4516
Adding
Sachin-Thakur Nov 9, 2023
b51d898
Adding
Sachin-Thakur Nov 9, 2023
02ef02f
Adding
Sachin-Thakur Nov 9, 2023
b068f7f
Adding
Sachin-Thakur Nov 9, 2023
4ae2b70
Adding
Sachin-Thakur Nov 9, 2023
c815870
Adding
Sachin-Thakur Nov 9, 2023
57976b3
Adding
Sachin-Thakur Nov 9, 2023
42378a8
Adding
Sachin-Thakur Nov 9, 2023
4693dd5
Adding
Sachin-Thakur Nov 9, 2023
0aab99c
adding
Sachin-Thakur Nov 9, 2023
5054bc3
adding
Sachin-Thakur Nov 9, 2023
9ec7c6a
Merge branch 's3x-version2' of https://github.com/vertica/dbt-vertica…
Sachin-Thakur Nov 9, 2023
847fdc7
Adding
Sachin-Thakur Nov 9, 2023
95282cd
adding
Sachin-Thakur Nov 9, 2023
a6659cc
adding
Sachin-Thakur Nov 9, 2023
df9d814
adding
Sachin-Thakur Nov 9, 2023
2f4bbd9
adding
Sachin-Thakur Nov 9, 2023
d72a632
adding
Sachin-Thakur Nov 9, 2023
54d1694
adding
Sachin-Thakur Nov 9, 2023
43ab8ae
adding
Sachin-Thakur Nov 10, 2023
9605e47
Adding
Sachin-Thakur Nov 16, 2023
60c253c
Adding
Sachin-Thakur Nov 16, 2023
db7df68
Adding
Sachin-Thakur Nov 16, 2023
931c36c
Adding
Sachin-Thakur Nov 16, 2023
a843389
Adding
Sachin-Thakur Nov 16, 2023
e245b34
Adding
Sachin-Thakur Nov 16, 2023
a1bb61a
Adding
Sachin-Thakur Nov 17, 2023
00abf9d
Adding
Sachin-Thakur Nov 17, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions .github/workflows/vertica-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ['3.7', '3.8', '3.9', '3.10', '3.11']
python-version: [ '3.8','3.9' ,'3.10', '3.11']
services:
vertica:
image: vertica/vertica-ce:latest
Expand Down Expand Up @@ -45,13 +45,11 @@ jobs:
- name: Test Basic
run: python -m pytest tests/functional/adapter/test_basic.py
- name: Test Constraints
run: python -m pytest tests/functional/adapter/test_constraints.py
run: python -m pytest tests/functional/adapter/constraints/test_constraints.py
- name: Test Incremental
run: python -m pytest tests/functional/adapter/incremental/
- name: Test Concurrency
run: python -m pytest tests/functional/adapter/concurrency/
- name: Test ephemeral
run: python -m pytest tests/functional/adapter/ephemeral/
- name: Test Doc Generate
run: python -m pytest tests/functional/adapter/test_doc_gen.py
- name: Test Data Type Boolean
Expand All @@ -61,5 +59,6 @@ jobs:
- name: Test Data Type Int
run: python -m pytest tests/functional/adapter/utils/data_type/




24 changes: 24 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,30 @@
## Changelog
- This file provides a full account of all changes to dbt-vertica.
- "Breaking changes" listed under a version may require action from end users.

### 1.6.0

#### Features:
- Added support for [`dbt-core version 1.6.0`](https://github.com/dbt-labs/dbt-core/discussions/7958) according to DBT guidelines.
- new `clone` command
- Droped support for Python 3.7

#### Fixes:
- ensure support for revamped `dbt debug`
- new limit arg for `adapter.execute()`
- Added new functional tests and parameterize them by overriding fixtures:
- TestIncrementalConstraintsRollback
- TestTableContractSqlHeader
- TestIncrementalContractSqlHeader
- TestModelConstraintsRuntimeEnforcement
- TestConstraintQuotedColumn
- TestEquals
- TestMixedNullCompare
- TestNullCompare
- TestVerticaCloneNotPossible
- TestValidateSqlMethod


### 1.5.0
#### Features:
- Added support for [`dbt-core version 1.5.0`](https://github.com/dbt-labs/dbt-core/discussions/7213) according to DBT guidelines.
Expand Down
2 changes: 1 addition & 1 deletion dbt/adapters/vertica/__version__.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,4 @@



version = "1.5.0"
version = "1.6.0"
13 changes: 8 additions & 5 deletions dbt/adapters/vertica/connections.py
Original file line number Diff line number Diff line change
Expand Up @@ -185,14 +185,17 @@ def cancel(self, connection):
connection.handle.cancel()

@classmethod
def get_result_from_cursor(cls, cursor: Any) -> agate.Table:
def get_result_from_cursor(cls, cursor: Any, limit: Optional[int]) -> agate.Table:
data: List[Any] = []
column_names: List[str] = []

if cursor.description is not None:
column_names = [col[0] for col in cursor.description]
rows = cursor.fetchall()

if limit:
rows = cursor.fetchmany(limit)
else:
rows = cursor.fetchall()
# rows = cursor.fetchall()
# check result for every query if there are some queries with ; separator
while cursor.nextset():
check = cursor._message
Expand All @@ -206,13 +209,13 @@ def get_result_from_cursor(cls, cursor: Any) -> agate.Table:
return dbt.clients.agate_helper.table_from_data_flat(data, column_names)

def execute(
self, sql: str, auto_begin: bool = False, fetch: bool = False
self, sql: str, auto_begin: bool = False, fetch: bool = False, limit: Optional[int] = None
) -> Tuple[AdapterResponse, agate.Table]:
sql = self._add_query_comment(sql)
_, cursor = self.add_query(sql, auto_begin)
response = self.get_response(cursor)
if fetch:
table = self.get_result_from_cursor(cursor)
table = self.get_result_from_cursor(cursor,limit)
else:
table = dbt.clients.agate_helper.empty_table()
while cursor.nextset():
Expand Down
2 changes: 2 additions & 0 deletions dbt/adapters/vertica/impl.py
Original file line number Diff line number Diff line change
Expand Up @@ -164,3 +164,5 @@ def get_incremental_strategy_macro(self, model_context, strategy: str):

# This returns a callable macro
return model_context[macro_name]
def debug_query(self) -> None:
self.execute("select 1 as id")
16 changes: 16 additions & 0 deletions dbt/include/vertica/macros/materializations/clone.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@

{% macro vertica__can_clone_table() %}
{{ return(True) }}
{% endmacro %}


{% macro vertica__create_or_replace_clone(this_relation, defer_relation) %}


DROP TABLE IF EXISTS {{this_relation}};
create table
{{ this_relation }} as select * from
{{ defer_relation }}


{% endmacro %}
2 changes: 1 addition & 1 deletion example/demo_dbt_vmart/dbt_project.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ version: '1.0.0'
config-version: 2

# This setting configures which "profile" dbt uses for this project.
profile: 'vmart_project'
profile: vmart_project

# These configurations specify where dbt should look for different types of files.
# The `model-paths` config, for example, states that models in this project can be
Expand Down
14 changes: 7 additions & 7 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,10 @@
import sys
import re

# require python 3.7 or newer
if sys.version_info < (3, 7):
# require python 3.8 or newer
if sys.version_info < (3, 8):
print("Error: dbt does not support this version of Python.")
print("Please upgrade to Python 3.7 or higher.")
print("Please upgrade to Python 3.8 or higher.")
sys.exit(1)


Expand Down Expand Up @@ -78,7 +78,7 @@ def _get_dbt_core_version():


package_name = "dbt-vertica"
package_version = "1.5.0"
package_version = "1.6.0"
description = """Official vertica adapter plugin for dbt (data build tool)"""
dbt_core_version = _get_dbt_core_version()

Expand Down Expand Up @@ -110,10 +110,10 @@ def _get_dbt_core_version():
]
},
install_requires=[
'dbt-core==1.5.0',
'dbt-core==1.6.0',
# "dbt-core~={}".format(dbt_core_version),
'vertica-python>=1.1.0',
'dbt-tests-adapter==1.5.0',
'dbt-tests-adapter==1.6.0',
'python-dotenv==0.21.1',
],
classifiers=[
Expand All @@ -127,5 +127,5 @@ def _get_dbt_core_version():
"Topic :: Software Development :: Libraries :: Python Modules",
"Operating System :: OS Independent"
],
python_requires=">=3.7.2",
python_requires=">=3.8.0",
)
Loading