Code

Update for mono repo#111

Code

Merged

slesaad merged 126 commits intodevNASA-IMPACT/veda-data-airflow:devfrom update-for-mono-repoNASA-IMPACT/veda-data-airflow:update-for-mono-repoCopy head branch name to clipboardMar 8, 2024

+372-434

Commits

Commits on Apr 19, 2023

• Deploy AIRFLOW to GHGC UAH

  

  Abdel Marouane

  committed
  b4d5995

  Copy full SHA for b4d5995

• Deploy AIRFLOW to GHGC UAH

  

  Abdel Marouane

  committed
  764658a

  Copy full SHA for 764658a

• Deploy AIRFLOW to GHGC UAH

  

  Abdel Marouane

  committed
  36a306f

  Copy full SHA for 36a306f

• Deploy AIRFLOW to GHGC UAH

  

  Abdel Marouane

  committed
  dedae97

  Copy full SHA for dedae97

• Deploy AIRFLOW to GHGC UAH

  

  Abdel Marouane

  committed
  99d04be

  Copy full SHA for 99d04be

• Deploy AIRFLOW to GHGC UAH

  

  Abdel Marouane

  committed
  d236065

  Copy full SHA for d236065

• Deploy AIRFLOW to GHGC UAH

  

  Abdel Marouane

  committed
  1ddf415

  Copy full SHA for 1ddf415

• Setup gitflow

  

  Abdel Marouane

  committed
  01bf7b8

  Copy full SHA for 01bf7b8

Commits on May 1, 2023

• Provision RDS cluster

  

  Abdel Marouane

  committed
  a813dff

  Copy full SHA for a813dff

Commits on May 9, 2023

• Update github actions and update bucket permissions (#2)

  Show description for 1bd28ea
  

  slesaad

  and

  Caden-Helbling

  authored
  1bd28ea

  Copy full SHA for 1bd28ea

Commits on May 10, 2023

• Merge branch 'dev' of https://github.com/NASA-IMPACT/ghgc-data-airflow into dev

  

  Abdel Marouane

  committed
  9f200b1

  Copy full SHA for 9f200b1

• Fix CICD

  

  Abdel Marouane

  committed
  abbba97

  Copy full SHA for abbba97

• Fix CICD

  

  Abdel Marouane

  committed
  e8c9aeb

  Copy full SHA for e8c9aeb

• Fix CICD

  

  Abdel Marouane

  committed
  504cb2e

  Copy full SHA for 504cb2e

• Fix CICD

  

  Abdel Marouane

  committed
  95fac5e

  Copy full SHA for 95fac5e

• Fix CICD

  

  Abdel Marouane

  committed
  6ea7a84

  Copy full SHA for 6ea7a84

• Fix CICD

  

  Abdel Marouane

  committed
  2886922

  Copy full SHA for 2886922

• Fix CICD

  

  Abdel Marouane

  committed
  661f506

  Copy full SHA for 661f506

• Fix CICD

  

  Abdel Marouane

  committed
  50a8626

  Copy full SHA for 50a8626

• Fix CICD

  

  Abdel Marouane

  committed
  bd34c37

  Copy full SHA for bd34c37

• Fix CICD

  

  Abdel Marouane

  committed
  7a2ad14

  Copy full SHA for 7a2ad14

• Fix CICD and remove optional lines

  

  Abdel Marouane

  committed
  78a649c

  Copy full SHA for 78a649c

• Fix CICD and remove optional lines with permission

  

  Abdel Marouane

  committed
  6c4332f

  Copy full SHA for 6c4332f

• Fix CICD and remove optional lines with vpc

  

  Abdel Marouane

  committed
  66736aa

  Copy full SHA for 66736aa

• Fix CICD and remove optional lines with vpc and subnet

  

  Abdel Marouane

  committed
  c8e4696

  Copy full SHA for c8e4696

• Fix CICD fix vpc and subnet

  

  Abdel Marouane

  committed
  fc5a40e

  Copy full SHA for fc5a40e

• Fix CICD fix variables

  

  Abdel Marouane

  committed
  69a48ec

  Copy full SHA for 69a48ec

Commits on May 25, 2023

• Fix optional var references in dags (#3)

  slesaad

  authored
  c9e2afe

  Copy full SHA for c9e2afe

• Fix containerOverrides expected datatype

  slesaad

  authored
  e6c76ef

  Copy full SHA for e6c76ef

• Uncomment subnets

  slesaad

  authored
  e675b71

  Copy full SHA for e675b71

• Fix GDAL session for non assume role

  slesaad

  authored
  ffe955b

  Copy full SHA for ffe955b

• Add some logging

  slesaad

  authored
  9535729

  Copy full SHA for 9535729

• Throw exception if stac build fails

  slesaad

  authored
  cb3f2f5

  Copy full SHA for cb3f2f5

• Update requirements

  slesaad

  authored
  bd176d8

  Copy full SHA for bd176d8

Commits on May 30, 2023

• Upgrading the TF base module to v1.1.4

  

  Abdel Marouane

  committed
  7009c1a

  Copy full SHA for 7009c1a

Commits on Jun 12, 2023

• Add reference to ghc-deploy-dev-mcp branch

  Caden-Helbling

  committed
  37a1bb8

  Copy full SHA for 37a1bb8

Commits on Jun 14, 2023

• Update subnet tagname (#4)

  Show description for ec5c619
  slesaad

  authored
  ec5c619

  Copy full SHA for ec5c619

• Update env files (#5)

  slesaad

  authored
  ebb1309

  Copy full SHA for ebb1309

• Update env files (#6)

  slesaad

  authored
  6b30c83

  Copy full SHA for 6b30c83

• Update boundary policy name (#7)

  slesaad

  authored
  6f1b34a

  Copy full SHA for 6f1b34a

• Update boundary policy name (#8)

  slesaad

  authored
  a8fa230

  Copy full SHA for a8fa230

Commits on Jun 15, 2023

• Try deployment to staging

  amarouane-ABDELHAK

  committed
  92f7aa5

  Copy full SHA for 92f7aa5

• Try deployment to staging

  amarouane-ABDELHAK

  committed
  b7073bc

  Copy full SHA for b7073bc

• Merge pull request #9 from NASA-IMPACT/main

  Show description for ba3e7d2
  amarouane-ABDELHAK

  authored
  ba3e7d2

  Copy full SHA for ba3e7d2

Commits on Jun 26, 2023

• Deploy MWAA

  amarouane-ABDELHAK

  committed
  5234648

  Copy full SHA for 5234648

• Deploy MWAA

  amarouane-ABDELHAK

  committed
  2e3032a

  Copy full SHA for 2e3032a

• Delete .envs

  amarouane-ABDELHAK

  committed
  f687e1f

  Copy full SHA for f687e1f

• Add vector-secret-name

  amarouane-ABDELHAK

  committed
  d108471

  Copy full SHA for d108471

• Add MWAA variables

  amarouane-ABDELHAK

  committed
  21673c5

  Copy full SHA for 21673c5

• Add comma for airflow vars:

  amarouane-ABDELHAK

  committed
  3785fcc

  Copy full SHA for 3785fcc

Commits on Jul 5, 2023

• Finalizing the deployment

  amarouane-ABDELHAK

  committed
  b1e90ca

  Copy full SHA for b1e90ca

• Add ingestor url

  amarouane-ABDELHAK

  committed
  67af4b3

  Copy full SHA for 67af4b3

• Add deploy debugger

  amarouane-ABDELHAK

  committed
  8e5f360

  Copy full SHA for 8e5f360

• Add deploy debugger

  amarouane-ABDELHAK

  committed
  2737807

  Copy full SHA for 2737807

• deploy with correct ingestor

  amarouane-ABDELHAK

  committed
  086b9cb

  Copy full SHA for 086b9cb

Commits on Jul 6, 2023

• Change stack name

  amarouane-ABDELHAK

  committed
  d7ca801

  Copy full SHA for d7ca801

Commits on Jul 17, 2023

• Make permission boundaries name optional

  amarouane-ABDELHAK

  committed
  2778d97

  Copy full SHA for 2778d97

• Make permission boundaries name optional

  amarouane-ABDELHAK

  committed
  f8f88ee

  Copy full SHA for f8f88ee

Commits on Jul 26, 2023

• Merge branch 'deploy_all_stack' into dev

  amarouane-ABDELHAK

  committed
  18a366f

  Copy full SHA for 18a366f

• Fix permission boundaries name

  amarouane-ABDELHAK

  committed
  1312d3f

  Copy full SHA for 1312d3f

• Remove public access for S3

  amarouane-ABDELHAK

  committed
  2512984

  Copy full SHA for 2512984

Commits on Aug 1, 2023

• Merge changes from upstream (#11)

  Show description for f92c171
  slesaad

  authored
  f92c171

  Copy full SHA for f92c171

• Update action.yml

  slesaad

  committed
  5ef139b

  Copy full SHA for 5ef139b

• Fix env vars

  slesaad

  committed
  3456a8e

  Copy full SHA for 3456a8e

• Source env var

  slesaad

  committed
  2efb4c8

  Copy full SHA for 2efb4c8

• Fix prefix env

  slesaad

  committed
  5ffa815

  Copy full SHA for 5ffa815

• Update action to deploy

  slesaad

  committed
  8351234

  Copy full SHA for 8351234

• Update stac_ingestor_url env var

  slesaad

  committed
  ae4a2d7

  Copy full SHA for ae4a2d7

• Update exported env vars

  slesaad

  committed
  1f1778c

  Copy full SHA for 1f1778c

• Update dummy dag

  slesaad

  committed
  e173407

  Copy full SHA for e173407

• Update dummy dag again

  slesaad

  committed
  a7ad81c

  Copy full SHA for a7ad81c

• Delete submodule

  slesaad

  committed
  7ca5707

  Copy full SHA for 7ca5707

• Use mwaa_tf_module v1.1.5

  slesaad

  committed
  8003455

  Copy full SHA for 8003455

• Update rasterio version

  slesaad

  committed
  8fb576b

  Copy full SHA for 8fb576b

• Make assume_role optional in cogify/transfer

  slesaad

  committed
  c8524b7

  Copy full SHA for c8524b7

Commits on Aug 2, 2023

• Fix inconsistencies

  slesaad

  committed
  861cd91

  Copy full SHA for 861cd91

• Update build_stac readme

  slesaad

  committed
  e0bc3d2

  Copy full SHA for e0bc3d2

• Fix rasterio open issue

  slesaad

  committed
  f214907

  Copy full SHA for f214907

• Fix iteration of dict items

  slesaad

  committed
  a9a81e0

  Copy full SHA for a9a81e0

• Update handler

  slesaad

  committed
  af930d0

  Copy full SHA for af930d0

• Add newline

  slesaad

  committed
  e469267

  Copy full SHA for e469267

Commits on Aug 4, 2023

• Merge veda/dev into dev

  slesaad

  committed
  14ddd5c

  Copy full SHA for 14ddd5c

• Merge pull request #12 from US-GHG-Center/dev

  Show description for 7c92726
  amarouane-ABDELHAK

  authored
  7c92726

  Copy full SHA for 7c92726

• Fix sts session for non role assumption

  slesaad

  committed
  97a52a1

  Copy full SHA for 97a52a1

• Update mwaa version

  slesaad

  committed
  e6b69a2

  Copy full SHA for e6b69a2

Commits on Aug 7, 2023

• Update mwaa_tf_module version, fix sts session bug (#13)

  Show description for 0cec9e1
  slesaad

  authored
  0cec9e1

  Copy full SHA for 0cec9e1

• Support for multi-asset items, upgrade mwaa_tf version

  slesaad

  authored
  a9d7c73

  Copy full SHA for a9d7c73

Commits on Aug 8, 2023

• Make ASSUME_ROLE_READ_ARN optional

  slesaad

  committed
  6d3a245

  Copy full SHA for 6d3a245

• Update exception catching

  slesaad

  committed
  5b3b710

  Copy full SHA for 5b3b710

• Str param type for empty CopySourceIfNoneMatch

  slesaad

  committed
  6ac6bdf

  Copy full SHA for 6ac6bdf

• Update permissions for specified buckets

  slesaad

  committed
  40c4e0f

  Copy full SHA for 40c4e0f

• Give access to all buckets starting with ghgc-

  slesaad

  committed
  245d0ef

  Copy full SHA for 245d0ef

Commits on Aug 9, 2023

• Fix data transfer copy error (#15)

  Show description for 116c808
  slesaad

  authored
  116c808

  Copy full SHA for 116c808

• Merge branch 'production' into main

  slesaad

  authored
  1fcdd56

  Copy full SHA for 1fcdd56

Commits on Aug 23, 2023

• Specify data asset roles, change active runs

  slesaad

  committed
  374cab0

  Copy full SHA for 374cab0

• Specify asset role, change dag max run for build_stac (#17)

  slesaad

  authored
  41f88c6

  Copy full SHA for 41f88c6

Commits on Sep 20, 2023

• Move proj and raster details into each asset (#19)

  Show description for fdfd528
  slesaad

  authored
  fdfd528

  Copy full SHA for fdfd528

• Update format for start and end datetime (#21)

  slesaad

  authored
  8d311b5

  Copy full SHA for 8d311b5

• Merge branch 'main' into dev

  slesaad

  authored
  537f43d

  Copy full SHA for 537f43d

Commits on Oct 30, 2023

• Add stac url output

  amarouane-ABDELHAK

  committed
  83fc487

  Copy full SHA for 83fc487

• update lock

  amarouane-ABDELHAK

  committed
  3fda022

  Copy full SHA for 3fda022

Commits on Oct 31, 2023

• update prefix

  amarouane-ABDELHAK

  committed
  10491ca

  Copy full SHA for 10491ca

Commits on Feb 22, 2024

• Add VEDA_ prefix to some env vars

  slesaad

  committed
  e3e294e

  Copy full SHA for e3e294e

Commits on Feb 23, 2024

• Remove ingestor_stack_name reference

  slesaad

  committed
  d55f818

  Copy full SHA for d55f818

• Update action versions

  slesaad

  committed
  1df5bde

  Copy full SHA for 1df5bde

• Prepend veda_ to var

  slesaad

  committed
  c62d2f7

  Copy full SHA for c62d2f7

• Prepend missing VEDA_

  slesaad

  committed
  0733999

  Copy full SHA for 0733999

Commits on Feb 27, 2024

• Merge branch 'dev' into update-for-mono-repo

  slesaad

  authored
  427e419

  Copy full SHA for 427e419

Commits on Mar 5, 2024

• Merge branch 'dev' into update-for-mono-repo

  slesaad

  committed
  7911753

  Copy full SHA for 7911753

• Create jwks_url from aws secrets, remove unused functions

  slesaad

  committed
  27a9c6d

  Copy full SHA for 27a9c6d

Commits on Mar 6, 2024

• Update env var names

  slesaad

  committed
  70c0d92

  Copy full SHA for 70c0d92

• Fix multi-asset metadata generation for proj and raster bands

  slesaad

  committed
  916adc5

  Copy full SHA for 916adc5

• Update role session name

  slesaad

  committed
  2707890

  Copy full SHA for 2707890

• Remove remenant veda_ prefix

  slesaad

  committed
  9f0a097

  Copy full SHA for 9f0a097

• Remove unnecessary branch from cicd

  slesaad

  committed
  518b0b5

  Copy full SHA for 518b0b5

• ghgc -> veda in sync-env.sh

  slesaad

  committed
  c386c91

  Copy full SHA for c386c91

Commits on Mar 7, 2024

• Merge branch 'dev' into update-for-mono-repo

  slesaad

  committed
  aa54162

  Copy full SHA for aa54162

• Make vector_vpc optional, add permission boundary to lambda role

  slesaad

  committed
  82b062d

  Copy full SHA for 82b062d

• Fix var name

  slesaad

  committed
  54d8ffd

  Copy full SHA for 54d8ffd

• Make vector sg optional

  slesaad

  committed
  554da6c

  Copy full SHA for 554da6c

Commits on Mar 8, 2024

• workflow api fixes, sts assume role

  smohiudd

  committed
  d3f1b48

  Copy full SHA for d3f1b48

• workflows and tf changes

  smohiudd

  committed
  e59d2a7

  Copy full SHA for e59d2a7

• fix task def name

  smohiudd

  committed
  61447cc

  Copy full SHA for 61447cc

• split programmatic and workflow auth clients in tf

  smohiudd

  committed
  35c19d3

  Copy full SHA for 35c19d3

• Update env var name for cognito client secret

  slesaad

  authored
  692eb27

  Copy full SHA for 692eb27

• Update .env.example

  Show description for e749c4b
  

  slesaad

  and

  anayeaye

  authored
  e749c4b

  Copy full SHA for e749c4b