Code

Command to migrate MariaDB database#61

Code

Merged

amyfromandi merged 54 commits intomainUW-Macrostrat/macrostrat:mainfrom maria-migrateUW-Macrostrat/macrostrat:maria-migrateCopy head branch name to clipboardSep 4, 2024

+1,873-2,204

Commits

Commits on Jul 17, 2024

• restoring maria-migrate branch

  amyfromandi

  committed
  5a469e2

  Copy full SHA for 5a469e2

• modified .gitignore to ignore specific files

  amyfromandi

  committed
  677cc4a

  Copy full SHA for 677cc4a

Commits on Jul 22, 2024

• Refactored some code

  amyfromandi

  committed
  871128b

  Copy full SHA for 871128b

Commits on Jul 25, 2024

• updated utils.py

  amyfromandi

  committed
  66c161a

  Copy full SHA for 66c161a

• added liths prescript query

  amyfromandi

  committed
  b2700bd

  Copy full SHA for b2700bd

• Merge branch 'main' into maria-migrate

  Show description for ce87076
  davenquinn

  committed
  ce87076

  Copy full SHA for ce87076

• Moved entire mariadb_migration system to subdirectory

  davenquinn

  committed
  b4da967

  Copy full SHA for b4da967

• Incorporated legacy migration command into new directory

  davenquinn

  committed
  28c062a

  Copy full SHA for 28c062a

Commits on Jul 26, 2024

• Created basic command to run MariaDB CLI

  davenquinn

  committed
  00c1a02

  Copy full SHA for 00c1a02

• Remove need to connect with --net=host

  davenquinn

  committed
  1f234f9

  Copy full SHA for 1f234f9

• Add command that restores a database onto the MariaDB server

  davenquinn

  committed
  7f9193c

  Copy full SHA for 7f9193c

• Removed overly complex logging approach

  davenquinn

  committed
  93c7fb6

  Copy full SHA for 93c7fb6

• find_row_variances() compares maridb data to macrostrat data to ensure all data exists before migration in dev

  amyfromandi

  committed
  225e016

  Copy full SHA for 225e016

• Refactor - move stream reader to a utils file

  davenquinn

  committed
  12f5509

  Copy full SHA for 12f5509

• Fix reference to decoder

  davenquinn

  committed
  76b1b2b

  Copy full SHA for 76b1b2b

• Rearrange CLI utils

  davenquinn

  committed
  4c58f5e

  Copy full SHA for 4c58f5e

• Moved pre- and post- scripts to external SQL files

  davenquinn

  committed
  8d7eccd

  Copy full SHA for 8d7eccd

• Removed extra files created by IntelliJ

  davenquinn

  committed
  9b8ba34

  Copy full SHA for 9b8ba34

• Set up namespaced packages correctly in intellij

  davenquinn

  committed
  6245e56

  Copy full SHA for 6245e56

• Fix a bit more sql

  davenquinn

  committed
  8161b71

  Copy full SHA for 8161b71

• added find_col_variance() method

  amyfromandi

  committed
  985bab4

  Copy full SHA for 985bab4

• Merge branch 'maria-migrate' of https://github.com/UW-Macrostrat/macrostrat into maria-migrae

  amyfromandi

  committed
  884814b

  Copy full SHA for 884814b

• Move change assessment methods to separate file

  davenquinn

  committed
  acf35b9

  Copy full SHA for acf35b9

Commits on Jul 27, 2024

• Updated MariaDB dump/restore functions

  davenquinn

  committed
  aa4c84d

  Copy full SHA for aa4c84d

Commits on Jul 29, 2024

• PostgreSQL migration now mostly works

  davenquinn

  committed
  1262ea3

  Copy full SHA for 1262ea3

• Get reporting to work

  davenquinn

  committed
  de6470e

  Copy full SHA for de6470e

• Somewhat improved column and row count functions

  davenquinn

  committed
  e239b9b

  Copy full SHA for e239b9b

• Updated mariadb migration functions

  davenquinn

  committed
  e7650e6

  Copy full SHA for e7650e6

• Fixed MYSQLDump command and deadlock in dump/restore

  davenquinn

  committed
  0ab3678

  Copy full SHA for 0ab3678

• added find_col_variances() function

  amyfromandi

  committed
  1a9fbf3

  Copy full SHA for 1a9fbf3

• Updated some formatting

  davenquinn

  committed
  5fc2636

  Copy full SHA for 5fc2636

• Streamlined migration scripts and renamed files

  davenquinn

  committed
  4fdc534

  Copy full SHA for 4fdc534

• Added the ability to run different steps of the migration process

  davenquinn

  committed
  795395c

  Copy full SHA for 795395c

Commits on Jul 31, 2024

• accommodated code for port missing in macrostrat.toml. also added ssl mode parameters for pgloader command to work

  

  Amy Fromandi

  authored and

  Amy Fromandi

  committed
  5064029

  Copy full SHA for 5064029

• updated pgloader code to use new pg_temp_engine url and creds

  amyfromandi

  committed
  5152d49

  Copy full SHA for 5152d49

• Merged maria-migrate into this branch

  amyfromandi

  committed
  fb9e628

  Copy full SHA for fb9e628

Commits on Aug 1, 2024

• Added most up-to-date find_row_variances() and find_col_variances

  amyfromandi

  committed
  02d7112

  Copy full SHA for 02d7112

• Got find_row_variances() and find_col_variances to function!

  amyfromandi

  committed
  6f3774f

  Copy full SHA for 6f3774f

• Created utility function we might use to run PGLoader

  davenquinn

  committed
  15d7a90

  Copy full SHA for 15d7a90

Commits on Aug 2, 2024

• Updated utility functions for creating temporary database users

  davenquinn

  committed
  88c0191

  Copy full SHA for 88c0191

• Updated function names somewhat

  davenquinn

  committed
  0c6dda9

  Copy full SHA for 0c6dda9

Commits on Aug 6, 2024

• fixed pgloader issue by adding mariadb_migrator as superuser

  amyfromandi

  committed
  cb633f6

  Copy full SHA for cb633f6

• Fixed pgloader and check-data stats

  amyfromandi

  committed
  761903c

  Copy full SHA for 761903c

• Merge pull request #74 from UW-Macrostrat/maria-migrate-cli

  Show description for 22214cf
  davenquinn

  authored
  22214cf

  Copy full SHA for 22214cf

Commits on Aug 7, 2024

• repointed migration scripts to point to macrostrat.macrostrat_temp schema

  amyfromandi

  committed
  daa13ad

  Copy full SHA for daa13ad

Commits on Aug 8, 2024

• added preserve_macrostrat_data() function for the final steps in the mariadb-migrate-postgres step

  amyfromandi

  committed
  454129f

  Copy full SHA for 454129f

• Merge branch 'maria-migrate-cli', remote-tracking branch 'origin' into maria-migrate

  amyfromandi

  committed
  1f1cf02

  Copy full SHA for 1f1cf02

Commits on Aug 9, 2024

• Added code to resolve table and column variances across macrostrat and macrostrat_temp

  amyfromandi

  committed
  fd039a5

  Copy full SHA for fd039a5

Commits on Aug 13, 2024

• post script and preserve-macrostrat-data script is finalized!

  amyfromandi

  committed
  4c729c3

  Copy full SHA for 4c729c3

• refactored a little bit of data

  amyfromandi

  committed
  f03941c

  Copy full SHA for f03941c

Commits on Aug 30, 2024

• removing unnecessary schlep scripts. index files in schlep-index.sql and kep the schlep scripts for the tables that are modified in the pre/post migratino scripts.

  amyfromandi

  committed
  265feaf

  Copy full SHA for 265feaf

Commits on Sep 3, 2024

• Delete cli/macrostrat/cli/commands/schlep.py

  amyfromandi

  authored
  8f8ab02

  Copy full SHA for 8f8ab02

• Delete cli/macrostrat/cli/commands/table_meta.py

  amyfromandi

  authored
  78067e6

  Copy full SHA for 78067e6

Commits on Sep 4, 2024

• Modified output to pass data variance test with known 'issues'

  amyfromandi

  committed
  57a2ccf

  Copy full SHA for 57a2ccf