Code

Refactor interface classes: phase 1#25

Code

Merged

LswaN58 merged 133 commits intomainopengamedata/ogd-common:mainfrom issue/3-refactor-interface-classesopengamedata/ogd-common:issue/3-refactor-interface-classesCopy head branch name to clipboardJan 7, 2025

+2,437-1,216

Commits

Commits on Oct 3, 2024

• From core branch, rename 'interfaces' to 'connectors'.

  Show description for 995a092
  LswaN58

  committed
  995a092

  Copy full SHA for 995a092

• Add some documentation strings to Schema base class.

  LswaN58

  committed
  45e385c

  Copy full SHA for 45e385c

Commits on Oct 4, 2024

• Add StorageConnector base class, from core branch.

  LswaN58

  committed
  51fc8d4

  Copy full SHA for 51fc8d4

• Add changes to Interface class from core branch.

  LswaN58

  committed
  02c68e7

  Copy full SHA for 02c68e7

• Update imports.

  LswaN58

  committed
  e0272d1

  Copy full SHA for e0272d1

• Schema reorganization and new schemas from core branch.

  LswaN58

  committed
  6ed3535

  Copy full SHA for 6ed3535

• Add most of the remaining changes from core branch.

  LswaN58

  committed
  dda3605

  Copy full SHA for dda3605

• Sort out updates to imports.

  LswaN58

  committed
  550f395

  Copy full SHA for 550f395

Commits on Nov 6, 2024

• Merge from master.

  LswaN58

  committed
  03bb434

  Copy full SHA for 03bb434

Commits on Nov 9, 2024

• Had a couple imports referring to core.

  LswaN58

  committed
  ae1c9ab

  Copy full SHA for ae1c9ab

• Update other places where we had imports pointing to core.

  LswaN58

  committed
  3388e48

  Copy full SHA for 3388e48

Commits on Nov 21, 2024

• Work in progress for the handling of date and version filters when getting list of IDs.

  LswaN58

  committed
  be70a62

  Copy full SHA for be70a62

• Merge branch 'main' into issue/3-refactor-interface-classes

  LswaN58

  committed
  ec7825b

  Copy full SHA for ec7825b

Commits on Nov 25, 2024

• Add an init file for the filter collections.

  LswaN58

  committed
  2ccb4a6

  Copy full SHA for 2ccb4a6

• Add an enum for the types of versions that exist.

  LswaN58

  committed
  59bdcc8

  Copy full SHA for 59bdcc8

Commits on Nov 26, 2024

• Add a ResourceName property, which for now at least is just the name attached to GameSourceSchema.

  LswaN58

  committed
  14190fb

  Copy full SHA for 14190fb

• Update the AvailableX functions in Interface base class.

  LswaN58

  committed
  dc7c014

  Copy full SHA for dc7c014

• When getting available versions/dates/ids, log the fact that we're doing it at INFO level.

  LswaN58

  committed
  e88d6ab

  Copy full SHA for e88d6ab

• Add basic versions of the GetXCollection functions.

  LswaN58

  committed
  a1b9b42

  Copy full SHA for a1b9b42

• Merge branch 'main' into issue/3-refactor-interface-classes

  LswaN58

  committed
  c950bf6

  Copy full SHA for c950bf6

• Add table schema param to the GetXCollection functions, and default filters.

  LswaN58

  committed
  10697d9

  Copy full SHA for 10697d9

• Remove EventInterface class, everything will just inherit from interface.

  LswaN58

  committed
  9f176d9

  Copy full SHA for 9f176d9

• Oops, AsDict wasn't marked as a property.

  LswaN58

  committed
  8b9742f

  Copy full SHA for 8b9742f

• Also, need to hint return type of _asDict.

  LswaN58

  committed
  f88735f

  Copy full SHA for f88735f

• Add a class for keeping track of an event dataset.

  LswaN58

  committed
  70fa0db

  Copy full SHA for 70fa0db

• Add a 'Dataset' class for features.

  LswaN58

  committed
  3908e3d

  Copy full SHA for 3908e3d

• Create placeholder FeatureTableSchema class.

  LswaN58

  committed
  6b9cbb1

  Copy full SHA for 6b9cbb1

• Remove unused import.

  LswaN58

  committed
  6cbd4f8

  Copy full SHA for 6cbd4f8

• Fix indentation in FeatureData.

  LswaN58

  committed
  36164f6

  Copy full SHA for 36164f6

• Update types of functions to return datasets.

  LswaN58

  committed
  02f0099

  Copy full SHA for 02f0099

• Actually, separate out some of the data retrieval and filter summarizing logic.

  LswaN58

  committed
  5c46e40

  Copy full SHA for 5c46e40

• Fix some imports in MySQLInterface.

  LswaN58

  committed
  bd064bf

  Copy full SHA for bd064bf

• Merge branch 'main' into issue/3-refactor-interface-classes

  LswaN58

  committed
  c06c5cb

  Copy full SHA for c06c5cb

• Fix imports in other interfaces.

  LswaN58

  committed
  dea4465

  Copy full SHA for dea4465

• GameSourceSchema should explicitly store a game ID, just for simplicity.

  LswaN58

  committed
  b8465ad

  Copy full SHA for b8465ad

• Bring back some logic from original interface implementation for converting row data to events.

  LswaN58

  committed
  486f45c

  Copy full SHA for 486f45c

• Add placeholder logic for equivalent conversion stuff for features.

  LswaN58

  committed
  dc835d3

  Copy full SHA for dc835d3

• Add prop for the game source schema to storage connector.

  LswaN58

  committed
  7a2e081

  Copy full SHA for 7a2e081

• Move super init to end of init function.

  LswaN58

  committed
  5c8f3b4

  Copy full SHA for 5c8f3b4

• Move super init to end of init function for GameSourceSchema as well.

  LswaN58

  committed
  08e94b5

  Copy full SHA for 08e94b5

• Switch logic to rely on TableSchema from the GameSourceSchema.

  LswaN58

  committed
  153cf66

  Copy full SHA for 153cf66

• Various small adjustments to sort out the split.

  LswaN58

  committed
  e4864b7

  Copy full SHA for e4864b7

• Better handling of types in _parse function.

  LswaN58

  committed
  693265f

  Copy full SHA for 693265f

• Add logic to handle additional time formatting.

  LswaN58

  committed
  72e4409

  Copy full SHA for 72e4409

• Nicer type-hint for indices param.

  LswaN58

  committed
  a64880e

  Copy full SHA for a64880e

• Add an enum to distinguish between event and feature tables.

  LswaN58

  committed
  b5cdd66

  Copy full SHA for b5cdd66

• Add a FromString to the TableType.

  LswaN58

  committed
  e01c688

  Copy full SHA for e01c688

Commits on Nov 27, 2024

• Think I've got a workable solution to splitting classes, now.

  LswaN58

  committed
  81b0743

  Copy full SHA for 81b0743

• Tweak some imports and type hints.

  LswaN58

  committed
  56b61a8

  Copy full SHA for 56b61a8

• Move a bunch of the conversion code out to a class in utils, clearer separation of logic that way.

  LswaN58

  committed
  fd3f41b

  Copy full SHA for fd3f41b

• Fix a missing import and some uses of old variable in debug prints.

  LswaN58

  committed
  434e271

  Copy full SHA for 434e271

• Move the logic for column map markdown to TableSchema, since it'll no longer be separate class.

  LswaN58

  committed
  91c954e

  Copy full SHA for 91c954e

• A few more organization improvements for markdown generation.

  LswaN58

  committed
  d6bb467

  Copy full SHA for d6bb467

• Remove duplicate declaration of abstract AsMarkdown prop.

  LswaN58

  committed
  6013478

  Copy full SHA for 6013478

• Pull in the props from ColumnMapSchema.

  LswaN58

  committed
  9508a74

  Copy full SHA for 9508a74

• Pass along TableSchema name to the subclass _fromDict.

  LswaN58

  committed
  2c38029

  Copy full SHA for 2c38029

• Add missing import, and put in header for _fromDict, to be filled in shortly.

  LswaN58

  committed
  ecaafaf

  Copy full SHA for ecaafaf

• Add placeholder body to _fromDict function.

  LswaN58

  committed
  badeb56

  Copy full SHA for badeb56

• Fix and improve debug outputs for EventTableSchema RowToEvent function.

  LswaN58

  committed
  4e37671

  Copy full SHA for 4e37671

• Make _conversion_warnings into a counter, that way we could theoretically print the counts after an export as part of debug outputs.

  LswaN58

  committed
  f59605c

  Copy full SHA for f59605c

• Remove unused import.

  LswaN58

  committed
  5db2a9d

  Copy full SHA for 5db2a9d

• Remove unused import.

  LswaN58

  committed
  47092dc

  Copy full SHA for 47092dc

• Add optional logger for _fromDict, since we'll be copying from the ColumnMapSchema FromDict.

  LswaN58

  committed
  1c8e1aa

  Copy full SHA for 1c8e1aa

• Fill in _fromDict body from what was in ColumnMapSchema.

  LswaN58

  committed
  d27d954

  Copy full SHA for d27d954

• Handle leftover elements.

  LswaN58

  committed
  ce7a2ce

  Copy full SHA for ce7a2ce

• Fix some imports and param names in testbed for GameSourceSchema.

  LswaN58

  committed
  492787d

  Copy full SHA for 492787d

• Update imports in some of the storage schemas.

  LswaN58

  committed
  394cdb9

  Copy full SHA for 394cdb9

• Make init param more in line with conventions.

  LswaN58

  committed
  7492c75

  Copy full SHA for 7492c75

• Move _parseElement to TableSchema base class.

  LswaN58

  committed
  c3910ba

  Copy full SHA for c3910ba

• Add comment block for _parseElement, along with a reminder to pick a more descriptive name.

  LswaN58

  committed
  2cb44ff

  Copy full SHA for 2cb44ff

• Fix up a lot of the FeatureTableSchema code. Still mostly a copy of EventTableSchema.

  LswaN58

  committed
  caab915

  Copy full SHA for caab915

Commits on Nov 28, 2024

• In GameSourceSchema, rename TableSchema prop -> TableSchemaName

  Show description for 4eb8195
  LswaN58

  committed
  4eb8195

  Copy full SHA for 4eb8195

• We're just integrating the logic of ColumnMapSchema back into TableSchema, so no need for the original file anymore.

  LswaN58

  committed
  6b86ac6

  Copy full SHA for 6b86ac6

• Some very preliminary tweaks to MySQLInterface, working towards the 'refactor' part of this project.

  LswaN58

  committed
  7c2ccaa

  Copy full SHA for 7c2ccaa

• Merge branch 'main' into issue/3-refactor-interface-classes

  LswaN58

  committed
  36555d3

  Copy full SHA for 36555d3

Commits on Nov 29, 2024

• Merge branch 'main' into issue/3-refactor-interface-classes

  LswaN58

  committed
  4fa59be

  Copy full SHA for 4fa59be

Commits on Nov 30, 2024

• Merge branch 'main' into issue/3-refactor-interface-classes

  LswaN58

  committed
  5011518

  Copy full SHA for 5011518

• Fix for name of parameter to GameSourceSchema class.

  LswaN58

  committed
  b7f99b7

  Copy full SHA for b7f99b7

• Fix GameSourceSchema needing to parse out game ID.

  LswaN58

  committed
  e8d59dd

  Copy full SHA for e8d59dd

• Fix import paths, which were presumably overwritten by merged changes or something.

  LswaN58

  committed
  da465bd

  Copy full SHA for da465bd

• Resolve type error message in GameSourceSchema testbed.

  LswaN58

  committed
  e1ab2c6

  Copy full SHA for e1ab2c6

• Add property for accessing the TableSchema of GameSourceSchema.

  LswaN58

  committed
  45a6dc8

  Copy full SHA for 45a6dc8

• Further WIP changes to get this up to new Interface function set.

  LswaN58

  committed
  edab15f

  Copy full SHA for edab15f

• Start outlining a class for getting SQL from filters.

  LswaN58

  committed
  d7fa2f5

  Copy full SHA for d7fa2f5

Commits on Dec 1, 2024

• Merge branch 'main' into issue/3-refactor-interface-classes

  LswaN58

  committed
  e5aac5a

  Copy full SHA for e5aac5a

Commits on Dec 4, 2024

• Move common XIndex and XColumn props up to TableSchema base class.

  LswaN58

  committed
  d6a7a65

  Copy full SHA for d6a7a65

Commits on Dec 5, 2024

• Don't specify a type for contents of the params Tuple.

  LswaN58

  committed
  7e3ad08

  Copy full SHA for 7e3ad08

• Finish implementation of filter converter function

  LswaN58

  committed
  828ecd6

  Copy full SHA for 828ecd6

Commits on Dec 9, 2024

• Start converting old ColumnMapSchema over to newer ElementMappingSchema approach.

  LswaN58

  committed
  f9227e0

  Copy full SHA for f9227e0

• Add an enum for type of ElementMapping, i.e. single, list or dict.

  LswaN58

  committed
  5ee1444

  Copy full SHA for 5ee1444

Commits on Dec 17, 2024

• Move configs subfolder of schemas out to be own folder.

  LswaN58

  committed
  73a5b92

  Copy full SHA for 73a5b92

• Move scemas/storage to configs, and rename DataSourceSchema -> DataStoreSchema.

  LswaN58

  committed
  f90ac8f

  Copy full SHA for f90ac8f

• Move feature-related schemas to configs, since they exist mostly for config.

  LswaN58

  committed
  19fc955

  Copy full SHA for 19fc955

• Add a stupidly thin wrapper around Schema, named Config, as base for the other config classes.

  LswaN58

  committed
  b20bc40

  Copy full SHA for b20bc40

• Rename TestConfigSchema -> TestConfig.

  LswaN58

  committed
  4340465

  Copy full SHA for 4340465

• Missed changes in last commit.

  LswaN58

  committed
  e83e3e4

  Copy full SHA for e83e3e4

• Rename IndexingSchema -> IndexingConfig.

  LswaN58

  committed
  3f41bdb

  Copy full SHA for 3f41bdb

• Rename DataStoreSchema -> DataStoreConfig.

  LswaN58

  committed
  3df0459

  Copy full SHA for 3df0459

• Move CredentialSchema to credentials folder, and rename to CredentialConfig.

  LswaN58

  committed
  1696caf

  Copy full SHA for 1696caf

• Missed changes in last commit.

  LswaN58

  committed
  16fe01c

  Copy full SHA for 16fe01c

• Rename unparsed_elements -> other_elements, to match convention.

  LswaN58

  committed
  8788698

  Copy full SHA for 8788698

• Create a config class for storing password credential.

  LswaN58

  committed
  33fc6cb

  Copy full SHA for 33fc6cb

• Add a class for handling key credentials.

  LswaN58

  committed
  37353f3

  Copy full SHA for 37353f3

• Fix import paths for DataStoreConfig.

  LswaN58

  committed
  7271648

  Copy full SHA for 7271648

• Rename FileSourceSchema -> FileStoreConfig.py

  LswaN58

  committed
  ab49b82

  Copy full SHA for ab49b82

• Update all the DataStoreConfig subclasses to use DataStoreConfig instead of DataSourceSchema as parent.

  LswaN58

  committed
  8c96390

  Copy full SHA for 8c96390

• Rename MySQLSourceSchema -> MySQLConfig, similar for SSHSchema.

  LswaN58

  committed
  a2a7465

  Copy full SHA for a2a7465

• Replace other uses of DataSourceSchema.

  LswaN58

  committed
  c9b060f

  Copy full SHA for c9b060f

• Update all uses of BigQuerySchema to BigQueryConfig.

  LswaN58

  committed
  fd797e8

  Copy full SHA for fd797e8

• Various places where MySQLSchema needed to be replaced by MySQLConfig.

  LswaN58

  committed
  1c8a4cc

  Copy full SHA for 1c8a4cc

• Rename generator-related schema files to configs.

  LswaN58

  committed
  f96721c

  Copy full SHA for f96721c

Commits on Dec 19, 2024

• Rename most of the feature-related schemas to config.

  LswaN58

  committed
  e2fe9d2

  Copy full SHA for e2fe9d2

• Search-and-replace occurrences of most of the game schemas to configs.

  LswaN58

  committed
  444f580

  Copy full SHA for 444f580

• Do search-and-replace on more schemas-turned-configs.

  LswaN58

  committed
  92380df

  Copy full SHA for 92380df

• Update a bunch of import paths for games configs, and regular configs.

  LswaN58

  committed
  4ff4560

  Copy full SHA for 4ff4560

• Update names used for IndexingConfig and TestConfig.

  LswaN58

  committed
  b1b72b7

  Copy full SHA for b1b72b7

• After we split out a bunch of classes to configs, do the same for the corresponding testbeds.

  LswaN58

  committed
  4976aa8

  Copy full SHA for 4976aa8

• Add an init for the interfaces test cases folder.

  LswaN58

  committed
  b2a4ef2

  Copy full SHA for b2a4ef2

• Start work to update the GH workflows based on new naming.

  LswaN58

  committed
  0526147

  Copy full SHA for 0526147

• Fix a couple bad imports I missed earlier.

  LswaN58

  committed
  c9dfc00

  Copy full SHA for c9dfc00

• Work on renaming the actual files for our testbeds.

  LswaN58

  committed
  a90db45

  Copy full SHA for a90db45

• Fix a couple more imports.

  LswaN58

  committed
  6401838

  Copy full SHA for 6401838

• Update workflows with new filenames, and split out GameConfigs from GameSchemas.

  LswaN58

  committed
  571cce7

  Copy full SHA for 571cce7

• Update config and TestDriver with new paths.

  LswaN58

  committed
  00f2390

  Copy full SHA for 00f2390

• Remove non-existant old classes from schemas/games/__init__.py.

  LswaN58

  committed
  ece4b65

  Copy full SHA for ece4b65

• Add an init file for the game configs.

  LswaN58

  committed
  2aae1de

  Copy full SHA for 2aae1de

• Add a couple more init files, and fix naming of CredentialSchema -> CredentialConfig.

  LswaN58

  committed
  8e60b9d

  Copy full SHA for 8e60b9d

• Update test config action to include option for enabling configs tests.

  LswaN58

  committed
  c8e5106

  Copy full SHA for c8e5106

• Update a couple parts of GameConfigs workflow to use 'configs' instead of 'schemas'.

  LswaN58

  committed
  e164388

  Copy full SHA for e164388

• Update CI script to include all the individual test workflows.

  LswaN58

  committed
  30ddec9

  Copy full SHA for 30ddec9

• Add init to game configs folder.

  LswaN58

  committed
  c921425

  Copy full SHA for c921425

• Merge pull request #67 from opengamedata/issue/64-separate-configs

  Show description for ea33c8d
  LswaN58

  authored
  ea33c8d

  Copy full SHA for ea33c8d

• On second thought, don't even make Config a subclass of Schema, make it a straight-up type alias instead.

  LswaN58

  committed
  533e296

  Copy full SHA for 533e296