Code

Add context storage benchmarking#144

Code

Merged

RLKRo merged 119 commits intodevdeeppavlov/chatsky:devfrom feat/db-benchmarkdeeppavlov/chatsky:feat/db-benchmarkCopy head branch name to clipboardSep 28, 2023

+1,976-3

Commits

Commits on Jun 7, 2023

• add context storage benchmarking

  RLKRo

  committed
  a870dd2

• add all dff.utils modules to doc

  RLKRo

  committed
  afcdce9

• fix doc

  RLKRo

  committed
  365e996

• add support for benchmarking multiple context storages at a time

  RLKRo

  committed
  4d6a23f

Commits on Jun 8, 2023

• add type annotations; option to pass multiple context storages; export as pdf

  RLKRo

  committed
  5815511

• add option to get results as a dataframe

  RLKRo

  committed
  17b11fc

• format

  RLKRo

  committed
  2c83245

Commits on Jun 9, 2023

• add tutorial for benchmark

  RLKRo

  committed
  e8564b1

• update benchmark dependencies

  RLKRo

  committed
  a07588c

Commits on Jun 22, 2023

• update benchmark utils

  Show description for 258be60
  RLKRo

  committed
  258be60

• add benchmark_dbs and benchmark_streamlit

  RLKRo

  committed
  b970e83

• update dependencies

  RLKRo

  committed
  3c42227

• use python3.8 compatible typing

  RLKRo

  committed
  15795df

• return ydb & reorder benchmark sets

  RLKRo

  committed
  13cc2d7

• add more benchmark cases

  RLKRo

  committed
  25c5b7e

• improve diff viewing

  RLKRo

  committed
  2df1054

• reduce dialog len for extreme cases

  RLKRo

  committed
  cd0eabf

• change benchmark format

  RLKRo

  committed
  185d950

• change benchmark format: generic factory

  RLKRo

  committed
  c16cd28

• bugfix: repeated format update

  RLKRo

  committed
  5de5a83

• move generic benchmark tools to utils

  RLKRo

  committed
  8fc62a8

• add average read+update column

  RLKRo

  committed
  1f450e5

• add mass compare tab

  RLKRo

  committed
  4404ef3

• update extreme cases params

  RLKRo

  committed
  5fb2e69

• set step_dialog_len to 1 by default

  RLKRo

  committed
  4c51473

Commits on Jun 27, 2023

• print exception message during benchmark

  RLKRo

  committed
  0a35013

• store read times under supposed dialog len

  RLKRo

  committed
  b6a38a1

• set streamlit version in dependencies

  Show description for eda7ccc
  RLKRo

  committed
  eda7ccc

• add exist_ok flag for saving to file

  RLKRo

  committed
  60bf152

• move average results calculations from streamlit to benchmark utils

  Show description for ec19a30
  RLKRo

  committed
  ec19a30

• rename lengths to dimensions

  RLKRo

  committed
  8abc60f

• remove context checking comments

  RLKRo

  committed
  c65ca37

• move partial benchmarking logic to partial benchmark file

  RLKRo

  committed
  fb999e3

• add partial file saving

  RLKRo

  committed
  e413025

• update report function

  Show description for 827861b
  RLKRo

  committed
  827861b

• add BenchmarkConfig class to avoid repetition of parameters

  RLKRo

  committed
  a25ee56

Commits on Jun 28, 2023

• revert add partial file saving

  RLKRo

  committed
  0424b43

• fix benchmark name

  RLKRo

  committed
  e6b88b7

• update streamlit for new benchmark_config

  RLKRo

  committed
  f68d71c

• update benchmark_new_format.py

  RLKRo

  committed
  0fe49a4

• fix bug with delisting benchmarks

  RLKRo

  committed
  9e8f310

• not include zero-point in graphs

  RLKRo

  committed
  7494772

Commits on Jun 29, 2023

• move get_context_updater to BenchmarkConfig

  RLKRo

  committed
  99a4e36

• add benchmark_dir variable for benchmark_dbs.py

  RLKRo

  committed
  f787577

• add get_context method to BenchmarkConfig

  RLKRo

  committed
  1dbb7f9

• change update benchmarking logic

  Show description for 8e2ab54
  RLKRo

  committed
  8e2ab54

Commits on Jul 2, 2023

• return empty dicts as update_times if context_updater is None

  RLKRo

  committed
  8741334

• remove write_times from average_results

  RLKRo

  committed
  5a4eb75

• add support for no update times

  RLKRo

  committed
  a4e71c7

• group sizes stat

  RLKRo

  committed
  d7cb290

Commits on Jul 3, 2023

• add benchmark tests

  RLKRo

  committed
  d3d66c0

• Merge branch 'dev' into feat/db-benchmark

  Show description for 9ee9e45
  RLKRo

  committed
  9ee9e45

• clear context storage at the end of each context_num cycle

  RLKRo

  committed
  e995195

Commits on Jul 4, 2023

• save benchmarks in a list

  RLKRo

  committed
  a92ca57

Commits on Jul 5, 2023

• add json schema for benchmark results

  RLKRo

  committed
  370d138

• move streamlit to utils

  RLKRo

  committed
  435af6e

• refactor benchmark streamlit

  RLKRo

  committed
  40a053c

• remove partial-dev comparison tools

  Show description for f8ecda9
  RLKRo

  committed
  f8ecda9

Commits on Jul 6, 2023

• add doc & update tutorial

  RLKRo

  committed
  32f5dd8

• move benchmark configs to utils

  RLKRo

  committed
  d9a161d

• remove partial from benchmark_dbs.py

  RLKRo

  committed
  f52931a

• remove format updater for benchmark files

  RLKRo

  committed
  41ea85c

• fix mass compare bug

  RLKRo

  committed
  465f41b

• lint & format

  RLKRo

  committed
  958db10

• add utils to backup_files for test_coverage

  RLKRo

  committed
  6d55a3f

• skip benchmark tests if benchmark not installed

  RLKRo

  committed
  a9402ee

• Revert "remove format updater for benchmark files"

  Show description for 7b5c502
  RLKRo

  committed
  7b5c502

• move format updater to utils

  RLKRo

  committed
  5e98c0e

• move format updating to a separate function

  RLKRo

  committed
  a9a7af2

• remove old format support from format updater

  RLKRo

  committed
  266e280

• use format updater in streamlit

  RLKRo

  committed
  d18025c

• add ability to upload files to streamlit, add all files from one directory

  RLKRo

  committed
  d9cbca5

Commits on Jul 10, 2023

• store benchmarks for a specific db in one file

  Show description for 65ef6a2
  RLKRo

  committed
  65ef6a2

• change report function, update tutorial

  Show description for 97002c8
  RLKRo

  committed
  97002c8

• remove literal include of the files

  RLKRo

  committed
  99ee6a2

• format

  RLKRo

  committed
  791be50

Commits on Jul 11, 2023

• add ability to edit name and description of benchmark sets

  RLKRo

  committed
  7c5f7ed

• add help for displayed metrics

  RLKRo

  committed
  2d7432b

Commits on Jul 13, 2023

• reformat

  RLKRo

  committed
  10e6f29

Commits on Jul 18, 2023

• preserve file order when delisting benchmarks

  RLKRo

  committed
  8f23fd5

Commits on Aug 16, 2023

• Merge branch 'dev' into feat/db-benchmark

  Show description for 5c04f6e
  RLKRo

  committed
  5c04f6e

• remove typing as tp

  RLKRo

  committed
  2b2c2bb

• add type annotations

  RLKRo

  committed
  acb0557

• move model configuration to kwargs

  RLKRo

  committed
  fbb3a5f

• change misc key type to str

  RLKRo

  committed
  bdc6277

Commits on Aug 17, 2023

• accept context factory for benchmark instead of context

  RLKRo

  committed
  a36ddd3

• add type hints for test cases

  RLKRo

  committed
  5d671a0

• uncomment dbs in benchmark_dbs.py

  RLKRo

  committed
  819dcae

• add an explanation in tutorial

  RLKRo

  committed
  8e53b2e

• remove benchmark_new_format.py

  RLKRo

  committed
  e50158c

• add info messages

  RLKRo

  committed
  55fdece

• randomize strings returned by get_dict

  RLKRo

  committed
  62a33d9

• add comments

  RLKRo

  committed
  c1f6fbe

• rename benchmark.context_storage to db_benchmark.benchmark

  RLKRo

  committed
  837e285

• move report function to a separate module

  RLKRo

  committed
  85df6d1

• import Path instead of pathlib

  RLKRo

  committed
  7054c1b

• fix doc

  RLKRo

  committed
  9cf1ef0

• reformat

  RLKRo

  committed
  7584902

• add imports to __init__.py

  RLKRo

  committed
  aa65bc7

• minor report change

  RLKRo

  committed
  25eabff

• replace deprecated method call

  RLKRo

  committed
  e928397

• rename context vars

  RLKRo

  committed
  72e991b

Commits on Aug 18, 2023

• generalize BenchmarkConfig

  RLKRo

  committed
  87b1820

• add .dockerignore && add benchmark files to ignore

  RLKRo

  committed
  2420455

• reformat

  RLKRo

  committed
  988f0ac

• move databases inside the benchmark_dir

  RLKRo

  committed
  4263b7e

• fix doc

  RLKRo

  committed
  ebcbc07

• Merge branch 'dev' into feat/db-benchmark

  Show description for 8968b28
  RLKRo

  committed
  8968b28

• use tutorial directives

  RLKRo

  committed
  5e604bd

Commits on Aug 22, 2023

• Merge branch 'dev' into feat/db-benchmark

  RLKRo

  committed
  0fd1fc6

Commits on Aug 24, 2023

• remove ability to add files from filesystem

  Show description for 3d54611
  RLKRo

  committed
  3d54611

• delete files from filesystem when sets are deleted via the interface

  Show description for e422621
  RLKRo

  committed
  e422621

• link files referenced in the documentation to docs/source

  RLKRo

  committed
  ec80dc3

• add dependency info for streamlit app

  RLKRo

  committed
  e441da0

• add streamlit screenshots inside the tutorial

  RLKRo

  committed
  512ed66

Commits on Sep 11, 2023

• Merge branch 'dev' into feat/db-benchmark

  Show description for 378d87e
  RLKRo

  committed
  378d87e

• reupload images

  RLKRo

  committed
  bb0e86b

• add more exception info

  RLKRo

  committed
  8c419b5

Commits on Sep 28, 2023

• Merge branch 'dev' into feat/db-benchmark

  Show description for 093fa52
  RLKRo

  committed
  093fa52