Release note for v0.4.0

[heronshoes]

RedAmber v0.4.0 has released 🎉

Upgraded to Arrow 11.0.0/Bumped RedAmber version 0.4.0

Upgraded dependency of Apache Arrow/Apache Arrow GLib to 11.0.0 (https://rubygems.org/gems/red-arrow).

Next version was planned to be 0.3.1, but there are so many changes from 0.3.0 and supported Arrow 11.0.0, so I decided to bump the version up to 0.4.0 .

Completed YARD documentation!

• Moved define_method for Vector functions to class method. This enabled YARD documentation for dynamically defined methods.
• Introduced custom style sheet for YARD to use monospace font in code block example.

New class SubFrames

I introduced the new class SubFrames(concept_en, concept_jp). SubFrames is subsets of parent dataframe collected by some rule and it will cover existing Group and incoming window feature. This feature is experimental and may be slow, but it is very powerful. In v0.4.0 we have;

• Many constructors
  • Create SubFrames from indices or filters, by arguments or blocks.
  • Create SubFrames by Group object
  • Create SubFrames from keys and group by values in columns specified by the key.
  • Create SubFrames by window in size rolling from by step.
  • Create SubFrames by Grouping/Windowing by position.
    The position is specified by Array's enumerator method such as each_slice or each_cons.
  • Create SubFrames by windowing with a kernel and step.
    Kernel is a boolean Array and it behaves like a masked window.
  • Generic builder of sub-dataframe from self.
  • Create SubFrames from an Array of DataFrames.
• Update values
  • SubFrames#assign
    update values/create new column for each sub-dataframes.
    By the same manner as DataFrame#assign !
• Iterator
  • SubFrames#each
    returns Enumerator or iterate each sub DataFrames.
  • SubFrames#map, #select, #reject, #, #filter_map
    returns SubFrames
  • Other iterators from Enumerable returns Array (such as #grep).
• Summarize/Concatenate to create a DataFrame
  • SubFrames#aggregate(group_keys, aggregations)
  • SubFrames#concatenate
  • SUbFrames#find

Stats from 0.3.0 to 0.4.0

• 58 files changed, 9057 insertions(+), 1849 deletions(-)
• 519 tests, 1424 assertions (0.2.3) =>
  605 tests, 1651 assertions (0.3.0) =>
  721 tests, 1897 assertions (0.4.0)
• Test coverage 100%
• Document coverage 100%

Changelog of this release is below. Your help is welcome!

[0.4.0] - 2023-02-25

• Breaking change

  • Upgrade dependency to Arrow 11.0.0 (Upgrade Arrow version to 11.0.0 #188)

• Bug fixes

  • Add :force_order option for DataFrame#join (Fix failures in tests and linters #174)
  • Return error for empty DataFrame in DataFrame#filter (Update DataFrame#filter #172)
  • Accept ChunkedArray in DataFrame#filter (Update DataFrame#filter #172)
  • Fix Vector#replace to accept Arrow::Array as a replacer (Update document #179)
  • Fix Vector#round_to_multiple to accept Float or Integer (Update document #180)
  • Change Vector atan2 to a class method (Update document #180)
  • Fix Vector#shift when boolean Vector (Miscellaneous Improvements #184)
  • Fix processing empty SubFrames (Introduce sub frames #183)
  • Do not check object id in DataFrame#rename, #drop for self (Upgrade Arrow version to 11.0.0 #188)

• New features and improvements

  • Accept a block in DataFrame#filter (Update DataFrame#filter #172)
  • Add Vector.aggregate? method (Introduce Vector#propagate #175)
  • Introduce Vector#propagate method (Introduce Vector#propagate #175)
  • Add Vector#rank methods (Introduce Vector#sample #176)
  • Add Vector#sample method (Introduce Vector#sample #176)
  • Add Vector#sort method (Introduce Vector#sample #176)
  • Promote DataFrame#shape_str to public (Miscellaneous Improvements #184)
  • Introduce Vector#concatenate (Miscellaneous Improvements #184)
  • Add #numeric? in refinements of Array (Miscellaneous Improvements #184)
  • Add Vector#cumulative_sum_checked and #cumsum (Miscellaneous Improvements #184)
  • Add Vector#resolve method (Miscellaneous Improvements #184)
  • Add DataFrame#tdra method (Miscellaneous Improvements #184)
  • Add #expand as an alias for Vector#propagate (Miscellaneous Improvements #184)
  • Add #glimpse as an alias for DataFrame#tdr (Miscellaneous Improvements #184)
  • New class SubFrames (Introduce sub frames #183)
    • Introduce class SubFrames
    • Memorize dataframes in SubFrames
    • Add @Frames to memorize sub DataFrames
    • Accept filters in SubFrames.new
    • Accept block in SubFrames.new
    • Add SubFrames.by_filter
    • Introduce methods creating SubFrames from DataFrame
    • Introduce SubFrames#each method
    • Add SubFrames#to_s method
    • Add SubFrames#concatenate method
    • Add SubFrames#offset_indices method
    • SubFrames#aggregate method
    • Redefine SubFrames#map to return SubFrames
    • Define SubFrame#map dynamically
    • Add SubFrames#assign method
    • Redefine SubFrames#select to return SubFrames
    • Add SubFrames#reject method
    • Add SubFrames#filter_map method
    • Refine DataFrame#indices memorizing @indices
    • Rename SubFrames#universal_frame as #baseframe
    • Set Group iteration feature to @api private

• Refactoring

  • Generate Vector functions in class method (Generate Vector functions in class method #177)
  • Set Constant visibility to private (Update document #179)
  • Separate test_vector_function (Update document #179)
  • Relocate methods in DataFrameIndexable (Update document #179)
  • Rename Array refinements to the same name as Vector (Miscellaneous Improvements #184)

• Improve in tests/CI

  • Tests

    • Update benchmarks to set 0.3.0 as a reference (Update benchmarks to set 0.3.0 as reference #167)
    • Move test of Vector#logb to proper location (Update document #180)

  • Cops

    • Update .rubocop.yml to align with latest cops (Fix failures in tests and linters #174)
    • Unify style of MethodCallIndentation as relative to reciever (Miscellaneous Improvements #184)

  • CI

    • Fix setting up Arrow by homebrew in CI (Update benchmarks to set 0.3.0 as reference #167)
    • Fix CI error on homebrew deleting python link (Update benchmarks to set 0.3.0 as reference #167)
    • Set cache-version to get new C extensions in CI (Fix CI error of bundler cache #173) Thanks to @kou for suggestion.

• Documentation

  • Update DataFrame.md about loading csv without headers (Update Doc #165)
    • Thanks to kojix2
  • Update YARD in DataFrame combinable (Update YARD #168)
  • Update comment for Ruby 2.7 support in README.md
  • Update license year
  • Update README (Update DataFrame#filter #172)
  • Update Vector.md and yardoc in #propagate (Introduce Vector#propagate #175)
  • Use customized style sheet for YARD (Update document #179)
  • Add examples for the doc of #pick and #drop (Update document #179)
  • Add examples to YARD in DataFrame reshaping methods (Update document #179)
  • Update documents in DataFrameDisplayable (Update document #179)
  • Update documents in DataFrameVariableOperation (Update document #179)
  • Update document for dynamically generated methods (Update document #179)
  • Unify style in document (Update document #179)
  • Update documents in DataFrameSelectable (Update document #179)
  • Update documents of basic Vector methods (Update document #179)
  • Update document in VectorUpdatable (Update document #179)
  • Update document of Group (Update document #179)
  • Update document of DataFrameLoadSave (Update document #180)
  • Add examples for document of ArrowFunction (Update document #180)
  • Update document of Vector_unary_aggregation (Update document #180)
  • Update document of Vector_unary_element_wise (Update document #180)
  • Update document of Vector_biary_element_wise (Update document #180)
  • Add documentation to give comparison of dataframes(Add documentation to give comparison of dataframes #169)
    • Thanks to Benson Muite
  • Update documents for consistency of method indentation (Prepare 0.4.0 #189)
  • Update CHANGELOG (Prepare 0.4.0 #189)
  • Update README for 0.4.0 (Prepare 0.4.0 #189)

• GitHub site

• Thanks

  • kojix2
  • Benson Muite