New `Piece` object, new commands

• the View on a Parse object can now be subscripted with a filename to obtain a Piece object, allowing
  for better access to the various files belonging to the same piece (based on their file names). These new objects
  facilitate access to the information which previously was available in one row of tge View.pieces() DataFrame.
• adds command ms3 empty to remove harmony annotations from scores
• adds command ms3 add to add harmony annotations from TSV files to scores
• re-factored ms3 compare to use new methods added to View objects
• methods based on View.iter() now accept the parameter fnames to filter out file names not included in the list
• while adding labels, use fallback values staff=-1 and voice=1 if not specified

Leveraging View.iter() & extended metadata

• changes to iter methods for iterating through DataFrames and metadata belonging together:
  • supressed the second item: instead of (metadata, paths, df1, df2...) yield (metadata, df1, df2...) where the
    metadata dict contains the paths
  • added methods iter_transformed() and iter_notes() to Parse and View objects
• added command ms3 transform
  • used to concatenate all parsed TSVs of a certain type into one file including the option to unfold and add quarterbeats
  • stores them with prefix concatenated_; ms3 now ignores all files beginning with this prefix
• changes in default TSV columns
  • metadata.tsv includes the new columns
    • length_qb: a scores length in quarterbeats (including all voltas)
    • length_qb_unfolded: the same but with unfolded repeats, if any
    • all_notes_qb: the sum of all note durations in quarterbeats
    • n_onsets: the number of all onsets
  • no empty volta columns are included (except for measures) when no voltas are present

Towards a new behaviour

This release makes the first transition from 0.4 to 0.5. It is a first attempt at a better way of dealing with corpora that adhere to a standardized structure (the one generated when using the default paths of the command ms3 extract or the method Parse.store_lists). If you find bugs, please report them, this would be much appreciated!

Changelog

• considerable changes to Parse objects (bugs might still be abundant, please report them)
  • abolished custom DataFrame indices
  • behaviour shaped towards ms3's standard corpus structure
    • automatic detection of corpora and generation of keys
    • this enables better matching of files that belong together through View objects (access via p['key'])
    • new method iter() for iterating through metadata and files that belong together
  • all JSON files passed under the paths argument are now scanned for a contained list of file paths to be extracted
    (as opposed to before where the JSON file had to be passed as a single path)
  • new iterator p.annotation_objects()
• new module transformations
  • just as utils, members can be imported directly via from ms3 import
  • includes a couple of functions that were previously part of utils or expand_dcml
  • includes a couple of new functions:
    • get_chord_sequences()
    • group_annotations_by_features()
    • make_gantt_data()
    • transform_annotations()
    • transform_multiple()
• handling hierarchical localkeys and pedals (i.e. we can modulate to the key of V/III)
• Renamed column 'durations_quarterbeats' to 'duration_qb'
• You can now set interval_index = True to add quarterbeat columns and an index with quarterbeat intervals
• New behaviour of the folder_re argument: It now gets to all paths matching the regEx rather than stopping at a
  higher level that doesn't match. Effectively, this allows, for example, to do Parse(path, folder_re='notes') to
  select all files from folders called notes.
• bug fixes (e.g. failing less on incoherent repeat structures)

Workflow version

A post-hoc release to give a version number to the commit that has been being checked out for the DCML annotation workflow since early 2021.

Form labels

• Made the use of labels_cfg more consistent.
• improved chord lists:
  • include system and tempo texts
  • new algorithm for correct spanner IDs (i.e. for Slurs, Pedal, HairPins, Ottava)
  • lyrics: still extracts only the last verse but now in the corresponding column, e.g. lyrics:3 for verse 3.
• new feature (still in beta): extraction of form labels
  • Score.mscx.form_labels
  • Parse.form_labels()
  • added form_labels -related parameters to Parse.get_lists() and Parse.store_lists()
  • added utils.expand_form_labels() for hierarchical display of form labels

Improved stability, many new features

• enabled import from ms3 for all utils
• new command ms3 update for converting files and moving annotations to the Roman Numeral Analysis layer
• new command ms3 metadata for writing manually changed information from metadata.tsv to the metadata fields of the corresponding MuseScore files
• improved the ms3 extract command:
  • added option -D for extracting and updating metadata.tsv and README.md
  • added option -q for adding 'quarterbeats' and 'durations_quarterbeats' columns
  • included default paths for the capital-letter parameters
• improved the ms3 compare command:
  • now works with 'expanded' TSVs, too (not only with 'labels')
  • allows 'label' column to include NaN values
• improvements to Parse() objects:
  • attempts to parse scores that need file conversion (e.g. XML, MIDI)
  • get_lists() method now allows for adding the columns quarterbeats and durations_quarterbeats, even without unfolding repeats
  • adding 'quarterbeats' without unfolding repeats excludes voltas
  • new method get_tsvs() for retrieving and concatenating parsed TSV files
  • Parse() now recognizes metadata.tsv files, expanded TSVs, and TSVs containing cadence labels only
  • parsed metadata.tsv files can be retrieved/included via the method metadata()
  • new method update_metadata() for the new ms3 metadata command
  • decided on standard index levels rel_paths and fnames
  • improved matching of corresponding score and TSV files
• improvements to Score() objects:
  • new property Score.mscx.volta_structure for retrieving information on first and second endings
• improvements to Annotations() objects:
  • correct propagation of localkey for voltas
• improvements to commandline interface:
  • added parameter -o for specifying output directory
  • harmonized the interface of the ms3 convert command
  • parameter exclude_re now also filters paths passed via -f
• changed logging behaviours:
  • write only WARNINGs to log file
  • combine loggers for filenames independently of file extensions
• improved extraction of instrument names for metadata
• improved ms3 compare functionality
• restructured code architecture
• renamed master branch to 'main'
• many bug fixes

Improvements for better GitHub workflows

• ms3 now reads DCML labels with cadence annotations
• unified command-line interface file options and included -f file.json
• Parse got more options for creating DataFrame index levels
• Parse.measures property for convenience
• bug fixes for better GitHub workflows

Moving towards the RNA layer

• Labels can be attached to MuseScore's Roman Numeral Analysis (RNA) layer
  • parameter label_type=1 in both Score.attach_labels() and Parse.attach_labels()
  • Annotations.remove_initial_dots() before inserting into the RNA layer
  • Annotations.add_initial_dots() before inserting into the absolute chord layer
• interpret all #vii in major contexts as vii when computing chord tones
• code cosmetics and bug fixes

Unfolding repeats

• ms3 extract and Parse.store_lists() now have the option unfold to account for repeats
• minor bug fixes

Temporary Conversions

• added 'ms3 compare' command
• support for parsing cap, capx, midi, musicxml, mxl, and xml files through temporary conversion
• support for parsing MuseScore 2 files through temporary conversion