- added explicit checks for all
dir.create()(closes #49) - checking dir parameter in
import_mediaFiles()(closes #108) add_files()now works with strings like_annot.jsonasfileExtension(closes #246)- now prechecking user defined options in
.onLoad()(closes #240) - added new
sessionPatternandbundlePatternargs tolist_bundles()andlist_sessions()(closes #201) - better output for warning in
get_trackdata()regarding heterogeneous sampling rates (closes #189) - using
clipackage to improve output ofsummary(emuDBhandle) - slight performance increase in EQL by avoiding full table scans due to REGEXP
- checking for repeating sequences in
normalize_length()(closes #211) - added explicit error message when bad column name is used in DBconfig (closes #212)
- removing linkDefinition on error if it was added be4 (closes #226)
- fixed bug in requery workflows like:
sl = query(ae, "[Phonetic== V]"); requery_hier(ae, sl[1:2,], level = "Text") - added missing loading of DBconfig to
write_bundleList()(fixes #248) - fixed bad handeling of dir paths containgin RegExp chars in
convert_TextGridCollection()(fixes #207)
- compete rewrite of hierarchical queries + requeries (now using CTEs) + clean-up of the EQL engine
serve()-> save bundle performance bump due to SQLite transaction improvement + using prep. statementsserve()function now downloads current release .zip instead of cloning repo to save time and bandwidth- fixed a few more
readr::file vs path parameter warnings - using
readr::write_lines()function inexport_TextGridCollection()to better support UTF-8 symbols - added
create_spectrogram_image_as_rasterfunction that used the EMU-webApp JavaScript code to draw a spectrogram as a raster
- fixed invalid TextGrid export on empty levels
load_emuDB()performance bump due to SQLite transaction improvement + using prep. statementscreate_itemsInLevel()now supports items of type SEGMENT- added deprecated/not recommended texts to \code{eplot()}, \code{dplot()} and \code{dur()} help pages (fixes #234)
- implemented first version of
delete_itemsInLevel() requery_seq()now returns correct attribute label vs. the level attribute- fixed
In rbind(names(params), unlist(params, recursive = F)): number of columns of result is not a multiple of vector length (arg 1)warning inrunBASwebservice_maus()withhttr::upload_file() - added missing
()toreturnstatement - no emuDBcache needed for
rename_emuDB()
- implemented
write_bundleList()andread_bundleList()functions - new
bundleListNameserve()parameter implemented - added warning to
serve()when either bundleComments or bundleFinishedEditing is set to true and no bundleListName was set (closes #268) - implemented
onTheFlyFunctionparameter forget_trackdata()which allows users to implement their own functions (input: path to wav file; output tibble/data.frame that has a column called"frame_time") - converting factors into characters in
get_trackdata()andnormalize_length()(closes #224 and #223) add_perspective()now sets"restrictions"->"showPerspectivesSidebar"totrueto make the side bar visiblenormalize_length()now doesn't throw tibble 3.0.0 warnings any morequery()RegEx operators=~now using SQLites native support (wasn't available in earlier versions)- replaced RCurl with httr dep
- moved some dependencies to
Suggestsfield in DESCRIPTION file - removed vignette stubs
- try catching rounding errors of sample rate and start time in
get_trackdata()
export_seglistToTxtCollection()now zero pads file names (should fix #219)- implemented
write_bundleList()andread_bundleList()functions - new
bundleListNameserve()parameter implemented
- using
rstudioapi::translateLocalUrl()for ws connection url to permit RStudio to connect - fixed bug in duplicate levels (bad level name insert into items table) that was causing the items array in the JSON file to stay empty
serve()working outside of RStudio again (overlooked thatrstudioapi::translateLocalUrl()needs RStudio)- improved error formatting in
load_emuDB()andexport_TextGridCollection - fixed
rename_bundles()issue with sub-string matching of names (fixes #220) requery_hier()works with non-main attributes again (always returned levels main attribute labels)
- changed default
resultTypesto"tibble"(query(),requery_hier(),requery_seq(),get_trackdata()) - changed url in printed output of serve to https
serve()doesn't block the R console any more- suppressing requery differing length warnings in
runBASwebservice_*functions requery_hier()now persists the input segment list length by inserting NA rows for missing segments- depricated vignettes are now stubs only (== empty containing notice pointing to the EMU-SDMS manual)
- implemented
rename_bundles()function - better
times_normcalculation fornormalize_length() - better error message when empty or non existing session is passed into
add_files() serve()useViewerparameter now implemented and the default.serve()now uses a single server to host the EMU-webApp and provide the websocket server for the emuDBserve()now works within RStudio when it is run as a web application
- rewrite of
list_files()that leads to massive speed bump - implemented
convert_wideToLong()function to convert trackdata tibble objects to their long form representation (useful for spectral analysis) query()sub-function now setsperl = TRUEarguments inregexpcall when usingsessionPatternandbundlePattern(useful for e.g. for negative look-aroundbundlePattern = "^((?!msajc003).)*$")- implemented
export_seglistToTxtCollection() - normalize length now also using
list()withdo.call(rbind, res_list) - implemted new
runMINNIboolean parameter inrunBASwebservice_all()
- https default on
serve() - fixed problems of
calcFreqs = Tinconvert_wideToLong()(possibly caused by newer version of dplyr?)
- fixed bad column init. in
normalize_length()column - removed detritus LaTex files (e.g. .log, .aux) from
vignette/picsdirectory (as requested by CRAN maintainer)
- URL encoding of bundle and session names to allow for URL string reserved characters in bundle and session names
- error message of
get_trackdata()now contains seglist row index if "Can not extract following" normalize_length()now allows for additional non-numeric columns- changed
stop()towarning()inget_trackdata()when samplerates are inconsistent (closes #190) - better error message when there is a naming mismatch of
_emuDBdir_DBconfig.json - using
sub()instead oftools::file_path_sans_ext()to handle_in file extensions query(),requery_seq()andrequery_hier()now supports theresultType"tibble"(andget_trackdata(),requery_seq(),requery_hier()andserve()support them as input)serve()function now usesseglist$startandseglist$endinstead ofseglist$sample_startandseglist$sample_end- implemented first version of
update_itemsInLevel()(only label updates for now) - now setting the
sample_startandsample_endvalues in query results when EVENT levels are queried (previously onlystartwas set) - implemented first versions of
create_links()(currently not checking for anything),create_itemsInLevel()(only EVENTs and ITEMs) andupdate_itemsInLevel()(only labels) anddelete_itemsInLevel() list_bundles()outside of loop for performance bump inget_trackdata()- better error message in
add_files()if no files are found consistentOutputTypeofget_trackdata()is not set toTRUEand is reset toTifresultTypeis"emuRtrackdata"or"tibble"(fixes #203)- avoiding negative
times_relandtimes_normvalues increate_emuRtrackdata()by setting them to 0 (caused by string to numeric conversion precision errors) requery_seq()now inserts NA values for the out of bounds rows instead of dropping them.- implemented
list_sampleRates()function get_trackdata()is now iteratively appending to alist()instead of into a SQLite temp table. This is a fairly large performance boost and also fixes #206.- added deprecation warnings to vignettes (added links to manual chapters)
- propper fix for "now ordering by
items_idxnot bystart_start_seq_idxwhich led to bad label sequences (fixes #140)" - fixed bad indexing in
normalize_length()when sl_rowIdx values are not ac(1, 2, 3, 4, ...)sequence - fixed
staticContoursSSFF tracks not being sent to EMU-webApp (fixes #195) - fixed bug with completely empty levels that caused a bad resort of levels in
_annot.jsons inrewrite_allAnnots() - fixed bug in
add_files()that was using the wrong variable (fixes #196) - added error message when querying levels without time-bearing sub-levels (closes #150)
- fixed bug in
create_emuRtrackdata()with handling trackdata object of classspectral - correct recalculation of ITEM IDs of missing levels in .hlb files (== only present in ESPS files) in
convert_legacyEmuDB() - fixed handling of completely empty levels in .hlb files
normalize_length()now handles various additional column types (not just"numeric")
- implemented new
consistentOutputTypeparameter forget_trackdata()to always return atrackdataoremuRtrackdataobject independent of what thecutandnpointsarguments are set to - now removing
levelCanvasOrderentry inremove_levelDefinition()(fixes #156) serve()method now uses GET to deliver media files to the EMU-webApp. This avoids the base64 conversion overhead and is a quite significant load time improvement- explicit error message in
convert_legacyEmuDB()when invalid redundant links are found - better error message in BPF parser
convert_legacyEmuDB()automatically converts.ssdmedia files to.wavand normalizes the annotations to start at 0 (only if attr(ssd,'startTime') is not 0).- added
sort()S3 method foremuRsegsobjects - checking for badly sorted
emuRsegsinrequery_hier()andrequery_seq()functions create_emuRtrackdata()returns a simpledata.frameobject not adata.tableobjectemuRtrackdataobject now contains atimes_norm(normalized time values between 0 and 1 for each segment) column by default- added note to
print.emuRsegs()to give the user a hint about missing columns - implemented
print.emuRtrackdata()to avoid overly verbose output - implemented
normalize_length()function as S3 function to normalize the length of each segment in anemuRtrackdataobject - added
absolute_file_pathcolumn to output oflist_files() - query engine does not rely on label index in label array any more (updated
convert_queryResultToEmuRsegs()to useresultAttrDefinstead oflabelIdx). Closes #164. - added
browserargument toserve()function which is passed on toutils::browseURL()function requery_seq()now usesstart_item_seq_idxandend_item_seq_idxof seglist instead ofstart_item_idandstart_item_idto simplify function- implemented
check_emuDBhandle()function that is used on every exported function that takes aemuDBhandleas an argument to check if the handle is still valid (closes #176) - implemented
"tibble"asresultTypeoption inget_trackdata(). This will probably replace the"emuRtrackdata"option in future (it contains exactly the same data/columns). - prechecking if attribute definition is already defined (closes #182)
get_trackdata()now uses temporary SQL tables to store the intermediate results (massive performance gains!). RemovednrOfAllocationRowsparameter as this is no longer needed as no matrix is used to store the intermediate results. (also closes #125)convert_TextGridCollectio()usingdir.exists()instead offile.exists()to check dirs- all read operations now use the readr package (avoids encoding problems like #187)
list_attributeDefinitions()now allows for a name vector to be passed in- rewrite of
rewrite_allAnnots()functions for faster rewrites of_annot.jsonfiles to disk - improved cleanup in testing DBconfig functions
- now ordering by
items_idxnot bystart_start_seq_idxwhich led to bad label sequences (fixes #140)
- fixed problem of updating cache. Didn't handle
data.frameobject that was thought to be a vector correctly! - fixed a bug in the BPF export function, which meant that WAVE files were only copied into one session
- added missing
$in pattern arguments inlist.filescall inlist_files(fixes #170) - not adding ssffTrackDefinition to DBconfig if user input is no (closes #171)
- fixed bad
seq_start_seq_idxandseq_seq_idxreturned byrequery_seq()(fixes #183) - fixed bad
seq_start_seq_idxreturned by internalquery_databaseHier()function - fixed bad sorting of
requery_seq()/requery_hier()whencalcTimes = F(still sorted bystart_sampleinstead of the correctseq_idx) - added
readr::parse_character()to data received inserve()as this is recoded in windows (fixes #188).
- tweaked
runBASwebservice_maus(); improved performance for presegmented bundles - performance bump for
fapply()by preallocating result matrix - performance bump for
trapply()by preallocating result matrix - performance bump for
mel.spectral()by preallocating result matrix - performance bump for
bark.spectral()by preallocating result matrix - updated DBI calls to comply with the latest best practices (using
DBI::dbExecute()instead ofDBI::dbGetQuery()for non-SELECTqueries) - BPF collection exporter documented and now public
export_TextGridCollection()now handles partial includes of bundle and session names correctly (issue #147)- added missing check if
anagestConfigis defined torename_attributeDefinition() - setting useBytes to T to avoid reencoding under windows
- fixed bug in
add_ssffTrackDefinition()that was trying to accessfpwhich was renamed in a refactor tofilesDf - fixed export to autodetect S3 methods (cbind & rbind for trackdata)
- some changes to the parameter names in the BAS webservice functions
- convert_txtCollection and convert_BPFCollection now name topmost item "bundle"
- added functions to set and get level descriptions in DBconfig
- BAS webservice functions now perform a cache update prior to departure
- added multiple perspectives to ae demo database
- choosing explicit paths with intersecting hierarchies now possible
- remove levelDef & linkDef now implement force parameters
- new function convert_txtCollection converts plain text collections into single-node emuDB
- new functions runBASwebservice_* that call various BAS webservices from inside emuR
- NULLing out empty DFs on list_level/linkDefs for more consistent API
newLinkDefTypeargument implemented inautobuild_linkFromTimes()to generate linkDefinition if so desired- automatically removing superlevel from
levelCanvasOrderifconvertSuperlevelis set toTRUEinautobuild_linkFromTimes()
- wrapped
readChars inenc2utf8to avoid encodings issues on windows - updating label table correctly on add_attributeDefinition() (#138)
- runBASwebservice_maus / minni / all now no longer ignore unlinked items (idx -1) but treat them as linkless segments
- commented out
cat()intrain()function be be less verbose - BAS webservice calls now get their own temp directories (UUID based). This avoids race conditions when several scripts are running in parallel.
- convert_txtCollection now treats perspectives as array (as it should)
- added new
EMUwebAppConfig -> perspectives -> signalCanvases -> minMaxValLimsconfig option to emuDB vignette - requery_hier + requery_seq now implement the same timeRefSegmentLevel parameter as query (#135)
- fixed requery_hier() bug of requery on same attribute definition
- fixed requery_hier() bug of requery on same level but different attribute definition
- rewrite of query engine to not require links_ext table any more (== redundant links)
- calcTimes parameter added to query() / requery_seq() / requery_hier() to make calculating times optional (extreme performance boost if no times have to be calculated)
- rewrite of annotJSONcharToBundleAnnotDFs() for faster loads emuDBs containing large annotJSONs
- replaced tidyjson as annot.json parser with own solution at tidyjson didn't scale well on larger annotation files
- added verbose parameter to export_TextGridCollection()
- improved pre-check of dir exists in export_TextGridCollection()
- added new replace_itemLabels function
- improved export_TextGridCollection() doc
- improved replace_itemLables() speed
- implemented rename_emuDB() (#116)
- implemented duplicate_level() (#113)
- implemented linkDuplicates parameter in duplicate_level()
- autobuild_linkFromTimes() speed improvements
- FUNCQ queries (start(),end(), medial()) now additionally support TRUE & FALSE and T & F values (vs. 0 & 1)
- added attrDefNames column to list_levelDefinitions() output
- can now deal with read only emuDBs by copying the cache to tempdir() and making it writable for the user
- added start_item_seq_idx and end_item_seq_idx to emuRsegs object
- added start_item_seq_idx and end_item_seq_idx type values to all intermediate result tables
- added optional function to reduce hierarchical query results to left and right most children only (large performance gain on calcTimes = T)
- rewriting annot.json files now updates MD5 sums as well (avoids unnecessary reload on next load_emuDB)
- rewriting annot.json files now writes all (including empty / missing) attributeDef. labels
- fixed bad DBconfig gen. on add_perspective
- fixed list_linkDefinitions() returning strings as factors
- fixed bad error message when passing in ITEM levels to autobuild_linkFromTimes()
- fixed incorrect handling of DBconfig when writeToFS was set to FALSE (writeToFS is now called rewriteAllAnnots)
- also allowing "time = " in TextTiers
- "levels of type 'EVENT' are not allowed to be super levels (== parents) in a domination relationship" constraint enforced in add_linkDefinition
- added "MEDIAFILE_SAMPLES" as constant name to access audio samples to get_trackdata() function
- improved error message to include tgPath in create_DBconfigFromTextGrid function
- no integer return value returned by create_emuRdemoData() any more! It was implicitly returned from wrassp function call...
- improved the slow overlap checking function in the BPF parser (is now O(n) instead of O(n^2))
- fixed col naming problems for new (unreleased) RSQLite version
- added export_TextGridCollection() function
- improved doc for get_trackdata
- constant naming of EMU-SDMS vs EMU_SDMS in various files
- rewriting all annotation file on add_levelDefinition, remove_levelDefinition
- fixed problem in conversion to JSON with empty items array (object '{}' vs array '[]')
- fixed problem of keywords "number" | "time" | "xmin" | ... in labels causing TextGrid parser to fail
- fixed problem with to lax RegEx in TextGrid parser
- fixed validation problem with missing levels regarding types
- get_trackdata with onTheFly calculation now reuses AsspDataObj if the current utterance is the same as the previous (large performance gain especially on long audio files)
- checking if DBconfig exists for better error message if 'name' field is not set correctly in DBconfig
- setting PRAGMA temp_store = 2; for SQLite connections
- not extracting tables to R if no RegEx needed to create filtered_tmp tables (performance gain when querying large emuDBs)
- convert_BPFCollection can now assigns the same label to more than one item when unifying tiers
- newline at the end of load_emuDB if no redundant links are built
- queries using dominates operator '^' don't use linksExt table anymore -> large performances benefits
- only using _filtered_tmp tables if RegEx patterns are used
- changed primary key on items table which leads to massive performance gains (deleting _emuDBcache.sqlite required)
- fixed error handling of create_emuRtrackdata + added @export to roxygen doc
- invalid annotJSONs generated by import_mediaFiles fixed
- convert_TextGridCollection can now handle nested folders again
- invalid UUIDs in DBConfig produced by convert_BPFCollection. Also added additional unit test to detect this.
- list_bundles uses session argument again
- fixed "Expression tree is too large (maximum depth 1000)" error in get_trackdata with long emuRsegs lists
- R depends version bump to 3.2.0 (as requested by CRAN maintainer)
- updated testthat::expect_less_than to expect_lt calls (due to deprecated warnings)
- Using new .keep_all = T parameter of dplyr
- removed legacy version of EQL vignette (overlooked as inst/doc was in .gitignore)
- skipping in-depth thorough tests on CRAN for query and autobuild SQL functions
- fixed problem of interm_res_tables already being present with queries that have multiple recursion depth on both sides of either -> or ^ operand (e.g. query (ae , "[[[Phonetic = n -> Phonetic =z] -> Phonetic = S ] ^ [Text = friends -> Text = she]]"))
- fixed bad URL in README.md
- added CITATION file
- renamed SQL tables & columns from camel case to underscore notation
- variable SQL backend implementation
- multiple check fixes on various platforms
serveproblem with internalVars bug fixed- file locking problem that caused vignettes to fail under windows problem fixed
- massive refactor of all functions that used to refer to an emuDB by
name and optionally by its UUID. They now use the new emuDBhandle object
that is now returned by the
load_emuDB()function. convert_XXX_to_emuDB()functions renamed toconvert_XXX()