TileDB 2.27.0-rc3

TileDB v2.27.0 Release Notes

Disk Format

• (documentation) The storage format specification was updated to document format changes of previous versions throughout the main document. #5329

Breaking behavior

• Deleting an array or group will also delete its root directory if it was left empty. #5269

New features

• Dumping fragment info now includes the name of each fragment's array schema. #5257
• Added support for the ssl.ca_path config option on Azure. #5286
• Connections to TileDB REST API now use TCP keepalive by default. #5319 and curl retries #5273.
• Added support for "current domain" on dense arrays. #5303

Configuration Options

• Add vfs.log_operations config option to log in trace level all VFS operations and their duration. #5278
• The default value of vfs.s3.region was changed to empty, which will let the AWS pick the default region from sources such as environment variables, profile configuration, and instance metadata. #5317
• Add rest.curl.tcp_keepalive config option that controls using TCP keepalive for TileDB REST connections. It is enabled by default. #5319

Improvements

• Fix read queries on sparse arrays where only aggregates are requested and no layout is specified. #5255
• Prevent potentially unsafe consolidation in dense arrays by fragment list. #5251
• Improve error messages for unsupported data types in aggregates. #5289
• Remove StorageManager from stats dumps. #5297
• Precompute memory usage for tile offsets and error if not loadable due to memory limits. #5310
• GCS object composition temporary files now end with .tiledb.tmp, which may be used to set a lifecycle rule. #5372
• Updated libmagic to version 5.45. #5332
• Linux aarch64 release #5271

Deprecations

• Deprecate tiledb_filestore_* APIs. #5371

Defects removed

• Fix an incompatibility with S3 Express One Zone, by stopping setting Content-MD5 on all S3 uploads. #5226
• Update range end when merging overlapping subarray ranges. #5268
• Fixed GZip compression of empty data. #5296
• Fix HTTP requests for AWS default credentials provider chain not honoring config options. #5315
• Fix dense reader fill value with sm.var_offsets.mode = elements #5331
• Schema evolution bug fix: Reads no longer fail after dropping a fixed attribute and adding it back as var-sized. #5321
• Fixed the tiledb_ctx_alloc_with_error function not being exported from the library. #5357
• Set increased traversal limit in enumeration deserialization #5365
• Fix heap corruption when getting context and query stats from C++ under certain circumstances. #5366
• Schema evolution bug fix: Reads no longer fail after dropping a fixed attribute and adding it back as var-sized, part 2. #5362
• Remove defective test case that's failing in nightly builds. #5369
• Follow up on #5359 #5368

API changes

C API

• Add tiledb_array_schema_get_enumeration API. #5359
• Add tiledb_group_add_member_with_type API to get group member with type in single call. #5336
• Introduce tiledb_array_load_enumerations_all_schemas. #5349
• Added tiledb_group_dump_str_v2 API that returns a tiledb_string_t*. The existing tiledb_group_dump_str API is deprecated. #5367
• Add tiledb_fragment_info_dump_str C API and replace FragmentInfo::dump(FILE*) with operator<< overload. #5266

C++ API

• Fix error log messages when using the Array class in the C++ API. #5262
• Add overload to the Array::create function that explicitly accepts a context. #5325
• Update Group::add_member API to accept an optional type. #5336
• Introduce ArrayExperimental::load_enumerations_all_schemas. #5349

Build System Changes

• Update vcpkg version baseline to microsoft/vcpkg@101cc9a. #5010
• Fixed cross-compiling support in the libmagic vcpkg port overlay. #5333
• The TILEDB_CCACHE option was fixed to have effect, after being accidentally disabled in version 2.26.0. #5342
• Fixed double installation of a header when serialization is enabled. #5354

TileDB 2.27.0-rc2

TileDB v2.27.0 Release Notes

Disk Format

• (documentation) The storage format specification was updated to document format changes of previous versions throughout the main document. #5329

Breaking behavior

• Deleting an array or group will also delete its root directory if it was left empty. #5269

New features

• Dumping fragment info now includes the name of each fragment's array schema. #5257
• Added support for the ssl.ca_path config option on Azure. #5286
• Connections to TileDB REST API now use TCP keepalive by default. #5319 and curl retries #5273.
• Added support for "current domain" on dense arrays. #5303

Configuration Options

• Add vfs.log_operations config option to log in trace level all VFS operations and their duration. #5278
• The default value of vfs.s3.region was changed to empty, which will let the AWS pick the default region from sources such as environment variables, profile configuration, and instance metadata. #5317
• Add rest.curl.tcp_keepalive config option that controls using TCP keepalive for TileDB REST connections. It is enabled by default. #5319

Improvements

• Fix read queries on sparse arrays where only aggregates are requested and no layout is specified. #5255
• Prevent potentially unsafe consolidation in dense arrays by fragment list. #5251
• Improve error messages for unsupported data types in aggregates. #5289
• Remove StorageManager from stats dumps. #5297
• Precompute memory usage for tile offsets and error if not loadable due to memory limits. #5310
• GCS object composition temporary files now end with .tiledb.tmp, which may be used to set a lifecycle rule. #5372
• Updated libmagic to version 5.45. #5332
• Linux aarch64 release #5271

Deprecations

• Deprecate tiledb_filestore_* APIs. #5371

Defects removed

• Fix an incompatibility with S3 Express One Zone, by stopping setting Content-MD5 on all S3 uploads. #5226
• Update range end when merging overlapping subarray ranges. #5268
• Fixed GZip compression of empty data. #5296
• Fix HTTP requests for AWS default credentials provider chain not honoring config options. #5315
• Fix dense reader fill value with sm.var_offsets.mode = elements #5331
• Schema evolution bug fix: Reads no longer fail after dropping a fixed attribute and adding it back as var-sized. #5321
• Fixed the tiledb_ctx_alloc_with_error function not being exported from the library. #5357
• Set increased traversal limit in enumeration deserialization #5365
• Fix heap corruption when getting context and query stats from C++ under certain circumstances. #5366
• Schema evolution bug fix: Reads no longer fail after dropping a fixed attribute and adding it back as var-sized, part 2. #5362
• Remove defective test case that's failing in nightly builds. #5369
• Follow up on #5359 #5368

API changes

C API

• Add tiledb_array_schema_get_enumeration API. #5359
• Add tiledb_group_add_member_with_type API to get group member with type in single call. #5336
• Introduce tiledb_array_load_enumerations_all_schemas. #5349
• Added tiledb_group_dump_str_v2 API that returns a tiledb_string_t*. The existing tiledb_group_dump_str API is deprecated. #5367
• Add tiledb_fragment_info_dump_str C API and replace FragmentInfo::dump(FILE*) with operator<< overload. #5266

C++ API

• Fix error log messages when using the Array class in the C++ API. #5262
• Add overload to the Array::create function that explicitly accepts a context. #5325
• Update Group::add_member API to accept an optional type. #5336
• Introduce ArrayExperimental::load_enumerations_all_schemas. #5349

Build System Changes

• Update vcpkg version baseline to microsoft/vcpkg@101cc9a. #5010
• Fixed cross-compiling support in the libmagic vcpkg port overlay. #5333
• The TILEDB_CCACHE option was fixed to have effect, after being accidentally disabled in version 2.26.0. #5342
• Fixed double installation of a header when serialization is enabled. #5354

TileDB 2.27.0-rc1

TileDB v2.27.0 Release Notes

Disk Format

• (documentation) The storage format specification was updated to document format changes of previous versions throughout the main document. #5329

Breaking behavior

• Deleting an array or group will also delete its root directory if it was left empty. #5269

New features

• Dumping fragment info now includes the name of each fragment's array schema. #5257
• Added support for the ssl.ca_path config option on Azure. #5286
• Connections to TileDB REST API now use TCP keepalive by default. #5319 and curl retries #5273.
• Added support for "current domain" on dense arrays. #5303

Configuration Options

• Add vfs.log_operations config option to log in trace level all VFS operations and their duration. #5278
• The default value of vfs.s3.region was changed to empty, which will let the AWS pick the default region from sources such as environment variables, profile configuration, and instance metadata. #5317
• Add rest.curl.tcp_keepalive config option that controls using TCP keepalive for TileDB REST connections. It is enabled by default. #5319

Improvements

• Fix read queries on sparse arrays where only aggregates are requested and no layout is specified. #5255
• Prevent potentially unsafe consolidation in dense arrays by fragment list. #5251
• Improve error messages for unsupported data types in aggregates. #5289
• Remove StorageManager from stats dumps. #5297
• Precompute memory usage for tile offsets and error if not loadable due to memory limits. #5310
• GCS object composition temporary files now end with .tiledb.tmp, which may be used to set a lifecycle rule. #5372
• Updated libmagic to version 5.45. #5332
• Linux aarch64 release #5271

Deprecations

• Deprecate tiledb_filestore_* APIs. #5371

Defects removed

• Fix an incompatibility with S3 Express One Zone, by stopping setting Content-MD5 on all S3 uploads. #5226
• Update range end when merging overlapping subarray ranges. #5268
• Fixed GZip compression of empty data. #5296
• Fix HTTP requests for AWS default credentials provider chain not honoring config options. #5315
• Fix dense reader fill value with sm.var_offsets.mode = elements #5331
• Schema evolution bug fix: Reads no longer fail after dropping a fixed attribute and adding it back as var-sized. #5321
• Fixed the tiledb_ctx_alloc_with_error function not being exported from the library. #5357
• Set increased traversal limit in enumeration deserialization #5365
• Fix heap corruption when getting context and query stats from C++ under certain circumstances. #5366
• Schema evolution bug fix: Reads no longer fail after dropping a fixed attribute and adding it back as var-sized, part 2. #5362
• Remove defective test case that's failing in nightly builds. #5369
• Follow up on #5359 #5368

API changes

C API

• Add tiledb_array_schema_get_enumeration API. #5359
• Add tiledb_group_add_member_with_type API to get group member with type in single call. #5336
• Introduce tiledb_array_load_enumerations_all_schemas. #5349
• Added tiledb_group_dump_str_v2 API that returns a tiledb_string_t*. The existing tiledb_group_dump_str API is deprecated. #5367
• Add tiledb_fragment_info_dump_str C API and replace FragmentInfo::dump(FILE*) with operator<< overload. #5266

C++ API

• Fix error log messages when using the Array class in the C++ API. #5262
• Add overload to the Array::create function that explicitly accepts a context. #5325
• Update Group::add_member API to accept an optional type. #5336
• Introduce ArrayExperimental::load_enumerations_all_schemas. #5349

Build System Changes

• Update vcpkg version baseline to microsoft/vcpkg@101cc9a. #5010
• Fixed cross-compiling support in the libmagic vcpkg port overlay. #5333
• The TILEDB_CCACHE option was fixed to have effect, after being accidentally disabled in version 2.26.0. #5342
• Fixed double installation of a header when serialization is enabled. #5354

TileDB 2.27.0-rc0

TileDB v2.27.0 Release Notes

Disk Format

• (documentation) The storage format specification was updated to document format changes of previous versions throughout the main document. #5329

Breaking behavior

• Deleting an array or group will also delete its root directory if it was left empty. #5269

New features

• Dumping fragment info now includes the name of each fragment's array schema. #5257
• Added support for the ssl.ca_path config option on Azure. #5286
• Connections to TileDB REST API now use TCP keepalive by default. #5319 and curl retries #5273.
• Added support for "current domain" on dense arrays. #5303

Configuration Options

• Add vfs.log_operations config option to log in trace level all VFS operations and their duration. #5278
• The default value of vfs.s3.region was changed to empty, which will let the AWS pick the default region from sources such as environment variables, profile configuration, and instance metadata. #5317
• Add rest.curl.tcp_keepalive config option that controls using TCP keepalive for TileDB REST connections. It is enabled by default. #5319

Improvements

• Fix read queries on sparse arrays where only aggregates are requested and no layout is specified. #5255
• Prevent potentially unsafe consolidation in dense arrays by fragment list. #5251
• Improve error messages for unsupported data types in aggregates. #5289
• Remove StorageManager from stats dumps. #5297
• Precompute memory usage for tile offsets and error if not loadable due to memory limits. #5310
• GCS object composition temporary files now end with .tiledb.tmp, which may be used to set a lifecycle rule. #5372
• Updated libmagic to version 5.45. #5332
• Linux aarch64 release #5271

Deprecations

• Deprecate tiledb_filestore_* APIs. #5371

Defects removed

• Fix an incompatibility with S3 Express One Zone, by stopping setting Content-MD5 on all S3 uploads. #5226
• Update range end when merging overlapping subarray ranges. #5268
• Fixed GZip compression of empty data. #5296
• Fix HTTP requests for AWS default credentials provider chain not honoring config options. #5315
• Fix dense reader fill value with sm.var_offsets.mode = elements #5331
• Schema evolution bug fix: Reads no longer fail after dropping a fixed attribute and adding it back as var-sized. #5321
• Fixed the tiledb_ctx_alloc_with_error function not being exported from the library. #5357
• Set increased traversal limit in enumeration deserialization #5365
• Fix heap corruption when getting context and query stats from C++ under certain circumstances. #5366
• Schema evolution bug fix: Reads no longer fail after dropping a fixed attribute and adding it back as var-sized, part 2. #5362
• Remove defective test case that's failing in nightly builds. #5369
• Follow up on #5359 #5368

API changes

C API

• Add tiledb_array_schema_get_enumeration API. #5359
• Add tiledb_group_add_member_with_type API to get group member with type in single call. #5336
• Introduce tiledb_array_load_enumerations_all_schemas. #5349
• Added tiledb_group_dump_str_v2 API that returns a tiledb_string_t*. The existing tiledb_group_dump_str API is deprecated. #5367
• Add tiledb_fragment_info_dump_str C API and replace FragmentInfo::dump(FILE*) with operator<< overload. #5266

C++ API

• Fix error log messages when using the Array class in the C++ API. #5262
• Add overload to the Array::create function that explicitly accepts a context. #5325
• Update Group::add_member API to accept an optional type. #5336
• Introduce ArrayExperimental::load_enumerations_all_schemas. #5349

Build System Changes

• Update vcpkg version baseline to microsoft/vcpkg@101cc9a. #5010
• Fixed cross-compiling support in the libmagic vcpkg port overlay. #5333
• The TILEDB_CCACHE option was fixed to have effect, after being accidentally disabled in version 2.26.0. #5342
• Fixed double installation of a header when serialization is enabled. #5354

TileDB 2.26.2

TileDB v2.26.2 Release Notes

Defects removed

• Fix HTTP requests for AWS default credentials provider chain not honoring config options. #5318

TileDB 2.26.1

TileDB v2.26.1 Release Notes

Improvements

• Fail early on dense reads when tile offsets are too large. #5311

TileDB 2.26.0

TileDB v2.26.0 Release Notes

Breaking behavior

• The superbuild architecture of the build system has been removed and TileDB is now a single-level CMake project. Build commands of the form make && make -C tiledb <targets> will have to be replaced by make <targets>. #5021

Breaking C API changes

• All deprecated C and C++ APIs were removed. #5146

New features

• Add support for getting the datatype of a ndrectangle dimension. #5229
• Add dim num support for ndrectangle. #5230

Improvements

• Added new functions to write schema dump to a string rather than stdout, so that non-console interactive environments such as Jupyter will be able to capture and print the output. #5026
• Improve dense read performance for small reads. #5145
• Skip caching redirect uri on array create. #5224
• Enable curl error retries. #5275

Deprecations

• Warn users using dense arrays with sparse fragments. #5116

Defects removed

• Fix fragment consolidation to allow using absolute URIs. #5135
• Reset offsets in buffer list for retries. #5220
• Fix symbol clashes between tiledb and pyarrow by building the AWS SDK with its internal symbols hidden. #5223

Configuration changes

• The sm.use_refactored_readers config option is no longer recognized. Refactored readers are used by default. To use the legacy reader, set the sm.query_(dense|sparse_global_order|sparse_unordered_with_dups)_reader config option (depending on the reader you are using) to legacy. #5183

API changes

C API

• Add ctx to CurrentDomain CAPI. #5219
• Add new CAPIs to dump array schema, attribute, dimension, domain, enumeration and group to a string. #5026

Build System Changes

• The version of OpenSSL linked to the release artifacts was updated to 3.1.4, and moving forward will be synced to the vcpkg builtin-baseline. #5174

TileDB 2.26.0-rc5

TileDB v2.26.0 Release Notes

Breaking behavior

• The superbuild architecture of the build system has been removed and TileDB is now a single-level CMake project. Build commands of the form make && make -C tiledb <targets> will have to be replaced by make <targets>. #5021

Breaking C API changes

• All deprecated C and C++ APIs were removed. #5146

New features

• Add support for getting the datatype of a ndrectangle dimension. #5229
• Add dim num support for ndrectangle. #5230

Improvements

• Added new functions to write schema dump to a string rather than stdout, so that non-console interactive environments such as Jupyter will be able to capture and print the output. #5026
• Improve dense read performance for small reads. #5145
• Skip caching redirect uri on array create. #5224
• Enable curl error retries. #5275

Deprecations

• Warn users using dense arrays with sparse fragments. #5116

Defects removed

• Fix fragment consolidation to allow using absolute URIs. #5135
• Reset offsets in buffer list for retries. #5220
• Fix symbol clashes between tiledb and pyarrow by building the AWS SDK with its internal symbols hidden. #5223

Configuration changes

• The sm.use_refactored_readers config option is no longer recognized. Refactored readers are used by default. To use the legacy reader, set the sm.query_(dense|sparse_global_order|sparse_unordered_with_dups)_reader config option (depending on the reader you are using) to legacy. #5183

API changes

C API

• Add ctx to CurrentDomain CAPI. #5219
• Add new CAPIs to dump array schema, attribute, dimension, domain, enumeration and group to a string. #5026

Build System Changes

• The version of OpenSSL linked to the release artifacts was updated to 3.1.4, and moving forward will be synced to the vcpkg builtin-baseline. #5174

TileDB 2.26.0-rc4

TileDB v2.26.0 Release Notes

Breaking behavior

• The superbuild architecture of the build system has been removed and TileDB is now a single-level CMake project. Build commands of the form make && make -C tiledb <targets> will have to be replaced by make <targets>. #5021

Breaking C API changes

• All deprecated C and C++ APIs were removed. #5146

New features

• Add support for getting the datatype of a ndrectangle dimension. #5229
• Add dim num support for ndrectangle. #5230

Improvements

• Added new functions to write schema dump to a string rather than stdout, so that non-console interactive environments such as Jupyter will be able to capture and print the output. #5026
• Improve dense read performance for small reads. #5145
• Skip caching redirect uri on array create. #5224
• Enable curl error retries. #5275

Deprecations

• Warn users using dense arrays with sparse fragments. #5116

Defects removed

• Fix fragment consolidation to allow using absolute URIs. #5135
• Reset offsets in buffer list for retries. #5220
• Fix symbol clashes between tiledb and pyarrow by building the AWS SDK with its internal symbols hidden. #5223

Configuration changes

• The sm.use_refactored_readers config option is no longer recognized. Refactored readers are used by default. To use the legacy reader, set the sm.query_(dense|sparse_global_order|sparse_unordered_with_dups)_reader config option (depending on the reader you are using) to legacy. #5183

API changes

C API

• Add ctx to CurrentDomain CAPI. #5219
• Add new CAPIs to dump array schema, attribute, dimension, domain, enumeration and group to a string. #5026
• Add serialization and API changes for post_array_schema_from_rest. #5261
• Add tiledb_array_schema_load_with_config C API to load the schema of an array with configuration. #5261

Build System Changes

• The version of OpenSSL linked to the release artifacts was updated to 3.1.4, and moving forward will be synced to the vcpkg builtin-baseline. #5174

TileDB 2.26.0-rc3

TileDB v2.26.0 Release Notes

Breaking behavior

• The superbuild architecture of the build system has been removed and TileDB is now a single-level CMake project. Build commands of the form make && make -C tiledb <targets> will have to be replaced by make <targets>. #5021

Breaking C API changes

• All deprecated C and C++ APIs were removed. #5146

New features

• Add support for getting the datatype of a ndrectangle dimension. #5229
• Add dim num support for ndrectangle. #5230

Improvements

• Added new functions to write schema dump to a string rather than stdout, so that non-console interactive environments such as Jupyter will be able to capture and print the output. #5026
• Improve dense read performance for small reads. #5145
• Skip caching redirect uri on array create. #5224
• Enable curl error retries. #5275

Deprecations

• Warn users using dense arrays with sparse fragments. #5116

Defects removed

• Fix fragment consolidation to allow using absolute URIs. #5135
• Reset offsets in buffer list for retries. #5220
• Fix symbol clashes between tiledb and pyarrow by building the AWS SDK with its internal symbols hidden. #5223

Configuration changes

• The sm.use_refactored_readers config option is no longer recognized. Refactored readers are used by default. To use the legacy reader, set the sm.query_(dense|sparse_global_order|sparse_unordered_with_dups)_reader config option (depending on the reader you are using) to legacy. #5183

API changes

C API

• Add ctx to CurrentDomain CAPI. #5219
• Add new CAPIs to dump array schema, attribute, dimension, domain, enumeration and group to a string. #5026
• Add serialization and API changes for post_array_schema_from_rest. #5261
• Add tiledb_array_schema_load_with_config C API to load the schema of an array with configuration. #5261

Build System Changes

• The version of OpenSSL linked to the release artifacts was updated to 3.1.4, and moving forward will be synced to the vcpkg builtin-baseline. #5174