-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
compress: add "slack" compression option
The "slack" option simply searches from the end of the block backwards to the last non-zero byte, and sets that position as the "compressed" size. Sponsored-by: Klara, Inc. Sponsored-by: Wasabi Technology, Inc. Signed-off-by: Rob Norris <[email protected]>
- Loading branch information
Showing
22 changed files
with
196 additions
and
13 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -18,7 +18,7 @@ | |
| .\" Copyright (c) 2019, Allan Jude | ||
| .\" Copyright (c) 2021, Colm Buckley <[email protected]> | ||
| .\" | ||
| .Dd October 2, 2024 | ||
| .Dd November 18, 2024 | ||
| .Dt ZPOOL-FEATURES 7 | ||
| .Os | ||
| . | ||
|
|
@@ -932,6 +932,41 @@ preventing hash collision attacks on systems with dedup. | |
| .Pp | ||
| .checksum-spiel skein | ||
| . | ||
| .feature com.klarasystems slack_compress no extensible_dataset | ||
| .Sy slack | ||
| is a compression option that simply removes the trailing run of zero bytes at | ||
| the end of each block. | ||
| It is designed for when very large block sizes are used with to store largely | ||
| incompressible data. | ||
| Normally in this scenario | ||
| .Sy compress Ns = Ns Sy none | ||
| would be used, | ||
| but with large block sizes that can often leave a multi-megabyte run of zeroes | ||
| at the end of a block, adding memory and checksumming overhead. | ||
| .Pp | ||
| When the | ||
| .Sy slack_compress | ||
| feature is set to | ||
| .Sy enabled , | ||
| the administrator can turn on | ||
| .Sy slack | ||
| compression of any dataset using | ||
| .Nm zfs Cm set Sy compress Ns = Ns Sy slack Ar dset | ||
| .Po see Xr zfs-set 8 Pc . | ||
| This feature becomes | ||
| .Sy active | ||
| once a | ||
| .Sy compress | ||
| property has been set to | ||
| .Sy slack , | ||
| and will return to being | ||
| .Sy enabled | ||
| once all filesystems that have ever had their | ||
| .Sy compress | ||
| property set to | ||
| .Sy slack | ||
| are destroyed. | ||
| . | ||
| .feature com.delphix spacemap_histogram yes | ||
| This features allows ZFS to maintain more information about how free space | ||
| is organized within the pool. | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| /* | ||
| * CDDL HEADER START | ||
| * | ||
| * The contents of this file are subject to the terms of the | ||
| * Common Development and Distribution License (the "License"). | ||
| * You may not use this file except in compliance with the License. | ||
| * | ||
| * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE | ||
| * or http://www.opensolaris.org/os/licensing. | ||
| * See the License for the specific language governing permissions | ||
| * and limitations under the License. | ||
| * | ||
| * When distributing Covered Code, include this CDDL HEADER in each | ||
| * file and include the License file at usr/src/OPENSOLARIS.LICENSE. | ||
| * If applicable, add the following below this CDDL HEADER, with the | ||
| * fields enclosed by brackets "[]" replaced with your own identifying | ||
| * information: Portions Copyright [yyyy] [name of copyright owner] | ||
| * | ||
| * CDDL HEADER END | ||
| */ | ||
|
|
||
| /* | ||
| * Copyright (c) 2024, Klara, Inc. | ||
| */ | ||
|
|
||
| #include <sys/zio_compress.h> | ||
| #include <sys/types.h> | ||
|
|
||
| /* | ||
| * Slack compression simply searches for the last non-zero byte in the buffer, | ||
| * and sets the position as the size of the "compressed" data. | ||
| */ | ||
|
|
||
| static size_t | ||
| zfs_slack_compress_buf(void *src, void *dst, size_t s_len, size_t d_len, | ||
| int level) | ||
| { | ||
| (void) level; | ||
|
|
||
| ASSERT3U(s_len, >, 0); | ||
| ASSERT0(P2PHASE(s_len, sizeof (uint64_t))); | ||
|
|
||
| uint64_t *buf = (uint64_t *)src; | ||
|
|
||
| int p = (s_len / sizeof (uint64_t)) - 1; | ||
| for (; p >= 0; p--) | ||
| if (buf[p] != 0) | ||
| break; | ||
|
|
||
| if (p < 0) | ||
| return (s_len); | ||
|
|
||
| size_t c_len = (p + 1) * sizeof (uint64_t); | ||
| if (c_len > d_len) | ||
| return (s_len); | ||
|
|
||
| memcpy(dst, src, c_len); | ||
| return (c_len); | ||
| } | ||
|
|
||
| static int | ||
| zfs_slack_decompress_buf(void *src, void *dst, size_t s_len, size_t d_len, | ||
| int level) | ||
| { | ||
| (void) level; | ||
| ASSERT3U(d_len, >=, s_len); | ||
| memcpy(dst, src, s_len); | ||
| if (d_len > s_len) | ||
| memset(dst+s_len, 0, d_len-s_len); | ||
| return (0); | ||
| } | ||
|
|
||
| ZFS_COMPRESS_WRAP_DECL(zfs_slack_compress) | ||
| ZFS_DECOMPRESS_WRAP_DECL(zfs_slack_decompress) |
Oops, something went wrong.