diff --git a/README.md b/README.md index 4a8c780..d732bd5 100644 --- a/README.md +++ b/README.md @@ -13,14 +13,11 @@ As the scheme removes the overhead of including root and intermediate certificat ## Preliminary Evaluation -This draft is a work in progress, however a preliminary evaluation is available: - | Scheme | p5 | p50 | p95 | |------------------------------------------------------|------|-------|-------| -| Original | 2308 | 4032 | 5609 | -| TLS Cert Compression | 1619 | 3243 | 3821 | -| Intermediate Suppression and TLS Cert Compression | 1020 | 1445 | 3303 | -| **This Draft** | 661 | 1060 | 1437 | +| Original / Uncompressed | 2308 | 4032 | 5609 | +| Existing TLS Certificate Compression | 1619 | 3243 | 3821 | +| **This Draft** | 881 | 1256 | 1716 | | Hypothetical Optimal Compression | 377 | 742 | 1075 | A complete table of results and benchmarking scripts can be found in [benchmarks](benchmarks/). diff --git a/benchmarks/README.md b/benchmarks/README.md index 3e4be35..c38ef91 100644 --- a/benchmarks/README.md +++ b/benchmarks/README.md @@ -2,31 +2,27 @@ This folder contains scripts for benchmarking the various compression schemes. -| Scheme | Storage Footprint | p5 | p50 | p95 | -|------------------------------------------------------|---------------------|------|-------|-------| -| Original | 0 | 2308 | 4032 | 5609 | -| TLS Cert Compression | 0 | 1619 | 3243 | 3821 | -| Intermediate Suppression | 0 | 1315 | 1688 | 4227 | -| Intermediate Suppression and TLS Cert Compression | 0 | 1020 | 1445 | 3303 | -| Pass 1 only (intermediate and root compression) | 0 | 1001 | 1429 | 2456 | -| Dictionary composed all intermediate and root certs | 3455467 | 721 | 1094 | 1631 | -| Pass 1 plus popular strings | 1848 | 718 | 1128 | 1627 | -| This Draft | 65336 | 661 | 1060 | 1437 | -| Pass 1 plus trained end-entity zstd dict | 3000 | 562 | 931 | 1454 | -| Pass 1 plus trained end-entity zstd dict | 100000 | 520 | 894 | 1291 | -| Hypothetical Optimal Compression | 0 | 377 | 742 | 1075 | +| Scheme | Storage Footprint | p5 | p50 | p95 | +|-------------------------------------------------------------------------------------------------|---------------------|------|-------|-------| +| Original | 0 | 2308 | 4031 | 5636 | +| TLS Cert Compression | 0 | 1673 | 3319 | 3963 | +| Intermediate Suppression | 0 | 1316 | 1689 | 4220 | +| Intermediate Suppression and TLS Cert Compression | 0 | 1035 | 1467 | 3370 | +| Hypothetical Optimal Compression | 0 | 380 | 746 | 1078 | +| Leaf Certificate Metadata Estimate | 0 | 568 | 721 | 1072 | +| Leaf Certificate Compressed Domains Estimate | 0 | 25 | 39 | 273 | +| CA Prefix Only | 0 | 1005 | 1440 | 2498 | +| Base: Dictionary Compressor Base: Zstandard + Offline Compression:False | 0 | 877 | 1293 | 1797 | +| Base: Dictionary Compressor Base: Zstandard + Offline Compression:True | 0 | 868 | 1286 | 1757 | +| **This Draft | 0 | 881 | 1256 | 1716 | +| Method 1: Baseline Base: Zstandard + Offline Compression:True | 3455467 | 721 | 1095 | 1633 | +| Method 1: Baseline Base: Zstandard + Offline Compression:False | 3455467 | 1179 | 2874 | 3344 | +| Method 2: CA Prefix with Training redacted=True, offlineComp=True | 3000 | 582 | 959 | 1538 | +| Method 2: CA Prefix with Training redacted=True, offlineComp=True | 100000 | 548 | 931 | 1393 | +| Method 2: CA Prefix and CommonStrings threshold=2000 Base: Zstandard + Offline Compression:True | 1848 | 724 | 1131 | 1641 | +| Method 2: CA Prefix and SystematicStrings Base: Zstandard + Offline Compression:True | 65336 | 661 | 1061 | 1447 | +| Method 2: CA Prefix and SystematicStrings Base: Zstandard + Offline Compression:False | 65336 | 690 | 1087 | 1515 | - -## Evaluated Schemes - -* **TLS Certificate Compression** - Using zstandard tuned for maximum compression -* **Intermediate Suppression** - Removes the intermediate and root certificates from the chain. Has no effect if the chain contains a certificate not in Mozilla's list of intermediates and roots. -* **Pass 1** - Just the compression scheme defined in the draft for intermediate and root certificates. -* **Dictionary of all intermediate and root certs** - Pure Zstandard with a very large dictionary composed of the concatination of all intermediate and root certificates in the CCADB. -* **Pass 1 plus popular strings** - Pass 1 followed by compression of the end-entity certificate using a dictionary of common strings extracted from witnessed certificate chains. -* **This Draft** - As described in this document. -* **Pass 1 plus trained zstd dict** - The end entity certiifcates from a number of certificate chains have their subject-specific (name, domains) removed and then are passed to the Zstandard dictionary training function with a size of 3 KB or 100 KB. -* **Hypothetical Optimal Compression** - This assumes an end-entity certificate can be reduced to purely the compressed domain names and public key, CA signature and SCTs. ## Methodology These compression schemes are defined in the associated scripts in the schemes folder. Each scheme is evaluated over a sample of certificate chains fetched from the Tranco top 100k. The confidence interval for each percentile is calculated and the upper bound is taken. \ No newline at end of file diff --git a/draft-ietf-tls-cert-abridge.md b/draft-ietf-tls-cert-abridge.md index a1f26e0..c4aea65 100644 --- a/draft-ietf-tls-cert-abridge.md +++ b/draft-ietf-tls-cert-abridge.md @@ -222,27 +222,23 @@ The second pass uses Brotli {{BROTLI}} to compress any redundant data in the end * `quality=5` * `lgwindow=17` -Benchmarks on real world certificate chains suggest that in this context higher values require greater CPU usage but do not result in better compression, +Benchmarks on real world certificate chains suggest that higher values require greater CPU usage but do not result in better compression. -# Preliminary Evaluation {#eval} +# Evaluation {#eval} [[**NOTE:** This section to be removed prior to publication.]] -The storage footprint refers to the on-disk size required for the end-entity dictionary. The other columns report the 5th, 50th and 95th percentile of the resulting certificate chains. The evaluation set was a ~75,000 certificate chains from the Tranco list using the python scripts in the draft's Github repository. +The columns report the 5th, 50th and 95th percentile of the resulting certificate chains wire sizes in bytes. The evaluation set was ~75000 certificate chains from the Tranco list using the python scripts in the draft's Github repository. -| Scheme | Storage Footprint | p5 | p50 | p95 | -|------------------------------------------------------|---------------------|------|-------|-------| -| Original | 0 | 2308 | 4032 | 5609 | -| TLS Cert Compression | 0 | 1619 | 3243 | 3821 | -| Intermediate Suppression and TLS Cert Compression | 0 | 1020 | 1445 | 3303 | -| **This Draft** | 65336 | 661 | 1060 | 1437 | -| **This Draft with opaque trained dictionary** | 3000 | 562 | 931 | 1454 | -| Hypothetical Optimal Compression | 0 | 377 | 742 | 1075 | +| Scheme | p5 | p50 | p95 | +|------------------------------------------------------|------|-------|-------| +| Original / Uncompressed | 2308 | 4032 | 5609 | +| Existing TLS Certificate Compression | 1619 | 3243 | 3821 | +| **This Draft** | 881 | 1256 | 1716 | +| Hypothetical Optimal Compression | 377 | 742 | 1075 | * 'Original' refers to the sampled certificate chains without any compression. * 'TLS Cert Compression' used ZStandard with the parameters configured for maximum compression as defined in {{TLSCertCompress}}. - * 'Intermediate Suppression and TLS Cert Compression' was modelled as the elimination of all certificates in the intermediate and root certificates with the Basic Constraints CA value set to true. If a cert chain included an unrecognized certificate with CA status, then no CA certificates were removed from that chain. The cert chain was then passed to 'TLS Cert Compression' as a second pass. - * 'This Draft with opaque trained dictionary' refers to pass 1 and pass 2 as defined by this draft, but instead using a 3000 byte dictionary for pass 2 which was produced by the Zstandard dictionary training algorithm. This illustrates a ceiling on what ought to be possible by improving the construction of the pass 2 dictionary in this document. However, using this trained dictionary directly will not treat all CA's equitably, as the dictionary will be biased towards compressing the most popular CAs more effectively. * 'Hypothetical Optimal Compression' is the resulting size of the cert chain after reducing it to only the public key in the end-entity certificate, the CA signature over the EE cert, the embedded SCT signatures and a compressed list of domains in the SAN extension. This represents the best possible compression as it entirely removes any CA certs, identifiers, field tags and lengths and non-critical extensions such as OCSP, CRL and policy extensions. # Deployment Considerations {#deployment}