Skip to content
This repository has been archived by the owner on Jun 7, 2022. It is now read-only.

Data encoding scenarios and demos

Jump to bottom
Darren Ldl edited this page Apr 13, 2019 · 4 revisions

Purpose

This page discuss scenarios related to encoding of data. Not all scenarios are designed to be realistic, they are designed to illustrate various techniques for you to pick and apply for your specific scenario.

Default encoding

Demo

Encoding a single file with no forward error correction

Demo

Sometimes the file itself already has forward error correction, or you just want a very quick encoding for temporary storage/transfer where you can tolerate slight losses, but sector level recoverability is still desirable.

Blkar by default uses forward error correction (FEC) during encoding, so you'll need to pick one of the SBX versions which does not have FEC enabled.

Suppose the file is called something.png, then we can do the following for example

blkar encode --sbx-version 1 something.png

This uses SBX version 1, the default hashing algorithm (SHA256), and outputs to something.png.sbx.

Encoding multiple files by encoding data from stdin

Demo

blkar cannot archive multiple files together, so you'll need to use things like tar to bundle files together first.

However, blkar encode mode supports receiving input from stdin since 2.1.0, so you can do this in one go easily

tar cvf - something | blkar encode - container.sbx

the - on blkar side asks blkar to take input from stdin instead of file.

Want to double check parameters and final container size

Sometimes you typed a bunch of stuff, and you're tired and it's 3am. You're very prone to making mistakes at this point, and you want to double check encoding parameters as encoding is the likely the most important stage.

Just add --info-only to whatever options you've already added, and blkar will switch to displaying the recorded and some calculated information instead

$ blkar encode something --sbx-version 17 --rs-data 20 --rs-parity 5 --burst 150 --info-only
File name                    : something
SBX container name           : something.sbx
SBX container version        : 17
SBX container block size     : 512
SBX container data  size     : 496
RS data   shard count        : 20
RS parity shard count        : 5
Burst error resistance level : 150
File size                    : 2249
SBX container size           : 1846784
File modification time       : 2019-04-05 08:55:45 (UTC)  2019-04-05 19:55:45 (Local)
Clone this wiki locally