Skip to content

Commit

Permalink
Backup
Browse files Browse the repository at this point in the history
  • Loading branch information
@vic-ma
vic-ma committed Jun 7, 2024
1 parent c7fc8ab commit 8448063
Showing 1 changed file with 34 additions and 190 deletions.
224 changes: 34 additions & 190 deletions course-definition.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,223 +26,67 @@ stages:
name: "Read header"
difficulty: very_easy
description_md: |-
In this stage, you'll add support for reading a single key from an RDB file.
Welcome to the RDB Persistence Extension! In this extension, you'll add support for reading [RDB files](https://redis.io/docs/management/persistence/) (Redis Database files).
### RDB file format
In this stage, you'll add support for two configuration parameters related to RDB persistence, as well as the [CONFIG GET](https://redis.io/docs/latest/commands/config-get/) command.
<details>
<summary>Click to expand/collapse</summary>
#### RDB file format overview
### RDB files
Here are the different sections of the RDB file, in order:
An RDB file is a point-in-time snapshot of a Redis dataset. When RDB persistence is enabled, the Redis server syncs its in-memory state with an RDB file, by doing the following:
1. Header section
2. Metadata section
3. Database section
4. End of file section
1. On startup, the Redis server loads the data from the RDB file.
2. While running, the Redis server periodically takes new snapshots of the dataset, in order to update the RDB file.
RDB files use special encodings to store different types of data. The ones relevant to this stage are "size encoding" and "string encoding." These are explained near the end of this page.
### `dir` and `dbfilename`
The following breakdown of the RDB file format is based on [Redis RDB File Format](https://rdb.fnordig.de/file_format.html) by Jan-Erik Rediger. We’ve only included the parts that are relevant to this stage.
The configuration parameters `dir` and `dbfilename` specify where an RDB file is stored:
- `dir` - the path to the directory where the RDB file is stored (example: `/tmp/redis-data`)
- `dbfilename` - the name of the RDB file (example: `rdbfile`)
#### Header section
### The `CONFIG GET` command
RDB files begin with a header section, which looks something like this:
```
52 45 44 49 53 30 30 30 37 // Magic string + version number (ASCII): "REDIS0007".
```
The header contains the magic string `REDIS`, followed by a four-character RDB version number. In this challenge, the test RDB files all use version 7. So, the header is always `REDIS0007`.
#### Metadata section
Next is the metadata section. It contains zero or more "metadata subsections," which each specify a single metadata attribute. Here's an example of a metadata subsection that specifies `redis-ver`:
```
FA // Indicates the start of a metadata subsection.
09 72 65 64 69 73 2D 76 65 72 // The name of the metadata attribute (string encoded): "redis-ver".
06 36 2E 30 2E 31 36 // The value of the metadata attribute (string encoded): "6.0.16".
```
The metadata name and value are always string encoded.
#### Database section
Next is the database section. It contains zero or more "database subsections," which each describe a single database. Here's an example of a database subsection:
```
FE // Indicates the start of a database subsection.
00 /* The index of the database (size encoded).
Here, the index is 0. */
FB // Indicates that hash table size information follows.
02 /* The size of the hash table that stores the keys and values (size encoded).
Here, the key-value hash table size is 2. */
01 /* The size of the hash table that stores the expires of the keys (size encoded).
Here, the expire hash table size is 1. */
The [`CONFIG GET`](https://redis.io/docs/latest/commands/config-get/) command returns the values of configuration parameters.
00 /* The 1-byte flag that specifies the value’s type and encoding.
Here, the flag is 0, which means "string." */
06 66 6F 6F 62 61 72 // The name of the key (string encoded). Here, it's "foobar".
06 62 61 7A 71 75 78 // The value (string encoded). Here, it's "bazqux".
It takes in one or more configuration parameters and returns a [RESP array](https://redis.io/docs/latest/develop/reference/protocol-spec/#arrays) of key-value pairs:
FC /* Indicates that this key has an expire,
and that the expire timestamp is expressed in milliseconds. */
15 72 E7 07 8F 01 00 00 /* The expire timestamp, expressed in Unix time,
stored as an 8-byte unsigned long, in little-endian (read right-to-left).
Here, the expire timestamp is 1713824559637. */
00 // Value type is string.
03 66 6F 6F // Key name is "foo".
03 62 61 72 // Value is "bar".
FD /* Indicates that this key has an expire,
and that the expire timestamp is expressed in seconds. */
52 ED 2A 66 /* The expire timestamp, expressed in Unix time,
stored as an 4-byte unsigned integer, in little-endian (read right-to-left).
Here, the expire timestamp is 1714089298. */
00 // Value type is string.
03 62 61 7A // Key name is "baz".
03 71 75 78 // Value is "qux".
```
Here's a more formal description of how each key-value pair is stored:
1. Optional expire information (one of the following):
* Timestamp in seconds:
1. `FD`
2. Expire timestamp in seconds (4-byte unsigned integer)
* Timestamp in milliseconds:
1. `FC`
2. Expire timestamp in milliseconds (8-byte unsigned long)
2. Value type (1-byte flag)
3. Key (string encoded)
4. Value (encoding depends on value type)
#### End of file section
This section marks the end of the file. It looks something like this:
```bash
$ redis-cli CONFIG GET dir
1) "dir"
2) "/tmp/redis-data"
```
FF /* Indicates that the file is ending,
and that the checksum follows. */
89 3b b7 4e f8 0f 77 19 // An 8-byte CRC64 checksum of the entire file.
```
#### Size encoding
Size-encoded values specify the size of something. Here are some examples:
- The database indexes and hash table sizes are size encoded.
- String encoding begins with a size-encoded value that specifies the number of characters in the string.
- List encoding begins with a size-encoded value that specifies the number of elements in the list.
The first two bits of a size-encoded value indicate how the value should be parsed. Here's a guide (bits are shown in both hexadecimal and binary):
```
/* If the first two bits are 0b00:
The size is the remaining 6 bits of the byte.
In this example, the size is 10: */
0A
00001010
/* If the first two bits are 0b01:
The size is the next 14 bits
(remaining 6 bits in the first byte, combined with the next byte),
in big-endian (read left-to-right).
In this example, the size is 700: */
42 BC
01000010 10111100
/* If the first two bits are 0b10:
Ignore the remaining 6 bits of the first byte.
The size is the next 4 bytes, in big-endian (read left-to-right).
In this example, the size is 17000: */
80 00 00 42 68
10000000 00000000 00000000 01000010 01101000
Although `CONFIG GET` can fetch multiple parameters at a time, the tester will only send `CONFIG GET` commands with one parameter at a time.
/* If the first two bits are 0b11:
The remaining 6 bits specify a type of string encoding.
See string encoding section. */
```
#### String encoding
A string-encoded value consists of two parts:
1. The size of the string (size encoded).
2. The string.
### Tests
Here's an example:
```
/* The 0x0D size specifies that the string is 13 characters long.
The remaining characters spell out "Hello, World!". */
0D 48 65 6C 6C 6F 2C 20 57 6F 72 6C 64 21
```
The tester will execute your program like this:
For sizes that begin with `0b11`, the remaining 6 bits indicate a type of string format:
```bash
./spawn_redis_server.sh --dir /tmp/redis-files --dbfilename dump.rdb
```
/* The 0xC0 size indicates the string is an 8-bit integer.
In this example, the string is "123". */
C0 7B
/* The 0xC1 size indicates the string is a 16-bit integer.
The remaining bytes are in little-endian (read right-to-left).
In this example, the string is "12345". */
C1 39 30
It'll then send the following commands:
/* The 0xC2 size indicates the string is a 32-bit integer.
The remaining bytes are in little-endian (read right-to-left),
In this example, the string is "1234567". */
C2 87 D6 12 00
/* The 0xC3 size indicates that the string is compressed with the LZF algorithm.
You will not encounter LZF-compressed strings in this challenge. */
C3 ...
```bash
$ redis-cli CONFIG GET dir
$ redis-cli CONFIG GET dbfilename
```
</details>
Your server must respond to each `CONFIG GET` command with a RESP array containing two elements:
### The `KEYS` command
<details>
<summary>Click to expand/collapse</summary>
1. The parameter **name**, encoded as a [RESP Bulk string](https://redis.io/docs/latest/develop/reference/protocol-spec/#bulk-strings)
2. The parameter **value**, encoded as a RESP Bulk string
The [`KEYS command`](https://redis.io/docs/latest/commands/keys/) returns all the keys that match a given pattern, as a RESP array:
```
$ redis-cli SET foo bar
OK
$ redis-cli SET baz qux
OK
$ redis-cli KEYS "f*"
1) "foo"
```
For example, if the value of `dir` is `/tmp/redis-files`, then the expected response to `CONFIG GET dir` is:
When the pattern is `*`, the command returns all the keys in the database:
```
$ redis-cli KEYS "*"
1) "baz"
2) "foo"
```
In this stage, you must add support for the `KEYS` command. However, you only need to support the `*` pattern.
</details>
### Tests
The tester will create an RDB file with a single key and execute your program like this:
```
$ ./spawn_redis_server.sh --dir <dir> --dbfilename <filename>
```
It'll then send a `KEYS "*"` command to your server.
```
$ redis-cli KEYS "*"
```
Your server must respond with a RESP array that contains the key from the RDB file:
```
*1\r\n$3\r\nfoo\r\n
```bash
*2\r\n$3\r\ndir\r\n$16\r\n/tmp/redis-files\r\n
```
### Notes
- The RDB file provided by `--dir` and `--dbfilename` might not exist. If the file doesn't exist, your program must treat the database as empty.
- RDB files use both little-endian and big-endian to store numbers. See the [MDN article on endianness](https://developer.mozilla.org/en-US/docs/Glossary/Endianness) to learn more.
- To generate an RDB file, use the [`SAVE` command](https://redis.io/docs/latest/commands/save/).
- You don't need to read the RDB file in this stage, you only need to store `dir` and `dbfilename`. Reading from the file will be covered in later stages.
- If your repository was created before 5th Oct 2023, it's possible that your `./spawn_redis_server.sh` script is not passing arguments to your program. To fix this, you'll need to edit `./spawn_redis_server.sh`. Check the [update CLI args PR](https://github.com/codecrafters-io/build-your-own-redis/pull/89/files) for details on how to do this.
marketing_md: |-
In this stage, we'll do XYZ.
Expand Down

0 comments on commit 8448063

Please sign in to comment.