-
Notifications
You must be signed in to change notification settings - Fork 45
/
course-definition.yml
3794 lines (2819 loc) · 135 KB
/
course-definition.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
slug: "redis"
name: "Build your own Redis"
short_name: "Redis"
release_status: "live"
description_md: |-
Redis is an in-memory data structure store often used as a database, cache, message broker and streaming engine. In this challenge
you'll build your own Redis server that is capable of serving basic commands, reading RDB files and more.
Along the way, you'll learn about TCP servers, the Redis Protocol and more.
short_description_md: |-
Learn about TCP servers, the Redis protocol and more
completion_percentage: 30
concept_slugs: ["network-protocols", "tcp-overview"]
languages:
- slug: "c"
- slug: "clojure"
- slug: "cpp"
- slug: "crystal"
- slug: "csharp"
- slug: "elixir"
- slug: "gleam"
- slug: "go"
- slug: "haskell"
- slug: "java"
- slug: "javascript"
- slug: "kotlin"
- slug: "ocaml"
- slug: "php"
- slug: "python"
- slug: "ruby"
- slug: "rust"
- slug: "typescript"
- slug: "scala"
- slug: "swift"
release_status: "alpha"
alpha_tester_usernames: ["JWShroyer"]
- slug: "zig"
marketing:
difficulty: medium
sample_extension_idea_title: "Persistence"
sample_extension_idea_description: "A Redis server that can read and write .rdb files"
testimonials:
- author_name: "Charles Guo"
author_description: "Software Engineer, Stripe"
author_avatar: "https://codecrafters.io/images/external/testimonials/charles-guo.png"
link: "https://github.com/shaldengeki"
text: |-
The Redis challenge was extremely fun. I ended up having to read the
Redis Protocol specification doc pretty carefully in its entirety! The result
felt like lightly-guided independent study, if that makes sense. (Which, again, was lots of fun)
- author_name: "Patrick Burris"
author_description: "Senior Software Developer, CenturyLink"
author_avatar: "https://codecrafters.io/images/external/testimonials/patrick-burris.jpeg"
link: "https://github.com/Jumballaya"
text: |-
I think the instant feedback right there in the git push is really cool.
Didn't even know that was possible!
extensions:
- slug: "persistence-rdb"
name: "RDB Persistence"
description_markdown: |
In this challenge extension you'll add [persistence][redis-persistence] support to your Redis implementation.
Along the way you'll learn about Redis's [RDB file format][rdb-file-format] and more.
[redis-persistence]: https://redis.io/docs/manual/persistence/
[rdb-file-format]: https://github.com/sripathikrishnan/redis-rdb-tools/blob/548b11ec3c81a603f5b321228d07a61a0b940159/docs/RDB_File_Format.textile
- slug: "replication"
name: "Replication"
description_markdown: |
In this challenge extension you'll add support for [Replication][redis-replication] to your Redis implementation.
Along the way you'll learn about how Redis's leader-follower replication works, the [PSYNC][redis-psync-command] command and more.
[redis-replication]: https://redis.io/docs/management/replication/
[redis-psync-command]: https://redis.io/commands/psync/
- slug: "streams"
name: "Streams"
description_markdown: |-
In this challenge extension you'll add support for the [Stream][redis-streams-data-type] data type to your Redis implementation.
Along the way you'll learn about commands like [XADD][xadd-command], [XRANGE][xrange-command] and more.
[redis-streams-data-type]: https://redis.io/docs/data-types/streams/
[xadd-command]: https://redis.io/commands/xadd/
[xrange-command]: https://redis.io/commands/xrange/
- slug: "transactions"
name: "Transactions"
description_markdown: |-
In this challenge extension you'll add support for [Transactions][redis-transactions] to your Redis implementation.
Along the way, you'll learn about the [MULTI][multi-command], [EXEC][exec-command], and [DISCARD][discard-command] commands, as well as how Redis handles transactions atomically.
[redis-transactions]: https://redis.io/docs/latest/develop/interact/transactions/
[multi-command]: https://redis.io/commands/multi/
[exec-command]: https://redis.io/commands/exec/
[discard-command]: https://redis.io/commands/discard/
stages:
- slug: "jm1"
concept_slugs:
[
"network-protocols",
"tcp-overview",
"go-tcp-server",
"rust-tcp-server",
"python-tcp-server",
]
name: "Bind to a port"
description_md: |-
In this stage, you'll implement a TCP server that listens on port 6379.
[TCP](https://en.wikipedia.org/wiki/Transmission_Control_Protocol) is the underlying protocol used by protocols like HTTP, SSH and others
you're probably familiar with. Redis clients & servers use TCP to communicate with each other.
Don't worry if you're unfamiliar with the TCP protocol, or what Redis clients & servers are. You'll learn more about this in the
next stages.
### Tests
The tester will execute your program like this:
```bash
$ ./your_program.sh
```
It'll then try to connect to your TCP server on port 6379. If the connection succeeds, you'll pass this stage.
### Notes
- 6379 is the default port that Redis uses.
- If you already have a Redis server running on your machine and listening on port 6379, you'll see a "port already in use" error when running your code. Try stopping the existing Redis server and running your code again.
{{#reader_is_bot}}
- In this stage, you can assume that you only need to handle a single client. We'll get to handling multiple clients & multiple requests per client in later stages.
{{/reader_is_bot}}
difficulty: very_easy
marketing_md: |-
In this stage, you'll start a TCP server on port 6379, which is the
default port that Redis uses.
tester_source_code_url: "https://github.com/codecrafters-io/redis-tester/blob/a58b9d58b33870fe26a164c0e323f809275a7250/internal/test_bind.go#L11"
- slug: "rg2"
concept_slugs:
[
"network-protocols",
"tcp-overview",
"go-tcp-server",
"rust-tcp-server",
"python-tcp-server",
]
name: "Respond to PING"
difficulty: easy
description_md: |-
In this stage, you'll implement support for the [PING](https://redis.io/commands/ping) command.
Redis clients communicate with Redis servers by sending "[commands](https://redis.io/commands/)". For each command, a Redis server sends a response back to the client.
Commands and responses are both encoded using the [Redis protocol](https://redis.io/topics/protocol) (we'll learn more about this in later stages).
[PING](https://redis.io/commands/ping/) is one of the simplest Redis commands. It's used to check whether a Redis server is healthy.
The response for the `PING` command is `+PONG\r\n`. This is the string "PONG" encoded using the [Redis protocol](https://redis.io/docs/reference/protocol-spec/).
In this stage, we'll cut corners by ignoring client input and hardcoding `+PONG\r\n` as a response. We'll learn to parse client input in later stages.
### Tests
The tester will execute your program like this:
```bash
$ ./your_program.sh
```
It'll then send a `PING` command to your server and expect a `+PONG\r\n` response.
```bash
$ redis-cli PING
```
Your server should respond with `+PONG\r\n`, which is "PONG" encoded as a [RESP simple string](https://redis.io/docs/reference/protocol-spec/#resp-simple-strings).
### Notes
- You can ignore the data that the tester sends you for this stage. We'll get to parsing
client input in later stages. For now, you can just hardcode `+PONG\r\n` as the response.
- You can also ignore handling multiple clients and handling multiple PING commands in the stage, we'll get to that in later stages.
- The exact bytes your program will receive won't be just `PING`, you'll receive something like this: `*1\r\n$4\r\nPING\r\n`,
which is the Redis protocol encoding of the `PING` command. We'll learn more about this in later stages.
marketing_md: |
In this stage, you'll respond to the
[PING](https://redis.io/commands/ping) command. You'll use [the Redis
protocol](https://redis.io/topics/protocol) to encode the reply.
tester_source_code_url: "https://github.com/codecrafters-io/redis-tester/blob/a58b9d58b33870fe26a164c0e323f809275a7250/internal/test_ping_pong.go#L9"
- slug: "wy1"
concept_slugs:
[
"network-protocols",
"tcp-overview",
"go-tcp-server",
"rust-tcp-server",
"python-tcp-server",
]
name: "Respond to multiple PINGs"
difficulty: easy
description_md: |-
In this stage, you'll respond to multiple
[PING](https://redis.io/commands/ping) commands sent by the same connection.
A Redis server starts to listen for the next command as soon as it's done responding to the previous one. This allows
Redis clients to send multiple commands using the same connection.
### Tests
The tester will execute your program like this:
```bash
$ ./your_program.sh
```
It'll then send multiple PING commands using the same connection. For example, it might send:
```bash
$ echo -e "PING\nPING" | redis-cli
```
The tester will expect to receive multiple `+PONG\r\n` responses (one for each command sent).
{{#lang_is_javascript}}
In most languages, you'd need to run a loop that reads input from a connection and sends a
response back. In JavaScript however, if you're listening to the
[`data`](https://nodejs.org/api/net.html#net_event_data) event, this should be automatically handled for you. **It
is very likely that the code you had for the previous stage will pass this stage without any changes!**
{{/lang_is_javascript}}
{{^lang_is_javascript}}
You'll need to run a loop that reads input from a connection and sends a
response back.
{{/lang_is_javascript}}
### Notes
- Just like the previous stage, you can hardcode `+PONG\r\n` as the response for this stage. We'll get to parsing
client input in later stages.
- The PING commands will be sent using the same connection. We'll get to handling multiple connections in later stages.
marketing_md: |-
In this stage, you'll respond to multiple
[PING](https://redis.io/commands/ping) commands sent by the same client.
tester_source_code_url: "https://github.com/codecrafters-io/redis-tester/blob/a58b9d58b33870fe26a164c0e323f809275a7250/internal/test_ping_pong.go#L35"
- slug: "zu2"
concept_slugs:
[
"network-protocols",
"tcp-overview",
"go-tcp-server",
"rust-tcp-server",
"python-tcp-server",
]
name: "Handle concurrent clients"
difficulty: medium
description_md: |-
In this stage, you'll add support for multiple concurrent clients.
In addition to handling multiple commands from the same client, Redis servers are also designed to handle multiple clients at once.
{{#lang_is_javascript}}
In most languages, you'd need to either use threads or implement an
[Event Loop](https://en.wikipedia.org/wiki/Event_loop) to do this. In JavaScript however, since [the concurrency
model itself is based on an event loop](https://developer.mozilla.org/en-US/docs/Web/JavaScript/EventLoop), most
standard library functions are designed to support this kind of concurrent behaviour out of the box. **It is very
likely that the code you had for the previous stage will pass this stage without any changes!**
{{/lang_is_javascript}}
{{^lang_is_javascript}}
To implement this, you'll need to either use threads, or, if you're feeling
adventurous, an [Event Loop](https://en.wikipedia.org/wiki/Event_loop) (like
the official Redis implementation does).
{{/lang_is_javascript}}
### Tests
The tester will execute your program like this:
```bash
$ ./your_program.sh
```
It'll then send two PING commands concurrently using two different connections:
```bash
# These two will be sent concurrently so that we test your server's ability to handle concurrent clients.
$ redis-cli PING
$ redis-cli PING
```
The tester will expect to receive two `+PONG\r\n` responses.
### Notes
- Since the tester client _only_ sends the PING command at the moment, it's okay to
ignore what the client sends and hardcode a response. We'll get to parsing
client input in later stages.
marketing_md: |-
In this stage, you'll add support for multiple concurrent clients to your
Redis server. To achieve this you'll use an [Event
Loop](https://en.wikipedia.org/wiki/Event_loop),
like the official Redis implementation does.
tester_source_code_url: "https://github.com/codecrafters-io/redis-tester/blob/a58b9d58b33870fe26a164c0e323f809275a7250/internal/test_ping_pong.go#L56"
- slug: "qq0"
name: "Implement the ECHO command"
difficulty: medium
description_md: |-
In this stage, you'll add support for the [ECHO](https://redis.io/commands/echo) command.
`ECHO` is a command like `PING` that's used for testing and debugging. It accepts a single argument and returns it back as a
RESP bulk string.
```bash
$ redis-cli PING # The command you implemented in previous stages
PONG
$ redis-cli ECHO hey # The command you'll implement in this stage
hey
```
### Tests
The tester will execute your program like this:
```bash
$ ./your_program.sh
```
It'll then send an `ECHO` command with an argument to your server:
```bash
$ redis-cli ECHO hey
```
The tester will expect to receive `$3\r\nhey\r\n` as a response (that's the string `hey` encoded as a [RESP bulk string](https://redis.io/docs/reference/protocol-spec/#bulk-strings).
### Notes
- We suggest that you implement a proper Redis protocol parser in this stage. It'll come in handy in later stages.
- Redis command names are case-insensitive, so `ECHO`, `echo` and `EcHo` are all valid commands.
- The tester will send a random string as an argument to the `ECHO` command, so you won't be able to hardcode the response to pass this stage.
- The exact bytes your program will receive won't be just `ECHO hey`, you'll receive something like this: `*2\r\n$4\r\nECHO\r\n$3\r\nhey\r\n`. That's
`["ECHO", "hey"]` encoded using the [Redis protocol](https://redis.io/docs/reference/protocol-spec/).
- You can read more about how "commands" are handled in the Redis protocol [here](https://redis.io/docs/reference/protocol-spec/#sending-commands-to-a-redis-server).
marketing_md: |-
In this stage, you'll respond to the
[ECHO](https://redis.io/commands/echo) command. You'll parse user input
according to the [the Redis protocol
specification](https://redis.io/topics/protocol).
tester_source_code_url: "https://github.com/codecrafters-io/redis-tester/blob/a58b9d58b33870fe26a164c0e323f809275a7250/internal/test_echo.go#L11"
- slug: "la7"
name: "Implement the SET & GET commands"
difficulty: medium
description_md: |-
In this stage, you'll add support for the [SET](https://redis.io/commands/set) &
[GET](https://redis.io/commands/get) commands.
The `SET` command is used to set a key to a value. The `GET` command is used to retrieve the value of a key.
```bash
$ redis-cli SET foo bar
OK
$ redis-cli GET foo
bar
```
The `SET` command supports a number of extra options like `EX` (expiry time in seconds), `PX` (expiry time in milliseconds) and more. We
won't cover these extra options in this stage. We'll get to them in later stages.
### Tests
The tester will execute your program like this:
```bash
./your_program.sh
```
It'll then send a `SET` command to your server:
```bash
$ redis-cli SET foo bar
```
The tester will expect to receive `+OK\r\n` as a response (that's the string `OK` encoded as a [RESP simple string](https://redis.io/docs/reference/protocol-spec/#resp-simple-strings)).
This command will be followed by a `GET` command:
```bash
$ redis-cli GET foo
```
The tester will expect to receive `$3\r\nbar\r\n` as a response (that's the string `bar` encoded as a [RESP bulk string](https://redis.io/docs/reference/protocol-spec/#bulk-strings).
### Notes
- If you implemented a proper Redis protocol parser in the previous stage, you should be able to reuse it in this stage.
- Just like the previous stage, the values used for keys and values will be random, so you won't be able to hardcode the response to pass this stage.
- If a key doesn't exist, the `GET` command should return a "null bulk string" (`$-1\r\n`). We won't explicitly test this in this stage, but you'll need it for the next stage (expiry).
marketing_md: |-
In this stage, you'll need to implement the
[SET](https://redis.io/commands/set) &
[GET](https://redis.io/commands/get) commands.
tester_source_code_url: "https://github.com/codecrafters-io/redis-tester/blob/a58b9d58b33870fe26a164c0e323f809275a7250/internal/test_get_set.go#L11"
- slug: "yz1"
name: "Expiry"
difficulty: medium
description_md: |-
In this stage, you'll add support for setting a key with an expiry.
The expiry for a key can be provided using the "PX" argument to the [SET](https://redis.io/commands/set) command. The expiry is provided in milliseconds.
```bash
$ redis-cli SET foo bar px 100 # Sets the key "foo" to "bar" with an expiry of 100 milliseconds
OK
```
After the key has expired, a `GET` command for that key should return a "null bulk string" (`$-1\r\n`).
{{#lang_is_haskell}}
The [time](https://hackage.haskell.org/package/time) package is available
to use as a dependency.
{{/lang_is_haskell}}
### Tests
The tester will execute your program like this:
```bash
$ ./your_program.sh
```
It'll then send a `SET` command to your server to set a key with an expiry:
```bash
$ redis-cli SET foo bar px 100
```
It'll then immediately send a `GET` command to retrieve the value:
```bash
$ redis-cli GET foo
```
It'll expect the response to be `bar` (encoded as a RESP bulk string).
It'll then wait for the key to expire and send another `GET` command:
```bash
$ sleep 0.2 && redis-cli GET foo
```
It'll expect the response to be `$-1\r\n` (a "null bulk string").
### Notes
- Just like command names, command arguments are also case-insensitive. So `PX`, `px` and `pX` are all valid.
- The keys, values and expiry times used in the tests will be random, so you won't be able to hardcode a response to pass this stage.
marketing_md: |-
In this stage, you'll add support for setting a key with an expiry. The
expiry is provided using the "PX" argument to the
[SET](https://redis.io/commands/set) command.
tester_source_code_url: "https://github.com/codecrafters-io/redis-tester/blob/master/internal/test_expiry.go"
# Persistence
- slug: "zg5"
primary_extension_slug: "persistence-rdb"
name: "RDB file config"
difficulty: easy
description_md: |
Welcome to the RDB Persistence Extension! In this extension, you'll add support for reading [RDB files](https://redis.io/docs/management/persistence/) (Redis Database files).
In this stage, you'll add support for two configuration parameters related to RDB persistence, as well as the [CONFIG GET](https://redis.io/docs/latest/commands/config-get/) command.
### RDB files
An RDB file is a point-in-time snapshot of a Redis dataset. When RDB persistence is enabled, the Redis server syncs its in-memory state with an RDB file, by doing the following:
1. On startup, the Redis server loads the data from the RDB file.
2. While running, the Redis server periodically takes new snapshots of the dataset, in order to update the RDB file.
### `dir` and `dbfilename`
The configuration parameters `dir` and `dbfilename` specify where an RDB file is stored:
- `dir` - the path to the directory where the RDB file is stored (example: `/tmp/redis-data`)
- `dbfilename` - the name of the RDB file (example: `rdbfile`)
### The `CONFIG GET` command
The [`CONFIG GET`](https://redis.io/docs/latest/commands/config-get/) command returns the values of configuration parameters.
It takes in one or more configuration parameters and returns a [RESP array](https://redis.io/docs/latest/develop/reference/protocol-spec/#arrays) of key-value pairs:
```bash
$ redis-cli CONFIG GET dir
1) "dir"
2) "/tmp/redis-data"
```
Although `CONFIG GET` can fetch multiple parameters at a time, the tester will only send `CONFIG GET` commands with one parameter at a time.
### Tests
The tester will execute your program like this:
```bash
./your_program.sh --dir /tmp/redis-files --dbfilename dump.rdb
```
It'll then send the following commands:
```bash
$ redis-cli CONFIG GET dir
$ redis-cli CONFIG GET dbfilename
```
Your server must respond to each `CONFIG GET` command with a RESP array containing two elements:
1. The parameter **name**, encoded as a [RESP Bulk string](https://redis.io/docs/latest/develop/reference/protocol-spec/#bulk-strings)
2. The parameter **value**, encoded as a RESP Bulk string
For example, if the value of `dir` is `/tmp/redis-files`, then the expected response to `CONFIG GET dir` is:
```bash
*2\r\n$3\r\ndir\r\n$16\r\n/tmp/redis-files\r\n
```
### Notes
- You don't need to read the RDB file in this stage, you only need to store `dir` and `dbfilename`. Reading from the file will be covered in later stages.
- If your repository was created before 5th Oct 2023, it's possible that your `./your_program.sh` script is not passing arguments to your program. To fix this, you'll need to edit `./your_program.sh`. Check the [update CLI args PR](https://github.com/codecrafters-io/build-your-own-redis/pull/89/files) for details on how to do this.
marketing_md: |
In this stage, you'll add support for reading the config values related to where RDB files are stored. You'll implement the `CONFIG GET` command.
- slug: "jz6"
primary_extension_slug: "persistence-rdb"
name: "Read a key"
difficulty: medium
description_md: |
In this stage, you'll add support for reading a single key from an RDB file.
### RDB file format
<details>
<summary>Click to expand/collapse</summary>
#### RDB file format overview
Here are the different sections of the RDB file, in order:
1. Header section
2. Metadata section
3. Database section
4. End of file section
RDB files use special encodings to store different types of data. The ones relevant to this stage are "size encoding" and "string encoding." These are explained near the end of this page.
The following breakdown of the RDB file format is based on [Redis RDB File Format](https://rdb.fnordig.de/file_format.html) by Jan-Erik Rediger. We’ve only included the parts that are relevant to this stage.
#### Header section
RDB files begin with a header section, which looks something like this:
```
52 45 44 49 53 30 30 31 31 // Magic string + version number (ASCII): "REDIS0011".
```
The header contains the magic string `REDIS`, followed by a four-character RDB version number. In this challenge, the test RDB files all use version 11. So, the header is always `REDIS0011`.
#### Metadata section
Next is the metadata section. It contains zero or more "metadata subsections," which each specify a single metadata attribute. Here's an example of a metadata subsection that specifies `redis-ver`:
```
FA // Indicates the start of a metadata subsection.
09 72 65 64 69 73 2D 76 65 72 // The name of the metadata attribute (string encoded): "redis-ver".
06 36 2E 30 2E 31 36 // The value of the metadata attribute (string encoded): "6.0.16".
```
The metadata name and value are always string encoded.
#### Database section
Next is the database section. It contains zero or more "database subsections," which each describe a single database. Here's an example of a database subsection:
```
FE // Indicates the start of a database subsection.
00 /* The index of the database (size encoded).
Here, the index is 0. */
FB // Indicates that hash table size information follows.
03 /* The size of the hash table that stores the keys and values (size encoded).
Here, the total key-value hash table size is 3. */
02 /* The size of the hash table that stores the expires of the keys (size encoded).
Here, the number of keys with an expiry is 2. */
```
```
00 /* The 1-byte flag that specifies the value’s type and encoding.
Here, the flag is 0, which means "string." */
06 66 6F 6F 62 61 72 // The name of the key (string encoded). Here, it's "foobar".
06 62 61 7A 71 75 78 // The value (string encoded). Here, it's "bazqux".
```
```
FC /* Indicates that this key ("foo") has an expire,
and that the expire timestamp is expressed in milliseconds. */
15 72 E7 07 8F 01 00 00 /* The expire timestamp, expressed in Unix time,
stored as an 8-byte unsigned long, in little-endian (read right-to-left).
Here, the expire timestamp is 1713824559637. */
00 // Value type is string.
03 66 6F 6F // Key name is "foo".
03 62 61 72 // Value is "bar".
```
```
FD /* Indicates that this key ("baz") has an expire,
and that the expire timestamp is expressed in seconds. */
52 ED 2A 66 /* The expire timestamp, expressed in Unix time,
stored as an 4-byte unsigned integer, in little-endian (read right-to-left).
Here, the expire timestamp is 1714089298. */
00 // Value type is string.
03 62 61 7A // Key name is "baz".
03 71 75 78 // Value is "qux".
```
Here's a more formal description of how each key-value pair is stored:
1. Optional expire information (one of the following):
* Timestamp in seconds:
1. `FD`
2. Expire timestamp in seconds (4-byte unsigned integer)
* Timestamp in milliseconds:
1. `FC`
2. Expire timestamp in milliseconds (8-byte unsigned long)
2. Value type (1-byte flag)
3. Key (string encoded)
4. Value (encoding depends on value type)
#### End of file section
This section marks the end of the file. It looks something like this:
```
FF /* Indicates that the file is ending,
and that the checksum follows. */
89 3b b7 4e f8 0f 77 19 // An 8-byte CRC64 checksum of the entire file.
```
#### Size encoding
Size-encoded values specify the size of something. Here are some examples:
- The database indexes and hash table sizes are size encoded.
- String encoding begins with a size-encoded value that specifies the number of characters in the string.
- List encoding begins with a size-encoded value that specifies the number of elements in the list.
The first two bits of a size-encoded value indicate how the value should be parsed. Here's a guide (bits are shown in both hexadecimal and binary):
```
/* If the first two bits are 0b00:
The size is the remaining 6 bits of the byte.
In this example, the size is 10: */
0A
00001010
/* If the first two bits are 0b01:
The size is the next 14 bits
(remaining 6 bits in the first byte, combined with the next byte),
in big-endian (read left-to-right).
In this example, the size is 700: */
42 BC
01000010 10111100
/* If the first two bits are 0b10:
Ignore the remaining 6 bits of the first byte.
The size is the next 4 bytes, in big-endian (read left-to-right).
In this example, the size is 17000: */
80 00 00 42 68
10000000 00000000 00000000 01000010 01101000
/* If the first two bits are 0b11:
The remaining 6 bits specify a type of string encoding.
See string encoding section. */
```
#### String encoding
A string-encoded value consists of two parts:
1. The size of the string (size encoded).
2. The string.
Here's an example:
```
/* The 0x0D size specifies that the string is 13 characters long.
The remaining characters spell out "Hello, World!". */
0D 48 65 6C 6C 6F 2C 20 57 6F 72 6C 64 21
```
For sizes that begin with `0b11`, the remaining 6 bits indicate a type of string format:
```
/* The 0xC0 size indicates the string is an 8-bit integer.
In this example, the string is "123". */
C0 7B
/* The 0xC1 size indicates the string is a 16-bit integer.
The remaining bytes are in little-endian (read right-to-left).
In this example, the string is "12345". */
C1 39 30
/* The 0xC2 size indicates the string is a 32-bit integer.
The remaining bytes are in little-endian (read right-to-left),
In this example, the string is "1234567". */
C2 87 D6 12 00
/* The 0xC3 size indicates that the string is compressed with the LZF algorithm.
You will not encounter LZF-compressed strings in this challenge. */
C3 ...
```
</details>
### The `KEYS` command
<details>
<summary>Click to expand/collapse</summary>
The [`KEYS command`](https://redis.io/docs/latest/commands/keys/) returns all the keys that match a given pattern, as a RESP array:
```
$ redis-cli SET foo bar
OK
$ redis-cli SET baz qux
OK
$ redis-cli KEYS "f*"
1) "foo"
```
When the pattern is `*`, the command returns all the keys in the database:
```
$ redis-cli KEYS "*"
1) "baz"
2) "foo"
```
In this stage, you must add support for the `KEYS` command. However, you only need to support the `*` pattern.
</details>
### Tests
The tester will create an RDB file with a single key and execute your program like this:
```
$ ./your_program.sh --dir <dir> --dbfilename <filename>
```
It'll then send a `KEYS "*"` command to your server.
```
$ redis-cli KEYS "*"
```
Your server must respond with a RESP array that contains the key from the RDB file:
```
*1\r\n$3\r\nfoo\r\n
```
### Notes
- The RDB file provided by `--dir` and `--dbfilename` might not exist. If the file doesn't exist, your program must treat the database as empty.
- RDB files use both little-endian and big-endian to store numbers. See the [MDN article on endianness](https://developer.mozilla.org/en-US/docs/Glossary/Endianness) to learn more.
- To generate an RDB file, use the [`SAVE` command](https://redis.io/docs/latest/commands/save/).
marketing_md: |
In this stage, you'll add support for reading a key from an RDB file that contains a single key-value pair. You'll do this by implementing the `KEYS *` command.
- slug: "gc6"
primary_extension_slug: "persistence-rdb"
name: "Read a string value"
difficulty: medium
description_md: |
In this stage, you'll add support for reading the value corresponding to a key from an RDB file.
Just like with the previous stage, we'll stick to supporting RDB files that contain a single key for now.
The tester will create an RDB file with a single key and execute your program like this:
```
./your_program.sh --dir <dir> --dbfilename <filename>
```
It'll then send a `GET <key>` command to your server.
```bash
$ redis-cli GET "foo"
```
The response to `GET <key>` should be a RESP bulk string with the value of the key.
For example, let's say the RDB file contains a key called `foo` with the value `bar`. The expected response will be `$3\r\nbar\r\n`.
Strings can be encoded in three different ways in the RDB file format:
- Length-prefixed strings
- Integers as strings
- Compressed strings
In this stage, you only need to support length-prefixed strings. We won't cover the other two types in this challenge.
We recommend using [this blog post](https://rdb.fnordig.de/file_format.html) as a reference when working on this stage.
marketing_md: |
In this stage, you'll add support for reading the value of a key from an RDB file that contains a single key-value pair.
- slug: "jw4"
primary_extension_slug: "persistence-rdb"
name: "Read multiple keys"
difficulty: medium
description_md: |
In this stage, you'll add support for reading multiple keys from an RDB file.
The tester will create an RDB file with multiple keys and execute your program like this:
```bash
$ ./your_program.sh --dir <dir> --dbfilename <filename>
```
It'll then send a `KEYS *` command to your server.
```bash
$ redis-cli KEYS "*"
```
The response to `KEYS *` should be a RESP array with the keys as elements.
For example, let's say the RDB file contains two keys: `foo` and `bar`. The expected response will be:
```
*2\r\n$3\r\nfoo\r\n$3\r\nbar\r\n
```
- `*2\r\n` indicates that the array has two elements
- `$3\r\nfoo\r\n` indicates that the first element is a bulk string with the value `foo`
- `$3\r\nbar\r\n` indicates that the second element is a bulk string with the value `bar`
marketing_md: |
In this stage, you'll add support for reading multiple keys from an RDB file. You'll do this by extending the `KEYS *` command to support multiple keys.
- slug: "dq3"
primary_extension_slug: "persistence-rdb"
name: "Read multiple string values"
difficulty: medium
description_md: |
In this stage, you'll add support for reading multiple string values from an RDB file.
The tester will create an RDB file with multiple keys and execute your program like this:
```bash
$ ./your_program.sh --dir <dir> --dbfilename <filename>
```
It'll then send multiple `GET <key>` commands to your server.
```bash
$ redis-cli GET "foo"
$ redis-cli GET "bar"
```
The response to each `GET <key>` command should be a RESP bulk string with the value corresponding to the key.
marketing_md: |
In this stage, you'll add support for reading multiple string values from an RDB file.
- slug: "sm4"
primary_extension_slug: "persistence-rdb"
name: "Read value with expiry"
difficulty: medium
description_md: |
In this stage, you'll add support for reading values that have an expiry set.
The tester will create an RDB file with multiple keys. Some of these keys will have an expiry set, and some won't. The expiry timestamps
will also be random, some will be in the past and some will be in the future.
The tester will execute your program like this:
```bash
$ ./your_program.sh --dir <dir> --dbfilename <filename>
```
It'll then send multiple `GET <key>` commands to your server.
```bash
$ redis-cli GET "foo"
$ redis-cli GET "bar"
```
When a key has expired, the expected response is `$-1\r\n` (a "null bulk string").
When a key hasn't expired, the expected response is a RESP bulk string with the value corresponding to the key.
marketing_md: |
In this stage, you'll add support for reading values that have an expiry set.
# Replication
- slug: "bw1"
primary_extension_slug: "replication"
name: "Configure listening port"
difficulty: easy
description_md: |
Welcome to the Replication extension!
In this extension, you'll extend your Redis server to support [leader-follower replication](https://redis.io/docs/management/replication/). You'll be able to run
multiple Redis servers with one acting as the "master" and the others as "replicas". Changes made to the master will be automatically replicated to replicas.
Since we'll need to run multiple instances of your Redis server at once, we can't run all of them on port 6379.
In this stage, you'll add support for starting the Redis server on a custom port. The port number will be passed to your program via the `--port` flag.
### Tests
The tester will execute your program like this:
```
./your_program.sh --port 6380
```
It'll then try to connect to your TCP server on the specified port number (`6380` in the example above). If the connection succeeds, you'll pass this stage.
### Notes
- Your program still needs to pass the previous stages, so if `--port` isn't specified, you should default to port 6379.
- The tester will pass a random port number to your program, so you can't hardcode the port number from the example above.
- If your repository was created before 5th Oct 2023, it's possible that your `./your_program.sh` script
might not be passing arguments on to your program. You'll need to edit `./your_program.sh` to fix this, check
[this PR](https://github.com/codecrafters-io/build-your-own-redis/pull/89/files) for details.
marketing_md: |
In this stage, you'll add support for parsing the `--port` flag and starting Redis on a custom port.
- slug: "ye5"
primary_extension_slug: "replication"
name: "The INFO command"
difficulty: easy
description_md: |
In this stage, you'll add support for the [INFO](https://redis.io/commands/info/) command.
The `INFO` command returns information and statistics about a Redis server. In this stage, we'll add support for the
`replication` section of the `INFO` command.
### The replication section
When you run the `INFO` command against a Redis server, you'll see something like this:
```
$ redis-cli INFO replication
# Replication
role:master
connected_slaves:0
master_replid:8371b4fb1155b71f4a04d3e1bc3e18c4a990aeeb
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:
```
The reply to this command is a [Bulk string](https://redis.io/docs/reference/protocol-spec/#bulk-strings) where each line is a key value pair, separated by ":".
Here are what some of the important fields mean:
- `role`: The role of the server (`master` or `slave`)
- `connected_slaves`: The number of connected replicas
- `master_replid`: The replication ID of the master (we'll get to this in later stages)
- `master_repl_offset`: The replication offset of the master (we'll get to this in later stages)
In this stage, you'll only need to support the `role` key. We'll add support for other keys in later stages.
### Tests
The tester will execute your program like this:
```
./your_program.sh --port <PORT>
```
It'll then send the `INFO` command with `replication` as an argument.
```bash
$ redis-cli -p <PORT> info replication
```
Your program should respond with a [Bulk string](https://redis.io/docs/reference/protocol-spec/#bulk-strings) where each line
is a key value pair separated by `:`. The tester will only look for the `role` key, and assert that the value is `master`.
### Notes
- In the response for the `INFO` command, you only need to support the `role` key for this stage. We'll add support for the other keys in later stages.
- The `# Replication` heading in the response is optional, you can ignore it.