This repository has been archived by the owner on Apr 19, 2022. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 1
/
index.xml
1161 lines (1161 loc) · 201 KB
/
index.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>DevStream – DevStream blog</title><link>https://www.devstream.io/</link><description>Recent content in DevStream blog on DevStream</description><generator>Hugo -- gohugo.io</generator><atom:link href="https://www.devstream.io/index.xml" rel="self" type="application/rss+xml"/><item><title>Blog: Creating a Local Kubernetes Cluster from the Ground Up</title><link>https://www.devstream.io/blog/creating-a-local-kubernetes-cluster-from-the-ground-up/</link><pubDate>Mon, 18 Apr 2022 00:00:00 +0000</pubDate><guid>https://www.devstream.io/blog/creating-a-local-kubernetes-cluster-from-the-ground-up/</guid><description>
<img src="https://www.devstream.io/blog/creating-a-local-kubernetes-cluster-from-the-ground-up/featured-background_hu21989ddc8b66511e4b50f875ce8b0608_6786932_640x0_resize_q75_catmullrom.jpg" width="640" height="480"/>
<div class="card rounded p-2 td-post-card mb-4 mt-4" style="max-width: 1210px">
<img class="card-img-top" src="https://www.devstream.io/blog/creating-a-local-kubernetes-cluster-from-the-ground-up/featured-background_hu21989ddc8b66511e4b50f875ce8b0608_6786932_1200x800_fill_q75_catmullrom_smart1.jpg" width="1200" height="800">
<div class="card-body px-0 pt-2 pb-0">
<p class="card-text">
Miyuan Cafe, Qinglong Lake Park, Chengdu, China
<small class="text-muted"><br/>Photo: Daniel Hu / CC-BY-CA</small></p>
</div>
</div>
<hr>
<h1 id="creating-a-local-kubernetes-cluster-from-the-groud-up---a-tutorial-of-kind">Creating a Local Kubernetes Cluster from the Groud Up - a Tutorial of <code>kind</code></h1>
<p>From the ground up? Yep, from the ground up!</p>
<hr>
<h2 id="overview">Overview</h2>
<p>Creating a Kubernetes cluster can be tricky.</p>
<p>There are multiple tools designed just for that job. There are even companies that provide &ldquo;installing K8s&rdquo; as a service. And, to create a production-ready Kubernetes cluster with all the best practices in mind requires detailed designing and planning.</p>
<p>So, the scope of this article isn&rsquo;t to help you to create a &ldquo;production-ready&rdquo; cluster.</p>
<p>However, after reading this article, you should be able to create a local &ldquo;testing&rdquo; Kubernetes cluster, and for the developing and testing of DevStream, it should more than suffice.</p>
<p>Even for a local testing cluster, there are multiple tools you can choose from. For example, there is <a href="https://minikube.sigs.k8s.io/docs/start/"><code>minikube</code></a>, and there is <a href="https://kind.sigs.k8s.io/"><code>kind</code></a>. <code>kind</code> is a tool for running local Kubernetes clusters using Docker container “nodes”. <code>kind</code> was primarily designed for testing Kubernetes.</p>
<p>In this article, we are going with <code>kind</code>. We are not opinionated; we are not saying that <code>kind</code> is better than <code>minikube</code> or vice versa; we are merely choosing a tool to get the job done. If you are more familiar with other tools, it&rsquo;s completely fine!</p>
<p>This article uses macOS as our local development environment. If you are using Windows or Linux, you can still read this post to get a general idea and achieve the same.</p>
<hr>
<h2 id="install-docker">Install Docker</h2>
<p>Docker works in a way using Linux&rsquo;s Namespace and Cgroup. It&rsquo;s quite easy to install Docker on Linux. On macOS and Windows, Docker runs with virtualization. However, we do not need to worry too much detail here, because it&rsquo;s quite simple to download and run Docker Desktop.</p>
<p>Go to <a href="https://www.docker.com/products/docker-desktop">https://www.docker.com/products/docker-desktop</a>, find the correct version of Docker Desktop (Intel/amd64, or M1/arm64):</p>
<p><img src="./a.png" alt="docker download"></p>
<p>Double click on the <code>Docker.dmg</code> file, and we see the installation interface like the following:</p>
<p><img src="./b.png" alt="docker install"></p>
<p>Simply drag &ldquo;Docker&rdquo; to our &ldquo;Applications,&rdquo; and within a few seconds, it&rsquo;s done! We can start it from the Launchpad:</p>
<p><img src="./c.png" alt="docker logo"></p>
<p>Wait a few seconds and we can see the starting page:</p>
<p><img src="./d.png" alt="docker started"></p>
<p>Click the “gear&quot; ⚙️ icon to change settings about Docker Desktop. For example, if we need to run a lot of containers, we might need to increase the memory. Here, we changed the memory to 4.00 GB:</p>
<p><img src="./e.png" alt="docker setup"></p>
<p>Remember to &ldquo;Apply &amp; Restart&rdquo; to ensure the changes are effective.</p>
<hr>
<h2 id="introduction-to-kind">Introduction to <code>kind</code></h2>
<p><code>kind</code> (Kubernetes-in-docker) uses a Docker container as a &ldquo;node&rdquo; to deploy Kubernetes. It&rsquo;s mainly used for testing Kubernetes itself.</p>
<p><code>kind</code> is simple, containing a command-line tool named <code>kind</code> and a Docker image which has Kubernetes and <code>systemd</code>. <code>kind</code> uses Docker on the host machine to create a container, which runs <code>systemd</code>, which in turn runs the container runtime, <code>kubelet</code>, and other Kubernetes components. So, we end up with a whole Kubernetes cluster in one container.</p>
<p>Note that although in the explanation above, the Cluster is only a single node cluster, it&rsquo;s possible to create a multi-node Kubernetes cluster.</p>
<hr>
<h2 id="creating-a-kubernetes-cluster-with-kind-at-the-click-of-a-button">Creating a Kubernetes Cluster with Kind at the Click of a Button</h2>
<ol>
<li>clone DevStream&rsquo;s repo: <a href="https://github.com/devstream-io/devstream">https://github.com/devstream-io/devstream</a></li>
<li>cd devstream; <code>make e2e-up</code></li>
</ol>
<p>That&rsquo;s it.</p>
<p>If you check out the Makefile, you will see it actually runs a shell script that runs <code>kind</code> to create the cluster.</p>
<p>However, you wouldn&rsquo;t be satisfied if we end this article right here, right now, would you.</p>
<p>So, let&rsquo;s have a deep dive into <code>kind</code>. Fasten your seat belt, because we are gonna fly!</p>
<hr>
<h2 id="creating-a-kubernetes-cluster-with-kind-at-two-clicks-of-a-button">Creating a Kubernetes Cluster with Kind at Two Clicks of a Button</h2>
<p>On GitHub, we can find the latest release of <code>kind</code>: <a href="https://github.com/kubernetes-sigs/kind/releases">https://github.com/kubernetes-sigs/kind/releases</a>.</p>
<p>Choose relatively new versions, and install:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-sh" data-lang="sh"><span style="display:flex;"><span><span style="color:#8f5902;font-style:italic"># method 1: download pre-compiled binary</span>
</span></span><span style="display:flex;"><span><span style="color:#204a87">cd</span> /tmp
</span></span><span style="display:flex;"><span>curl -Lo ./kind https://github.com/kubernetes-sigs/kind/releases/download/v0.12.0/kind-darwin-arm64
</span></span><span style="display:flex;"><span>chmod +x ./kind
</span></span><span style="display:flex;"><span>sudo mv kind /usr/local/bin/
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#8f5902;font-style:italic"># method 2: go get and compile locally</span>
</span></span><span style="display:flex;"><span>go get sigs.k8s.io/[email protected]
</span></span></code></pre></div><p>We can also download the Docker image beforehand. Here we choose v1.22 of Kubernetes:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-sh" data-lang="sh"><span style="display:flex;"><span>kindest/node:v1.22.0@sha256:b8bda84bb3a190e6e028b1760d277454a72267a5454b57db34437c34a588d047
</span></span></code></pre></div><p>Create cluster:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-sh" data-lang="sh"><span style="display:flex;"><span>kind create cluster --image<span style="color:#ce5c00;font-weight:bold">=</span>kindest/node:v1.22.0 --name<span style="color:#ce5c00;font-weight:bold">=</span>dev
</span></span></code></pre></div><p>Sample output:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-sh" data-lang="sh"><span style="display:flex;"><span>Creating cluster <span style="color:#4e9a06">&#34;dev&#34;</span> ...
</span></span><span style="display:flex;"><span> ✓ Ensuring node image <span style="color:#ce5c00;font-weight:bold">(</span>kindest/node:v1.22.0<span style="color:#ce5c00;font-weight:bold">)</span> 🖼
</span></span><span style="display:flex;"><span> ✓ Preparing nodes 📦
</span></span><span style="display:flex;"><span> ✓ Writing configuration 📜
</span></span><span style="display:flex;"><span> ✓ Starting control-plane 🕹️
</span></span><span style="display:flex;"><span> ✓ Installing CNI 🔌
</span></span><span style="display:flex;"><span> ✓ Installing StorageClass 💾
</span></span><span style="display:flex;"><span>Set kubectl context to <span style="color:#4e9a06">&#34;kind-dev&#34;</span>
</span></span><span style="display:flex;"><span>You can now use your cluster with:
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>kubectl cluster-info --context kind-dev
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>Have a question, bug, or feature request? Let us know! https://kind.sigs.k8s.io/#community 🙂
</span></span></code></pre></div><p>Follow the hints above, next, let&rsquo;s run <code>kubectl cluster-info --context kind-dev</code> to switch the context and make sure you are in the right Kubernetes context.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-sh" data-lang="sh"><span style="display:flex;"><span>$ kubectl get node
</span></span><span style="display:flex;"><span>NAME STATUS ROLES AGE VERSION
</span></span><span style="display:flex;"><span>dev-control-plane Ready control-plane,master 7m4s v1.22.0
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>$ kubectl get pod -n kube-system
</span></span><span style="display:flex;"><span>NAME READY STATUS RESTARTS AGE
</span></span><span style="display:flex;"><span>coredns-78fcd69978-hch75 1/1 Running <span style="color:#0000cf;font-weight:bold">0</span> 10m
</span></span><span style="display:flex;"><span>coredns-78fcd69978-ztqn4 1/1 Running <span style="color:#0000cf;font-weight:bold">0</span> 10m
</span></span><span style="display:flex;"><span>etcd-dev-control-plane 1/1 Running <span style="color:#0000cf;font-weight:bold">0</span> 10m
</span></span><span style="display:flex;"><span>kindnet-l8qxq 1/1 Running <span style="color:#0000cf;font-weight:bold">0</span> 10m
</span></span><span style="display:flex;"><span>kube-apiserver-dev-control-plane 1/1 Running <span style="color:#0000cf;font-weight:bold">0</span> 10m
</span></span><span style="display:flex;"><span>kube-controller-manager-dev-control-plane 1/1 Running <span style="color:#0000cf;font-weight:bold">0</span> 10m
</span></span><span style="display:flex;"><span>kube-proxy-mzfgc 1/1 Running <span style="color:#0000cf;font-weight:bold">0</span> 10m
</span></span><span style="display:flex;"><span>kube-scheduler-dev-control-plane 1/1 Running <span style="color:#0000cf;font-weight:bold">0</span> 10m
</span></span></code></pre></div><p>Now we have a local cluster for testing and developing Kubernetes.</p>
<hr>
<h2 id="creating-a-kubernetes-cluster-with-kind-at-three-clicks-of-a-button">Creating a Kubernetes Cluster with Kind at Three Clicks of a Button</h2>
<p>A minimum highly-available Kubernetes cluster is composed of 3 nodes. In this section, let&rsquo;s see how to create a multi-node, highly-available cluster locally using <code>kind</code>.</p>
<h3 id="kind-cluster-config"><code>kind</code> Cluster Config</h3>
<p>We can pass a config file to <code>kind</code> by using the <code>--config</code> parameter. Let&rsquo;s have a look at a <code>kind</code> config file:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#8f5902;font-style:italic"># this config file contains all config fields with comments</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span><span style="color:#8f5902;font-style:italic"># NOTE: this is not a particularly useful config file</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span><span style="color:#204a87;font-weight:bold">kind</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">Cluster</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span><span style="color:#204a87;font-weight:bold">apiVersion</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">kind.x-k8s.io/v1alpha4</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span><span style="color:#8f5902;font-style:italic"># patch the generated kubeadm config with some extra settings</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span><span style="color:#204a87;font-weight:bold">kubeadmConfigPatches</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span>- <span style="color:#000;font-weight:bold">|</span><span style="color:#8f5902;font-style:italic">
</span></span></span><span style="display:flex;"><span><span style="color:#8f5902;font-style:italic"> apiVersion: kubelet.config.k8s.io/v1beta1
</span></span></span><span style="display:flex;"><span><span style="color:#8f5902;font-style:italic"> kind: KubeletConfiguration
</span></span></span><span style="display:flex;"><span><span style="color:#8f5902;font-style:italic"> evictionHard:
</span></span></span><span style="display:flex;"><span><span style="color:#8f5902;font-style:italic"> nodefs.available: &#34;0%&#34;</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span><span style="color:#8f5902;font-style:italic"># patch it further using a JSON 6902 patch</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span><span style="color:#204a87;font-weight:bold">kubeadmConfigPatchesJSON6902</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span>- <span style="color:#204a87;font-weight:bold">group</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">kubeadm.k8s.io</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#204a87;font-weight:bold">version</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">v1beta2</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#204a87;font-weight:bold">kind</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">ClusterConfiguration</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#204a87;font-weight:bold">patch</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000;font-weight:bold">|</span><span style="color:#8f5902;font-style:italic">
</span></span></span><span style="display:flex;"><span><span style="color:#8f5902;font-style:italic"> - op: add
</span></span></span><span style="display:flex;"><span><span style="color:#8f5902;font-style:italic"> path: /apiServer/certSANs/-
</span></span></span><span style="display:flex;"><span><span style="color:#8f5902;font-style:italic"> value: my-hostname</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span><span style="color:#8f5902;font-style:italic"># 1 control plane node and 3 workers</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span><span style="color:#204a87;font-weight:bold">nodes</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span><span style="color:#8f5902;font-style:italic"># the control plane node config</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span>- <span style="color:#204a87;font-weight:bold">role</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">control-plane</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span><span style="color:#8f5902;font-style:italic"># the three workers</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span>- <span style="color:#204a87;font-weight:bold">role</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">worker</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span>- <span style="color:#204a87;font-weight:bold">role</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">worker</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span>- <span style="color:#204a87;font-weight:bold">role</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">worker</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span></code></pre></div><p>We can see that the config has two sections: the upper part being <code>kubeadm</code> related stuff and the lower part being nodes related settings. Apparently, the &ldquo;nodes&rdquo; part is where we are going to edit to achieve a multi-node cluster.</p>
<h3 id="1-control-plane-node-3-worker-nodes-cluster">1 Control Plane Node, 3 Worker Nodes Cluster</h3>
<p>Let&rsquo;s create a config named <code>multi-node-config.yaml</code> with the following content:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#204a87;font-weight:bold">kind</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">Cluster</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span><span style="color:#204a87;font-weight:bold">apiVersion</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">kind.x-k8s.io/v1alpha4</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span><span style="color:#204a87;font-weight:bold">nodes</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span>- <span style="color:#204a87;font-weight:bold">role</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">control-plane</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span>- <span style="color:#204a87;font-weight:bold">role</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">worker</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span>- <span style="color:#204a87;font-weight:bold">role</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">worker</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span>- <span style="color:#204a87;font-weight:bold">role</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">worker</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span></code></pre></div><p>Then run:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-sh" data-lang="sh"><span style="display:flex;"><span>$ kind create cluster --config multi-node-config.yaml <span style="color:#4e9a06">\
</span></span></span><span style="display:flex;"><span><span style="color:#4e9a06"></span> --image<span style="color:#ce5c00;font-weight:bold">=</span>kindest/node:v1.22.0 --name<span style="color:#ce5c00;font-weight:bold">=</span>dev4
</span></span></code></pre></div><p>We will get some output similar to the previous single-node cluster output, except for the &ldquo;Joining worker nodes&rdquo; part:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-sh" data-lang="sh"><span style="display:flex;"><span>Creating cluster <span style="color:#4e9a06">&#34;dev4&#34;</span> ...
</span></span><span style="display:flex;"><span> ✓ Ensuring node image <span style="color:#ce5c00;font-weight:bold">(</span>kindest/node:v1.22.0<span style="color:#ce5c00;font-weight:bold">)</span> 🖼
</span></span><span style="display:flex;"><span> ✓ Preparing nodes 📦 📦 📦 📦
</span></span><span style="display:flex;"><span> ✓ Writing configuration 📜
</span></span><span style="display:flex;"><span> ✓ Starting control-plane 🕹️
</span></span><span style="display:flex;"><span> ✓ Installing CNI 🔌
</span></span><span style="display:flex;"><span> ✓ Installing StorageClass 💾
</span></span><span style="display:flex;"><span> ✓ Joining worker nodes 🚜
</span></span><span style="display:flex;"><span>Set kubectl context to <span style="color:#4e9a06">&#34;kind-dev4&#34;</span>
</span></span><span style="display:flex;"><span>You can now use your cluster with:
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>kubectl cluster-info --context kind-dev4
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>Thanks <span style="color:#204a87;font-weight:bold">for</span> using kind! 😊
</span></span></code></pre></div><p>Let&rsquo;s run the following command to check out our new cluster:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-sh" data-lang="sh"><span style="display:flex;"><span>$ kubectl cluster-info --context kind-dev4
</span></span><span style="display:flex;"><span>Kubernetes control plane is running at https://127.0.0.1:51851
</span></span><span style="display:flex;"><span>CoreDNS is running at https://127.0.0.1:51851/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>To further debug and diagnose cluster problems, use <span style="color:#4e9a06">&#39;kubectl cluster-info dump&#39;</span>.
</span></span><span style="display:flex;"><span>$ kubectl get node
</span></span><span style="display:flex;"><span>NAME STATUS ROLES AGE VERSION
</span></span><span style="display:flex;"><span>dev4-control-plane Ready control-plane,master 3m28s v1.22.0
</span></span><span style="display:flex;"><span>dev4-worker Ready &lt;none&gt; 2m54s v1.22.0
</span></span><span style="display:flex;"><span>dev4-worker2 Ready &lt;none&gt; 2m54s v1.22.0
</span></span><span style="display:flex;"><span>dev4-worker3 Ready &lt;none&gt; 2m54s v1.22.0
</span></span></code></pre></div><p>From the result above, we can see that this cluster has 1 control plane node and 3 worker nodes.</p>
<h3 id="3-control-plane-nodes-3-worker-nodes-highly-available-cluster">3 Control Plane Nodes, 3 Worker Nodes, Highly Available Cluster</h3>
<p><em>Note: &ldquo;Highly available&rdquo; here only means that we have three control plane nodes. It&rsquo;s not strictly &ldquo;highly available&rdquo; because apparently, the three control plane nodes are actually on the same host, so when the host is down, everything is gone.</em></p>
<p>Prepare the <code>ha-config.yaml</code> with the following content:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#204a87;font-weight:bold">kind</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">Cluster</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span><span style="color:#204a87;font-weight:bold">apiVersion</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">kind.x-k8s.io/v1alpha4</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span><span style="color:#204a87;font-weight:bold">nodes</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span>- <span style="color:#204a87;font-weight:bold">role</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">control-plane</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span>- <span style="color:#204a87;font-weight:bold">role</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">control-plane</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span>- <span style="color:#204a87;font-weight:bold">role</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">control-plane</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span>- <span style="color:#204a87;font-weight:bold">role</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">worker</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span>- <span style="color:#204a87;font-weight:bold">role</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">worker</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span>- <span style="color:#204a87;font-weight:bold">role</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">worker</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span></code></pre></div><p>Run:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-sh" data-lang="sh"><span style="display:flex;"><span>$ kind create cluster --config ha-config.yaml <span style="color:#4e9a06">\
</span></span></span><span style="display:flex;"><span><span style="color:#4e9a06"></span> --image<span style="color:#ce5c00;font-weight:bold">=</span>kindest/node:v1.22.0 --name<span style="color:#ce5c00;font-weight:bold">=</span>dev6
</span></span></code></pre></div><p>We can see familiar outputs, with the exception being &ldquo;Configuring the external load balancer” and “Joining more control-plane nodes&rdquo;:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-sh" data-lang="sh"><span style="display:flex;"><span>Creating cluster <span style="color:#4e9a06">&#34;dev6&#34;</span> ...
</span></span><span style="display:flex;"><span> ✓ Ensuring node image <span style="color:#ce5c00;font-weight:bold">(</span>kindest/node:v1.22.0<span style="color:#ce5c00;font-weight:bold">)</span> 🖼
</span></span><span style="display:flex;"><span> ✓ Preparing nodes 📦 📦 📦 📦 📦 📦
</span></span><span style="display:flex;"><span> ✓ Configuring the external load balancer ⚖️
</span></span><span style="display:flex;"><span> ✓ Writing configuration 📜
</span></span><span style="display:flex;"><span> ✓ Starting control-plane 🕹️
</span></span><span style="display:flex;"><span> ✓ Installing CNI 🔌
</span></span><span style="display:flex;"><span> ✓ Installing StorageClass 💾
</span></span><span style="display:flex;"><span> ✓ Joining more control-plane nodes 🎮
</span></span><span style="display:flex;"><span> ✓ Joining worker nodes 🚜
</span></span><span style="display:flex;"><span>Set kubectl context to <span style="color:#4e9a06">&#34;kind-dev6&#34;</span>
</span></span><span style="display:flex;"><span>You can now use your cluster with:
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>kubectl cluster-info --context kind-dev6
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>Have a nice day! 👋
</span></span></code></pre></div><p>Some fun facts:</p>
<ul>
<li>the number of boxes after &ldquo;Preparing nodes&rdquo; equals the number of nodes</li>
<li>the final greeting message is different: it was &ldquo;Thanks for using kind! 😊&rdquo; previously and now it&rsquo;s &ldquo;Have a nice day! 👋&rdquo;</li>
</ul>
<p>Heck, those <code>kind</code> developers sure went some extra miles to enhance the user experience!</p>
<p>Check the cluster:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-sh" data-lang="sh"><span style="display:flex;"><span>$ kubectl cluster-info --context kind-dev6
</span></span><span style="display:flex;"><span>Kubernetes control plane is running at https://127.0.0.1:52937
</span></span><span style="display:flex;"><span>CoreDNS is running at https://127.0.0.1:52937/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>To further debug and diagnose cluster problems, use <span style="color:#4e9a06">&#39;kubectl cluster-info dump&#39;</span>.
</span></span><span style="display:flex;"><span>$ kubectl get node
</span></span><span style="display:flex;"><span>NAME STATUS ROLES AGE VERSION
</span></span><span style="display:flex;"><span>dev6-control-plane Ready control-plane,master 8m19s v1.22.0
</span></span><span style="display:flex;"><span>dev6-control-plane2 Ready control-plane,master 7m46s v1.22.0
</span></span><span style="display:flex;"><span>dev6-control-plane3 Ready control-plane,master 7m20s v1.22.0
</span></span><span style="display:flex;"><span>dev6-worker Ready &lt;none&gt; 7m v1.22.0
</span></span><span style="display:flex;"><span>dev6-worker2 Ready &lt;none&gt; 7m v1.22.0
</span></span><span style="display:flex;"><span>dev6-worker3 Ready &lt;none&gt; 7m v1.22.0
</span></span></code></pre></div><p>We can see this cluster has 3 control plane nodes.</p>
<p>So far, we have mastered how to use <code>kind</code> to create a multi-node Kubernetes cluster locally.</p>
<hr>
<h2 id="advanced-kind-features">Advanced <code>kind</code> Features</h2>
<p>Now that we know how to create clusters using <code>kind</code>, let&rsquo;s have a look at some advanced operations which could help you better use the clusters.</p>
<h3 id="port-mapping">Port Mapping</h3>
<p>Imagine you are running an Nginx container listening on port 8080 in a <code>kind</code> cluster but you wish the outside world (outside of the cluster) to access the Nginx port. To achieve this, we can add the <code>extraPortMappings</code> configuration:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#204a87;font-weight:bold">kind</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">Cluster</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span><span style="color:#204a87;font-weight:bold">apiVersion</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">kind.x-k8s.io/v1alpha4</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span><span style="color:#204a87;font-weight:bold">nodes</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span>- <span style="color:#204a87;font-weight:bold">role</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">control-plane</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#204a87;font-weight:bold">extraPortMappings</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"> </span>- <span style="color:#204a87;font-weight:bold">containerPort</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#0000cf;font-weight:bold">8080</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#204a87;font-weight:bold">hostPort</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#0000cf;font-weight:bold">8080</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#204a87;font-weight:bold">listenAddress</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#4e9a06">&#34;0.0.0.0&#34;</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#204a87;font-weight:bold">protocol</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">tcp</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span></code></pre></div><p>In this way, the port on the pod 8080 is mapped to the port on the host.</p>
<h3 id="expose-kube-apiserver">Expose <code>kube-apiserver</code></h3>
<p>Sometimes we want to install Kubernetes with <code>kind</code> on one host, but access the cluster from another host. By default, the <code>kube-apiserver</code> installed by <code>kind</code> listens on 127.0.0.1 (with a random port.) To make the <code>kind</code> cluster accessible from another host, we need to make <code>kube-apiserver</code> listen on a network interface (for example, eth0.)</p>
<p>In the config file, we add <code>networking.apiServerAddress</code>. The IP is your local nic&rsquo;s IP:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#204a87;font-weight:bold">kind</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">Cluster</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span><span style="color:#204a87;font-weight:bold">apiVersion</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">kind.x-k8s.io/v1alpha4</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span><span style="color:#204a87;font-weight:bold">networking</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#204a87;font-weight:bold">apiServerAddress</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#4e9a06">&#34;192.168.39.1&#34;</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span></code></pre></div><h3 id="enable-feature-gates">Enable Feature Gates</h3>
<p>Feature gates are a set of <code>key=value</code> pairs that describe Kubernetes features that are only available in Alpha, Beta or GA stage.</p>
<p>If we want to try some of those features, we can enable Feature Gates. In the config file, use:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#204a87;font-weight:bold">kind</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">Cluster</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span><span style="color:#204a87;font-weight:bold">apiVersion</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">kind.x-k8s.io/v1alpha4</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span><span style="color:#204a87;font-weight:bold">featureGates</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#204a87;font-weight:bold">FeatureGateName</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#204a87;font-weight:bold">true</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span></code></pre></div><h3 id="importing-images">Importing Images</h3>
<p>Because the Kubernetes cluster is in fact running in a Docker container, by default, it can&rsquo;t access Docker images that are on the host. However, we can import those images from hosts to the Kubernetes cluster created by <code>kind</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-sh" data-lang="sh"><span style="display:flex;"><span><span style="color:#8f5902;font-style:italic"># import my-image:v1</span>
</span></span><span style="display:flex;"><span>kind load docker-image my-image:v1 --name dev
</span></span><span style="display:flex;"><span><span style="color:#8f5902;font-style:italic"># import my-image.tar</span>
</span></span><span style="display:flex;"><span>kind load image-archive my-image.tar --name dev
</span></span></code></pre></div><p>With this method, when we are building a new Docker image on the host which we want to run in a <code>kind</code> Kubernetes cluster, we can:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-sh" data-lang="sh"><span style="display:flex;"><span>docker build -t my-image:v1 ./my-image-dir
</span></span><span style="display:flex;"><span>kind load docker-image my-image:v1
</span></span><span style="display:flex;"><span>kubectl apply -f my-image.yaml
</span></span></code></pre></div><p>How to see the images that are available in the <code>kind</code> Kubernetes cluster? Easy:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-sh" data-lang="sh"><span style="display:flex;"><span>docker <span style="color:#204a87">exec</span> -it dev-control-plane crictl images
</span></span></code></pre></div><p><code>dev-control-plane</code> is the name of the <code>kind</code> cluster.</p>
<p>You can also use <code>crictl -h</code> to see all the supported commands. For example, we can delete an image by using <code>crictl rmi &lt;image_name&gt;</code>.</p>
<hr>
<h2 id="summary">Summary</h2>
<p>Have fun playing with Kubernetes locally!</p>
<p>If you like this article, please like, comment, and subscribe. See you in the next article!</p></description></item><item><title>Blog: v0.4.0 Release Note</title><link>https://www.devstream.io/blog/v0.4.0-release-note/</link><pubDate>Fri, 15 Apr 2022 00:00:00 +0000</pubDate><guid>https://www.devstream.io/blog/v0.4.0-release-note/</guid><description>
<h2 id="download-devstream-v040">Download DevStream v0.4.0</h2>
<p>Official Releases for Different Platforms:</p>
<ul>
<li><a href="https://devstream.gateway.scarf.sh/releases/v0.4.0/dtm-darwin-arm64">dtm-darwin-arm64</a></li>
<li><a href="https://devstream.gateway.scarf.sh/releases/v0.4.0/dtm-darwin-amd64">dtm-darwin-amd64</a></li>
<li><a href="https://devstream.gateway.scarf.sh/releases/v0.4.0/dtm-linux-amd64">dtm-linux-amd64</a></li>
</ul>
<h2 id="major-changes-since-last-release-v032">Major Changes since Last Release (v0.3.2)</h2>
<h3 id="plugin">Plugin</h3>
<ul>
<li>a new gitlabci-generic plugin</li>
<li>GitHub related plugins now support organization</li>
</ul>
<h3 id="core">Core</h3>
<ul>
<li><code>dtm list plugins</code> command, which shows all supported plugins, thanks to @summingyu</li>
<li><code>dtm show config</code> command, which shows the default configuration of a given plugin</li>
<li><code>dtm show status</code> command, which shows the status of a given plugin</li>
<li>outputs now support string interpolation</li>
<li>variables support in the config file</li>
<li><code>dtm</code> command autocomplete, thanks to @imxw</li>
</ul>
<h3 id="docs">Docs</h3>
<ul>
<li>a new doc on how to create a doc (for readthedocs.io)</li>
<li>installation doc for macOS, thanks to @algobot76</li>
<li>a new doc on configuration</li>
<li>a new doc on <code>dtm</code> command autocomplete</li>
</ul>
<h3 id="bug-fix">Bug Fix</h3>
<ul>
<li><code>dtm destroy</code> and <code>dtm delete --force</code> now honor the <code>dependsOn</code> order.</li>
</ul>
<h3 id="others">Others</h3>
<ul>
<li>CI optimization, thanks to @imxw</li>
</ul>
<h2 id="detailed-changes">Detailed Changes</h2>
<ul>
<li>docs: update stream to devstream by @daniel-hutao in <a href="https://github.com/devstream-io/devstream/pull/382">https://github.com/devstream-io/devstream/pull/382</a></li>
<li>fix: golang lint ci failed by @daniel-hutao in <a href="https://github.com/devstream-io/devstream/pull/386">https://github.com/devstream-io/devstream/pull/386</a></li>
<li>doc: installation on mac by @algobot76 in <a href="https://github.com/devstream-io/devstream/pull/385">https://github.com/devstream-io/devstream/pull/385</a></li>
<li>docs: change <code>demo</code> to <code>repo</code> by @imxw in <a href="https://github.com/devstream-io/devstream/pull/390">https://github.com/devstream-io/devstream/pull/390</a></li>
<li>feat: remove dtm md5 validation and uploading .md5 file by @lfbdev in <a href="https://github.com/devstream-io/devstream/pull/389">https://github.com/devstream-io/devstream/pull/389</a></li>
<li>feat(cmd): support list command by @summingyu in <a href="https://github.com/devstream-io/devstream/pull/391">https://github.com/devstream-io/devstream/pull/391</a></li>
<li>Feat: support &ldquo;default config print&rdquo; logic by @daniel-hutao in <a href="https://github.com/devstream-io/devstream/pull/392">https://github.com/devstream-io/devstream/pull/392</a></li>
<li>docs: add a doc of docs, and refactor existing docs by @IronCore864 in <a href="https://github.com/devstream-io/devstream/pull/394">https://github.com/devstream-io/devstream/pull/394</a></li>
<li>feat: new plugin - gitlabci-generic by @IronCore864 in <a href="https://github.com/devstream-io/devstream/pull/396">https://github.com/devstream-io/devstream/pull/396</a></li>
<li>ci: optimize ci pipeline by @imxw in <a href="https://github.com/devstream-io/devstream/pull/398">https://github.com/devstream-io/devstream/pull/398</a></li>
<li>docs: update roadmap accordig to recent changes by @IronCore864 in <a href="https://github.com/devstream-io/devstream/pull/397">https://github.com/devstream-io/devstream/pull/397</a></li>
<li>Feat: support default config print logic by @imxw in <a href="https://github.com/devstream-io/devstream/pull/399">https://github.com/devstream-io/devstream/pull/399</a></li>
<li>Feat: Support default config print logic by @summingyu by @summingyu in <a href="https://github.com/devstream-io/devstream/pull/400">https://github.com/devstream-io/devstream/pull/400</a></li>
<li>feat: outputs support interpolation; add more e2e tests by @IronCore864 in <a href="https://github.com/devstream-io/devstream/pull/403">https://github.com/devstream-io/devstream/pull/403</a></li>
<li>feat: <code>util/github</code> org supporting by @daniel-hutao in <a href="https://github.com/devstream-io/devstream/pull/405">https://github.com/devstream-io/devstream/pull/405</a></li>
<li>fix: blog URL updated by @daniel-hutao in <a href="https://github.com/devstream-io/devstream/pull/407">https://github.com/devstream-io/devstream/pull/407</a></li>
<li>docs(README): replace discord badge with slack by @basicthinker in <a href="https://github.com/devstream-io/devstream/pull/408">https://github.com/devstream-io/devstream/pull/408</a></li>
<li>Feat: decuple the dtm with personal account by @daniel-hutao in <a href="https://github.com/devstream-io/devstream/pull/409">https://github.com/devstream-io/devstream/pull/409</a></li>
<li>Fix destroy force delete order issue by @IronCore864 in <a href="https://github.com/devstream-io/devstream/pull/411">https://github.com/devstream-io/devstream/pull/411</a></li>
<li>feat: new command <code>dtm show status</code> implementation by @daniel-hutao in <a href="https://github.com/devstream-io/devstream/pull/410">https://github.com/devstream-io/devstream/pull/410</a></li>
<li>fix: make state.ToList stable by @daniel-hutao in <a href="https://github.com/devstream-io/devstream/pull/412">https://github.com/devstream-io/devstream/pull/412</a></li>
<li>ci: use the test account to run the e2e by @daniel-hutao in <a href="https://github.com/devstream-io/devstream/pull/414">https://github.com/devstream-io/devstream/pull/414</a></li>
<li>feat: config supports variables by @IronCore864 in <a href="https://github.com/devstream-io/devstream/pull/415">https://github.com/devstream-io/devstream/pull/415</a></li>
<li>docs: add <code>fig</code> doc by @imxw in <a href="https://github.com/devstream-io/devstream/pull/413">https://github.com/devstream-io/devstream/pull/413</a></li>
<li>chore: adding the missing config field for most plugins. by @daniel-hutao in <a href="https://github.com/devstream-io/devstream/pull/416">https://github.com/devstream-io/devstream/pull/416</a></li>
<li>docs: update roadmap with more plugins by @IronCore864 in <a href="https://github.com/devstream-io/devstream/pull/418">https://github.com/devstream-io/devstream/pull/418</a></li>
<li>docs/tests: add docs and e2e tests for variables by @IronCore864 in <a href="https://github.com/devstream-io/devstream/pull/419">https://github.com/devstream-io/devstream/pull/419</a></li>
<li>release: v0.4.0 by @daniel-hutao in <a href="https://github.com/devstream-io/devstream/pull/420">https://github.com/devstream-io/devstream/pull/420</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a href="https://github.com/devstream-io/devstream/compare/v0.3.2...v0.4.0">https://github.com/devstream-io/devstream/compare/v0.3.2...v0.4.0</a></p></description></item><item><title>Blog: 9 Extraordinary Terraform Best Practices That Will Change Your Infra World</title><link>https://www.devstream.io/blog/9-extraordinary-terraform-best-practices-that-will-change-your-infra-world/</link><pubDate>Tue, 12 Apr 2022 00:00:00 +0000</pubDate><guid>https://www.devstream.io/blog/9-extraordinary-terraform-best-practices-that-will-change-your-infra-world/</guid><description>
<img src="https://www.devstream.io/blog/9-extraordinary-terraform-best-practices-that-will-change-your-infra-world/featured-background_hue3380180632e3277a681d06ca14cda6b_666339_640x0_resize_q75_catmullrom.jpg" width="640" height="427"/>
<p><em>Note: this article is orignally published at <a href="https://blog.gitguardian.com/9-extraordinary-terraform-best-practices/">GitGuardian Blog</a>.</em></p>
<p>Before you quickly glance over the title and think &ldquo;Oh god, yet another tutorial on how to give proper names to variables, how to use modules, how to manage states; nothing I haven&rsquo;t already know&rdquo; then close the tab right away, let me assure you this: this article is none of those.</p>
<p>This &ldquo;best practices&rdquo; article aims to tell you something you haven&rsquo;t read a hundred times. This article won&rsquo;t give you the answer to everything because there isn&rsquo;t one right answer that fits all. This article aims to make you think about your unique situation and make the best decisions by you and for you.</p>
<p>Without further adieu, let&rsquo;s start with writing Terraform code:</p>
<div class="card rounded p-2 td-post-card mb-4 mt-4" style="max-width: 1210px">
<img class="card-img-top" src="https://www.devstream.io/blog/9-extraordinary-terraform-best-practices-that-will-change-your-infra-world/featured-background_hue3380180632e3277a681d06ca14cda6b_666339_1200x800_fill_q75_catmullrom_smart1.jpg" width="1200" height="800">
<div class="card-body px-0 pt-2 pb-0">
<p class="card-text">
Pidu District, Chengdu, China.
<small class="text-muted"><br/>Photo: Tiexin Guo / CC-BY-CA</small></p>
</div>
</div>
<hr>
<h2 id="1-clean-code">1 Clean Code</h2>
<p>No tool or programming language is perfect, and Terraform is no different. It has limitations.</p>
<p>For example, for old Terraform users from 0.12 or even earlier age, you might remember that before Terraform 0.13, you can&rsquo;t even use <code>for_each</code> for modules. In August 2020, with the release of Terraform 0.13, HashiCorp finally introduced the ability to loop over modules with a single module call.</p>
<p>Once you have accepted the quirks and features, you can utilize a bunch of best practices to organize your code and to use it better. Even though Terraform isn&rsquo;t strictly a programming language, similar rules of writing code apply to Terraform as well.</p>
<p>But before we talk about Terraform code, let&rsquo;s have a quick look at coding or programming in general.</p>
<h3 id="11-on-code">1.1 On &ldquo;Code&rdquo;</h3>
<p>I want to start this conversation by quoting Knuth:</p>
<blockquote>
<p>Programs are meant to be read by humans and only incidentally for computers to execute.</p>
</blockquote>
<p>The computer has no problem with ambiguous variable names, extended functions, or a single file of thousands of lines of code. It will still execute properly. All the methodologies and ideas like refactoring, clean code, naming conventions, modules, packages, code smell, etc., are invented so that we <em>humans</em> can read the code better, not computers can run it better.</p>
<h3 id="12-code-evolving-code">1.2 Code, Evolving Code</h3>
<p>Programs evolve. Code changes.</p>
<p>It&rsquo;s rare that you finished a piece of code and leave it there for the rest of your life. But that&rsquo;s not how projects work. If that was the case, we wouldn&rsquo;t be talking about Terraform best practices: you would only use it once anyway.</p>
<p>It&rsquo;s normal that when we are at work, we have projects all the time. The time when there isn&rsquo;t any project is scarce. Because business wants to improve, and the project is the way to move from the current state to the next desired state. Changing from one state to another is &ldquo;project&rdquo;. By Nature, &ldquo;project&rdquo; means change, and the code is also changing constantly.</p>
<h3 id="13-writing-clean-code">1.3 Writing Clean Code</h3>
<p>In order to make the change more manageable, we write clean code.</p>
<p>We limit the length of the line width because humans are not good at reading very wide lines of words; we carefully choose the names of the variables so that we immediately know what it means the next time we read it; we try to reduce the length of functions because shorter functions are not only easier to test but also easier to understand; we try to split a file with thousands of lines of code into smaller chunks.</p>
<p>Computers don&rsquo;t care about any of these, at all. Be it one large file or ten smaller chunks of files, it will run. Clean code makes it easier and faster to read, to understand, to build upon it.</p>
<p>You may already understand where I&rsquo;m going with this, and you are right: we want to do whatever it is so that our code is easier to read, manage, and change.</p>
<h3 id="14-write-clean-terraform-code">1.4 Write Clean Terraform Code</h3>
<p>I don&rsquo;t need to tell you how to name your variables, why it&rsquo;s a bad practice to name a server &ldquo;test-server&rdquo; while it&rsquo;s in the production environment, should you add a tag to a resource or not, etc.</p>
<p>Use your best judgment.</p>
<p>Do whatever it is to make the code manageable (or just Google and read any other available Terraform best practices out there.)</p>
<hr>
<h2 id="2-know-your-stuff">2 Know Your Stuff</h2>
<h3 id="21-play-with-it-first">2.1 Play with It First</h3>
<p>For people who are new to the cloud or new to a specific service in the cloud, I don&rsquo;t recommend using Terraform as the first attempt to create that resource. This is true no matter which IaC tool you are using, not specific to Terraform.</p>
<p>First, go to the console, try things manually, read the official documents and FAQs, figure out what parameters are mandatory and what are optional, and what are the possible values for each specific parameter. This would definitely help. Don&rsquo;t worry because some big guy in the DevOps world told you that &ldquo;the moment you click a button in the console, you create technical debt.&rdquo; Forget about it. Get comfortable with it first.</p>
<p>Once you grasp the keys of the resource you are going to create, you can automate it using Terraform.</p>
<p>For experienced Terraform users, see if this relates to you: AWS released a new service or a new resource. You haven&rsquo;t used it yet. You tell yourself: &ldquo;I&rsquo;m a veteran; only noob plays with console. I&rsquo;ll just get started with a perfect Terraform module right away. Why bother playing with it in the console first anyway?&rdquo; Then after hours of working and debugging, you find out that you are stuck only because of a small parameter or configuration of the resource which you didn&rsquo;t figure out clearly in the first place.</p>
<h3 id="22-know-your-infrastructure">2.2 Know Your Infrastructure</h3>
<p>You need to know what exactly is created and managed by your Terraform code.</p>
<p>This is especially important when using third-party modules because there are so many parameters and different use cases, it&rsquo;s hard to know exactly which scenario to pick, what resources will be created, and what values to set for those bunch of parameters.</p>
<p>Oftentimes, when I need to provision some resources in the cloud using Terraform, I find that I can do it quicker if I write the resources and modules myself (of course, I could also re-use modules I wrote before) than finding a third-party module off the internet because a lot of third-party modules are heavily future-proofed; they try to solve everybody&rsquo;s problem with the same module: they are doing it the &ldquo;monolithic&rdquo; way.</p>
<p>Here I don&rsquo;t mean to assign blame to anybody, but for example, if you try to search a module, like EKS, from Terraform registry, you will find out that it has a whopping 62 input parameters. If you want to create an EKS cluster in an existing VPC, using self-managed worker nodes, with certain launch templates, what parameters to set? Have fun figuring that out.</p>
<p>Sometimes, &ldquo;do not re-use the wheel&rdquo; is the better way to go: Terraform isn&rsquo;t easy to get started, but once you get fluent with it, it&rsquo;s relatively easy to use. Creating a resource or a module isn&rsquo;t rocket science. You can manage it within a reasonable period of time. Weigh the advantages and disadvantages of using third-party modules before you decide.</p>
<hr>
<h2 id="3-the-myth-of-future-proof">3 The Myth of Future-Proof</h2>
<p>When you only need one feature, implement one, don&rsquo;t even try to implement another feature you might use in the future. Premature future-proof or over-future-proof generates not-so-clean code. This is also true even if you are using another Infrastructure as Code tool other than Terraform.</p>
<p>Many &ldquo;best practices&rdquo; would tell you to never use a local backend, always use a remote backend, run your Terraform from within a CI tool, always use modules, etc.</p>
<p>I&rsquo;m telling you none of those.</p>
<p>Because <em>there is no &ldquo;one-size-fits-all&rdquo; answer; it all depends.</em></p>
<p>For a simple example, if I&rsquo;m working with a minimum viable product (MVP) or even a proof of concept (POC), why bother wasting time creating a remote state with state lock and executing the job from a CI running in K8s in the cloud and creating ridiculously small modules just to have modules because others told you it&rsquo;s &ldquo;best practice&rdquo; to do so?</p>
<p>You can&rsquo;t possibly know what you exactly are going to need in the future. Chances are, even if you did some future-proof work, created some perfect module, and set up perfect remote state management, in the future, when you really need to use it, you would probably refactor it anyway. It&rsquo;s not like creating a module is hard. It isn&rsquo;t; it&rsquo;s not rocket science.</p>
<p>When you try to future-proof your code, you write if-else. You write conditions and branches so that your code could work for more than one scenario. Refactor is all about reducing if-else and simplifying the code. Why introduce complexity when you don&rsquo;t really need it now?</p>
<p>But hey, don&rsquo;t get me wrong: I&rsquo;m not telling you to give up modules and remote states and some fancy features and what have you; the point is, creating a flexible base so that it could adapt to possible future changes, but don&rsquo;t waste too much time and energy future-proofing it.</p>
<hr>
<h2 id="4-do-one-thing-and-do-it-right">4 Do One Thing and Do It Right</h2>
<p>Just like the example given in the previous section, there are too many examples of Terraform modules and code that try to be the &ldquo;complete package&rdquo; by supporting every single possible scenario.</p>
<p>For experienced Terraform users, you might already be familiar with this: to make your module &ldquo;complete&rdquo; and useful in every scenario, you use complicated input variable structure, you create even more complicated local variables with short-hand conditions, you even need to use built-in functions to merge multiple variables as one so that if one variable is empty, you can still get the value from another variable and no exception would be thrown.</p>
<p>For starters, when you look at the code like this, it&rsquo;s not &ldquo;declarative&rdquo; anymore, because when you read something as complicated as that, you can&rsquo;t really know the description of the infrastructure that you are going to create with that code, you can&rsquo;t know what value is being set to this specific parameter.</p>
<p>Maybe writing a module for a specific scenario isn&rsquo;t that bad. When you have a slightly different use case, create another module. This might generate duplicated code, so you need to:</p>
<hr>
<h2 id="5-the-art-of-finding-the-balance-dry-vs-readability">5 The Art of Finding the Balance: DRY V.S. Readability</h2>
<p>DRY means Don&rsquo;t Repeat Yourself, and this principle is loved by many programmers.</p>
<p>Yet, you also have to find the right balance between &ldquo;duplicated code&rdquo; and &ldquo;readability.&rdquo; This is also true for any programming language because code is for humans to read.</p>
<p>When you want to achieve two things in one piece of code, you will need extra input parameters. You will need if-else. You will need to generate various outputs too. Adding too many features into one piece of code will invariably reduce the readability because brains are not so great with if-else and parameters.</p>
<p>On the other hand, you can choose to have two pieces of code for two slightly different features, with both having straight-forward logic flow and easy to read, but in this way, you probably will have some duplicated code. Using the right technique, for example, extracting the similar part out and creating a small module for it (if it will be commonly used) might be an answer.</p>
<p>Finding the right balance between duplicated code and readability is an art that requires experience to be made perfect, and only you can decide for yourself. &ldquo;You must have less than 10% duplicated code&rdquo; or &ldquo;reuse modules as much as possible&rdquo; are simply not pragmatic or helpful suggestions.</p>
<hr>
<h2 id="6-separate-infrastructure-with-configuration">6 Separate Infrastructure with Configuration</h2>
<h3 id="61-a-story">6.1 A Story</h3>
<p>Once, I was in a project where we use IaC to create Kubernetes clusters; then, some customization is done on top of it to install required components inside the cluster.</p>
<p>In that project, Terraform is used, then Terraform Kubernetes provider is also used to install things inside the cluster. So far, so good, because Terraform is idempotent (more on that later) by design.</p>
<p>The thing is, if a certain resource is already in the cluster, like a ConfigMap, the Kubernetes provider can&rsquo;t &ldquo;upsert&rdquo; it, and it would fail because it already existed. The provider breaks the idempotency.</p>
<p>This is an example of why you want to separate your IaC part from the configuration management part because not only does it make sense logically, but also it reduces the complexity.</p>
<p>In the example above, if we used Terraform only to create the cluster and nothing else, then use CI/CD tools to do <code>kubectl apply</code>, there would be no trouble at all.</p>
<h3 id="62-whats-infrastructure-as-code-iac">6.2 What&rsquo;s Infrastructure as Code (IaC)</h3>
<p>Infrastructure as Code (IaC) manages infrastructure in a descriptive model:</p>
<ul>
<li>It uses code files as the definition rather than interactive tools.</li>
<li>It tries to achieve 100% automation.</li>
<li>It doesn&rsquo;t matter if you run your own data center or you use the public cloud.</li>
</ul>
<p>You write code to manage your networks, servers (physical servers or virtual machines), connections, connection topology, load balancers, etc.</p>
<h3 id="63-whats-configuration-management-cm">6.3 What&rsquo;s Configuration Management (CM)</h3>
<p>Configuration Management (CM), on the other hand, maintains computer systems, software, dependencies, settings, etc., in a desired, consistent state.</p>
<p>Think physical data center as another example: purchasing servers, putting a newly purchased server onto a rack, installing servers, connecting networking cables to the switches so that it&rsquo;s connected to the existing networks (or think of launching a new virtual machine and assigning network interfaces to it) belongs to the definition of &ldquo;infrastructure.&rdquo; These are infrastructure parts, done by specific teams. In contrast, after the server is launched, configuring the servers to run specific software, for example, installing an HTTP server software and configuring it belongs to &ldquo;configuration management&rdquo;, and it probably can be done by another team which doesn&rsquo;t need to worry about the underlying infrastructure at all.</p>
<h3 id="64-the-iac-and-cm-separation">6.4 The IaC and CM Separation</h3>
<p>In the real world, things are not as simple as the &ldquo;black or white&rdquo; example above because we have many different tools and technologies allowing us to do Infrastructure as Code, or configuration management, or both at the same time.</p>
<p>For example, although Terraform is considered an IaC tool, it can do some configurations and installations on certain servers. And although Ansible is considered a configuration management tool, it can launch virtual machines.</p>
<p>Finding the right boundary for you, figuring out which part you would like Terraform to manage, and how Terraform interacts with your choice of CM tools is crucial, especially for large projects.</p>
<p>In an ever-changing world, the entropy in your system is only increasing. In the long run, you will benefit greatly from &ldquo;simplify&rdquo; and &ldquo;do one thing and do it right.&rdquo;</p>
<hr>
<h2 id="7-make-your-terraform-code-idempotent">7 Make Your Terraform Code Idempotent</h2>
<p>Idempotent means no matter how many times you run your IaC and, what your starting state is, you will end up with the same end state.</p>
<p>The same principle applies to configuration management too.</p>
<p>Why Do We Need Idempotency?</p>
<p>Idempotency is nice to have because infrastructure and configuration are not getting simpler as time goes on. Even if you just started fresh, you will handle complicated situations in no time. Idempotency simplifies the provisioning of infrastructure and the management of configurations, and it reduces the chances of inconsistent results.</p>
<p>For example, you need to set up A, then set up B after A is finished. If setting up A failed, you want to re-run your automation so that it can retry setting up B without trying to create A again (if it tries to create A again, it will fail because A already exists).</p>
<p>How to make it idempotent? Read on.</p>
<hr>
<h2 id="8-make-your-terraform-code-declarative">8 Make Your Terraform Code Declarative</h2>
<p>To achieve idempotency, a declarative style of code is preferred in most cases.</p>
<p>Declarative means defining the final state you want to have, rather than what command to execute in the code.</p>
<p>For example, you want to install an HTTP webserver. The task should be described as &ldquo;ensure an HTTP server is installed&rdquo; (i.e., if the HTTP server isn&rsquo;t installed, install it; if already installed, do nothing), instead of &ldquo;run this apt command to install the server.&rdquo;</p>
<p>When you look at your Infrastructure as Code, it should be like reading a document, a description of what you will have if you run this code, no matter how many times you run it.</p>
<p>When writing your infrastructure code or even creating a Terraform provider, you need to have &ldquo;side effects&rdquo; in mind. If this part runs a shell command or script, what happens if I run &ldquo;terraform apply&rdquo; again?</p>
<hr>
<h2 id="9-forget-about-cloud-agnostic--vendor-lock-in">9 Forget about Cloud Agnostic / Vendor Lock-in</h2>
<p>This might be controversial, but I&rsquo;d like to make it clear: Terraform isn&rsquo;t &ldquo;Cloud-Agnostic&rdquo;, and vendor lock-in doesn&rsquo;t matter (at least it not as much as you might think.)</p>
<p>In many cases, people are fighting hard to avoid vendor lock-in. Because we want to have a &ldquo;backup plan&rdquo; if things don&rsquo;t work out nicely with the current vendor. We want to have the option of moving to another vendor with as little trouble as possible.</p>
<p>It&rsquo;s not the case in the real world.</p>
<p>When you buy servers in bulk, you probably sign multi-year contracts with the vendor for a better price. When you are using the cloud, you rarely decide to move to another cloud. Having a multi-cloud setup, maybe yes, but migrating from one cloud to another isn&rsquo;t common, although situations like that do exist.</p>
<p>Even if you want to use Terraform to manage your AWS resource because you might want to move to GCP or Azure in the future, and you know Terraform works with GCP and Azure, in reality, you can&rsquo;t re-use your code. It goes without saying that if you want to switch from one cloud to another, you need to rewrite all your Terraform code: different cloud has different Terraform providers, and their resource name and parameters differ greatly.</p>
<p>Admit it or not, you are vendor locked-in, one way or another.</p>
<p>Once you are clear of this, it&rsquo;s, in fact, easier for you to choose the right tool for the job: because you are not afraid of vendor lock-in anymore, and you don&rsquo;t put it as the top priority one when making comparisons. Instead, you start to see the features, advantages, and disadvantages of each choice. For example, if you are already using Terraform with AWS, but for this specific piece of infrastructure, it might be even easier to use AWS CDK or some other tool (for example, <code>eksctl</code> to create a K8s cluster), why not?</p>
<h2 id="summary">Summary</h2>
<p>With a not-so-flat learning curve, Terraform can be intimidating, but once mastered, it provides you enough flexibility to manage your infrastructure. There are pitfalls and issues if you don&rsquo;t use it properly, but with some care and continuous refactoring, it can be manageable, and the code can be kept clean and easy to read. I hope these &ldquo;best practices&rdquo; help!</p></description></item><item><title>Blog: Dagger (the CI/CD Tool, not the Knife) In-Depth: Everything You Need to Know (as of Apr 2022)</title><link>https://www.devstream.io/blog/dagger-the-ci/cd-tool-not-the-knife-in-depth-everything-you-need-to-know-as-of-apr-2022/</link><pubDate>Tue, 12 Apr 2022 00:00:00 +0000</pubDate><guid>https://www.devstream.io/blog/dagger-the-ci/cd-tool-not-the-knife-in-depth-everything-you-need-to-know-as-of-apr-2022/</guid><description>
<img src="https://www.devstream.io/blog/dagger-the-ci/cd-tool-not-the-knife-in-depth-everything-you-need-to-know-as-of-apr-2022/featured-background_hu8915b1886940075ec02a53536d606ede_167855_640x0_resize_q75_catmullrom.jpg" width="640" height="355"/>
<div class="card rounded p-2 td-post-card mb-4 mt-4" style="max-width: 1210px">
<img class="card-img-top" src="https://www.devstream.io/blog/dagger-the-ci/cd-tool-not-the-knife-in-depth-everything-you-need-to-know-as-of-apr-2022/featured-background_hu8915b1886940075ec02a53536d606ede_167855_1200x800_fill_q75_catmullrom_smart1.jpg" width="1200" height="800">
<div class="card-body px-0 pt-2 pb-0">
<p class="card-text">
Chaotianmen, Chongqing, China
<small class="text-muted"><br/>Photo: Tiexin Guo / CC-BY-CA</small></p>
</div>
</div>
<h2 id="1-what-is-dagger">1 What is Dagger?</h2>
<p>TL;DR: Dagger runs your CI/CD pipelines locally in a Docker container, and can run the container in any CI environment (as long as that CI can run a container, of course.)</p>
<p>Do you want the long answer to this million-dollar question? It&rsquo;s hard to answer, honestly. News is calling it a &ldquo;DevOps platform&rdquo;; the VC that funded Dagger even called it a &ldquo;DevOps operating system.&rdquo;</p>
<p>But, in fact, Dagger is neither of those things.</p>
<p>Before we can answer what Dagger is, let&rsquo;s have a look at it in-depth:</p>
<h2 id="2-quirks-and-features">2 Quirks and Features</h2>
<h3 id="21-buildkit">2.1 BuildKit</h3>
<p>In Dagger, the configuration is executed in BuildKit, which is the execution engine at the heart of Docker.</p>
<p>BuildKit was developed as part of the Moby project, the latter of which is an open framework to assemble specialized container systems without reinventing the wheel by Docker. Basically, it&rsquo;s a toolkit for converting source code to build artifacts in an efficient, expressive, and repeatable manner. It was announced in 2017 and began shipping with Docker Engine in 2018’s version 18.09.</p>
<h3 id="22-cue">2.2 CUE</h3>
<p>Unlike most popular CI systems out there, you don&rsquo;t write YAML in Dagger; you write CUE.</p>
<p>I feel you because I didn&rsquo;t know what&rsquo;s CUE either. Turned out, CUE is an open-source data validation language and inference engine with its roots in logic programming.</p>
<p>It aims to simplify tasks involving defining and using data. It&rsquo;s actually a superset of JSON, so users familiar with JSON should feel comfortable with it already and can get started quickly. It also has got built-in auto-formatting (yay.)</p>
<p>Although the language is not a general-purpose programming language, it has many applications, such as data validation, data templating, configuration (that&rsquo;s probably why Dagger decided to use it in the first place), querying, code generation, and even scripting.</p>
<h3 id="23-wait-a-minute">2.3 Wait a Minute</h3>
<p>Since it&rsquo;s already reusing Docker&rsquo;s parts for configuration execution, why not reuse Docker&rsquo;s other part, Dockerfile, for configuration?</p>
<p>What&rsquo;s the purpose of using another language just for the configuration?</p>
<p>Solomon Hykes, the founder of Dagger, actually answered exact this question on their official Discord channel:</p>
<blockquote>
<p>We needed a modern declarative language with a type system, a package manager, native yaml and json interop, a formal spec, and a standalone community not locked to one tool.</p>
<p>Also Dockerfiles are specific to build, but Dagger is more general-purpose automation</p>
<p>There was no way at all Dockerfile could support our requirements (speaking as one of the original authors of the Dockerfile syntax)</p>
</blockquote>
<h2 id="3-enough-tech-spec-what-does-dagger-do">3 Enough Tech Spec. What Does Dagger Do?</h2>
<h3 id="31-what-dagger-isnt">3.1 What Dagger Isn&rsquo;t</h3>
<p>First, let me tell you what Dagger isn&rsquo;t by quoting the official documentation:</p>
<blockquote>
<p>Dagger does not replace your CI: it improves it by adding a portable development layer on top of it.</p>
</blockquote>
<p>OK, so it&rsquo;s not yet another CI (or CD, for that matter).</p>
<p>Dagger didn&rsquo;t even try to <em>replace</em> your existing CI, at all. But rather, it <em>improves</em> your CI, by adding a wrapper layer.</p>
<p>I know the term &ldquo;wrapper&rdquo; doesn&rsquo;t sound fancy, so let&rsquo;s call it by its official reference, and that is &ldquo;portable development layer&rdquo;.</p>
<h3 id="32-so-just-a-wrapper">3.2 So, Just a Wrapper?</h3>
<p>Disappointed? Don&rsquo;t conclude too quick; follow me. First, let&rsquo;s look at some other DevOps/cloud related examples:</p>
<ul>
<li>Think of Terraform. You&rsquo;ve got multiple environments to manage. Even with reuseable modules and roles, you still have duplicated code across envs. Then comes Terragrunt, which is a (thin) wrapper that provides extra tools for keeping your configurations simple without repeating yourself.</li>
<li>Think of AWS CDK. It actually is a wrapper layer on top of CloudFormation, which lets you use your familiar programming languages to define and provision AWS cloud infrastructure, so that you don&rsquo;t have to deal with CloudFormation&rsquo;s non-human-readable configurations. Of course, your code still converts to a format that CloudFormation understands, and your infrastructure is still managed by CloudFormation; AWS CDK doesn&rsquo;t really interact with AWS APIs directly. That&rsquo;s why it&rsquo;s only a wrapper layer on top of CloudFormation.</li>
<li>Think of CDKTF (CDK for Terraform); it&rsquo;s no different than AWS CDK, perhaps because CDKTF is inspired by AWS CDK and also uses AWS&rsquo;s <code>jsii</code> library to be polyglot. It&rsquo;s a wrapper layer on top of Terraform that translates your code into Terraform HCL so that you don&rsquo;t have to learn HCL. But in essence, your infrastructure is still managed by Terraform HCL, not your code directly. So, it&rsquo;s yet another wrapper.</li>
</ul>
<p>You must have already figured out where I am going with this, and you are right: yes, Dagger is no different. It is a wrapper.</p>
<p>But, of course, the wrapper has to do something to be useful, right? Then what exactly does Dagger do? What exactly does Dagger wrap? Good Questions.</p>
<h3 id="33-what-dagger-can-do">3.3 What Dagger Can Do</h3>
<p>In any CI system, you define some steps and actions in a certain format (YAML, most likely) and run it in your CI system. For example, in Jenkins, maybe you will write some groovy file. In GitHub Actions, you write some YAML with multiple steps.</p>
<p>Basically, Dagger runs those &ldquo;steps and actions&rdquo; in a Docker container. Then where do you run the Dagger docker container itself? Great question: you can either run it locally (because you can install Docker desktop, right?) or in your existing CI (since most CIs can run a docker container.)</p>
<p>If you think about it: Dagger doesn&rsquo;t wrap your CI pipelines or systems. It wraps those detailed steps and actions into a Docker container and still runs in your existing CI. It&rsquo;s like writing a big Dockerfile, and when you run the container, it does git clone, source code static scan, test, build, artifact upload, and what have you.</p>
<h2 id="4-what-dagger-really-is">4 What Dagger Really Is</h2>
<p>Yes, Dagger is a wrapper, that part is true.</p>
<p>But, it doesn&rsquo;t wrap CI systems; it wraps your pipeline steps and actions into a container (you have to rewrite those steps and actions in Dagger&rsquo;s syntax, though), and the wrapped result can run in another CI (as long as that CI can run a container.)</p>
<p>In this sense, Dagger <em>is</em> yet another CI, except that CI runs in a container and most CI systems happen to be able to run containers.</p>
<h2 id="5-benefits">5 Benefits</h2>
<p>I think there are 3 major advantages of using Dagger</p>
<h3 id="51-local-development">5.1 Local Development</h3>
<p>Firstly, there is no need to install <em>any</em> dependencies specific to this application, because Dagger manages all the intermediate steps and transient dependencies inside the Docker container.</p>
<p>This might not be an advantage if we are talking about CI, but it is an advantage if we are talking about local development.</p>
<p>Think of Go where you have to install modules or think of Nodejs where you might even need to switch Node versions then do an NPM install. Now you can do all of those inside a container and only get the final result to your local laptop.</p>
<h3 id="52-on-premise-ci">5.2 &ldquo;On-Premise&rdquo; CI</h3>
<p>You can run your pipeline locally now since you can easily have the Docker desktop up and running locally.</p>
<p>I&rsquo;m not sure if this is a solid requirement, though. Maybe it is? Because we all have powerful laptops now; why waste money on some CI systems when you can run them locally?</p>
<p>The idea of running it anywhere as long as Docker is available is intriguing, though. If you don&rsquo;t want to buy CI as a service, you can run Dagger in your own infrastructure.</p>
<h3 id="53-migrate-to-another-ci">5.3 Migrate to Another CI</h3>
<p>Since your &ldquo;steps and actions&rdquo; now are running in a container, you can run it elsewhere, in another CI system.</p>
<p>Should you need to migrate to another CI, you do not need to rewrite your CI steps anymore. For example, you don&rsquo;t want to use your company&rsquo;s old Jenkins instance anymore, but you are already using Dagger with Jenkins, and now you want to give GitHub Actions a try.</p>
<p>It&rsquo;s worth noting that in this scenario, there are still two things to do:</p>
<ul>
<li>If you are using Jenkins now, and want to migrate those Jenkins pipelines into Dagger, you need to do it manually. The cost is the same (if not more) as rewriting your whole pipeline in GitHub Actions&rsquo; syntax.</li>
<li>You will still need to learn about your new CI system: how a job is triggered, the syntax, etc.</li>
</ul>
<p>Here you can see Dagger does provide a solution to avoid CI lock-in (to some extent.) But it is not a game changing solution that could resolve your flexibility concern.</p>
<p>For a DevOps engineer, a DevOps toolchain that could accommodate different needs and priorities from different teams is more than appealing. That being said, each component is modular and pluggable, and you could free yourself from tedious work like launching, integrating, and managing all these pieces.</p>
<p>Ideally, you could define your desired DevOps tools in a single human-readable YAML config file, and with one single command, you have your whole DevOps toolchain and SDLC workflow set up or changed.</p>
<p>If you are intrigued by the simplicity of &ldquo;DevOps toolchain as code&rdquo;, don&rsquo;t hesitate to check out DevStream <a href="https://github.com/devstream-io/devstream">here</a>.</p>
<h2 id="6-should-i-use-dagger-now">6 Should I use Dagger Now?</h2>
<p>Is it promising? Maybe. Should I start using it now? My answer is No. Three reasons:</p>
<ul>
<li>The Dagger project itself still uses GitHub Actions. Why? Probably because it has limitations and can&rsquo;t do everything you can achieve with GitHub Actions.</li>
<li>You probably won&rsquo;t change your CI system every 6 months. If you only change it one time in 4 years, why bother adding that wrapper?</li>
<li>Dagger is only recently announced. It hasn&rsquo;t supported a whole lot of CI systems yet. Maybe the CI you want to switch to doesn&rsquo;t support it yet.</li>
</ul>
<p>Like, comment, subscribe. See you in the next article!</p></description></item><item><title>Blog: v0.3.1 Release Note</title><link>https://www.devstream.io/blog/v0.3.1-release-note/</link><pubDate>Thu, 31 Mar 2022 00:00:00 +0000</pubDate><guid>https://www.devstream.io/blog/v0.3.1-release-note/</guid><description>
<h2 id="download-devstream-v031">Download DevStream v0.3.1</h2>
<p>Official Releases for Different Platforms:</p>
<ul>
<li><a href="https://devstream.gateway.scarf.sh/releases/v0.3.1/dtm-darwin-arm64">dtm-darwin-arm64</a></li>
<li><a href="https://devstream.gateway.scarf.sh/releases/v0.3.1/dtm-darwin-amd64">dtm-darwin-amd64</a></li>
<li><a href="https://devstream.gateway.scarf.sh/releases/v0.3.1/dtm-linux-amd64">dtm-linux-amd64</a></li>
</ul>
<p>We also support installation by brew: <code>brew install devstream-io/devstream/dtm</code>. Example:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>$ brew install devstream-io/devstream/dtm
</span></span><span style="display:flex;"><span><span style="color:#ce5c00;font-weight:bold">==</span>&gt; Downloading https://github.com/devstream-io/homebrew-devstream/releases/download/dtm-0.3.1/dtm-0.3.1.arm64_monterey.bottle.tar.gz
</span></span><span style="display:flex;"><span><span style="color:#ce5c00;font-weight:bold">==</span>&gt; Downloading from https://objects.githubusercontent.com/github-production-release-asset-2e65be/474804179/268d59c6-9b12-419e-ac75-e77e87428d3b?X-Amz-Algorit
</span></span><span style="display:flex;"><span><span style="color:#8f5902;font-style:italic">######################################################################## 100.0%</span>
</span></span><span style="display:flex;"><span><span style="color:#ce5c00;font-weight:bold">==</span>&gt; Installing dtm from devstream-io/devstream
</span></span><span style="display:flex;"><span><span style="color:#ce5c00;font-weight:bold">==</span>&gt; Pouring dtm-0.3.1.arm64_monterey.bottle.tar.gz
</span></span><span style="display:flex;"><span>🍺 /opt/homebrew/Cellar/dtm/0.3.1: <span style="color:#0000cf;font-weight:bold">3</span> files, 13.5MB
</span></span><span style="display:flex;"><span><span style="color:#ce5c00;font-weight:bold">==</span>&gt; Running <span style="color:#4e9a06">`</span>brew cleanup dtm<span style="color:#4e9a06">`</span>...
</span></span><span style="display:flex;"><span>Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
</span></span><span style="display:flex;"><span>Hide these hints with HOMEBREW_NO_ENV_HINTS <span style="color:#ce5c00;font-weight:bold">(</span>see <span style="color:#4e9a06">`</span>man brew<span style="color:#4e9a06">`</span><span style="color:#ce5c00;font-weight:bold">)</span>.
</span></span><span style="display:flex;"><span>$ dtm version
</span></span><span style="display:flex;"><span>0.3.1
</span></span></code></pre></div><p><em>Developers&rsquo; note: technically, this version isn&rsquo;t backward compatible with v0.3.0; so it shouldn&rsquo;t be v0.3.1, but rather, v1.0.0. However, we are not ready for making it &ldquo;v1.0.0&rdquo; just yet, so please forgive us for this one-time exception of not following the SEMVER rules.</em></p>
<h2 id="major-changes-since-v030">Major Changes since v0.3.0</h2>
<p>First things first: we have a new logo now! Check out our <a href="https://github.com/devstream-io/devstream#readme=">README.md</a>.</p>
<p>Our website is live now, too. Visit us:</p>
<ul>
<li><a href="https://www.devstream.io">homepage</a></li>
<li><a href="https://docs.devstream.io/en/latest/">docs</a></li>
<li><a href="https://medium.com/devstream">Medium</a></li>
<li><a href="https://dev.to/devstream">dev.to</a></li>
</ul>
<h2 id="core">Core</h2>
<ul>
<li>Version improvement. This is a breaking change. Now, dtm will automatically download matching versions of plugins without having to specify the version of each plugin in the config.</li>
<li>Installation via brew is supported now: brew install devstream-io/devstream/dtm. Thanks to @algobot76.</li>
</ul>
<h2 id="develop">Develop</h2>
<ul>
<li>dtm develop now generates more scaffolding code for you so that you can easily create a new plugin. If you are interested, read <a href="https://www.devstream.io/blog/creating-a-devstream-dtm-plugin-for-anything/">this blog post</a>.</li>
<li>We support multi-threaded build now, thanks to @algobot76.</li>
<li>Makefile is greatly improved so that when you create a new plugin, you don&rsquo;t have to change the Makefile at all. Thanks to @summingyu.</li>
<li>We can automatically release a new version now.</li>
<li>A big refactor including directory name, document name, etc. Thanks to @imxw, @summingyu, etc.</li>
</ul>
<h2 id="docs">Docs</h2>
<ul>
<li>A new doc about the &ldquo;output&rdquo; feature is created.</li>
<li>A new doc about the dtm destroy command is created.</li>
<li>Our docs are now available on <a href="https://docs.devstream.io/en/latest/">readthedocs.io</a></li>
</ul>
<h2 id="detailed-changes">Detailed Changes</h2>
<ul>
<li>Refactor: unify the parameter type naming of various plugins to Options by @summingyu in <a href="https://github.com/devstream-io/devstream/pull/327">https://github.com/devstream-io/devstream/pull/327</a></li>
<li>Docs dtm destroy by @IronCore864 in <a href="https://github.com/devstream-io/devstream/pull/340">https://github.com/devstream-io/devstream/pull/340</a></li>
<li>docs: add output documentation by @IronCore864 in <a href="https://github.com/devstream-io/devstream/pull/341">https://github.com/devstream-io/devstream/pull/341</a></li>
<li>fix(develop template): fix update_go_nameTpl by @warren830 in <a href="https://github.com/devstream-io/devstream/pull/344">https://github.com/devstream-io/devstream/pull/344</a></li>
<li>fix(develop template): fix NAME_plugin_md_dirTpl by @warren830 in <a href="https://github.com/devstream-io/devstream/pull/343">https://github.com/devstream-io/devstream/pull/343</a></li>
<li>refactor(trello): consistent entry logic with other plugins by @daniel-hutao in <a href="https://github.com/devstream-io/devstream/pull/346">https://github.com/devstream-io/devstream/pull/346</a></li>
<li>chore: change params to options for main.go template by @imxw in <a href="https://github.com/devstream-io/devstream/pull/347">https://github.com/devstream-io/devstream/pull/347</a></li>
<li>refactor(argocdapp/devlake/trello/gitlabci/githubactions-nodejs): keep consistent with the <code>plugin template</code> by @warren830 in <a href="https://github.com/devstream-io/devstream/pull/345">https://github.com/devstream-io/devstream/pull/345</a></li>
<li>Feat: version improvement by @lfbdev in <a href="https://github.com/devstream-io/devstream/pull/337">https://github.com/devstream-io/devstream/pull/337</a></li>
<li>fix: main-builder.yaml -&gt; main-pushed.yml at README.md by @daniel-hutao in <a href="https://github.com/devstream-io/devstream/pull/349">https://github.com/devstream-io/devstream/pull/349</a></li>
<li>feat(develop/create-plugin): adding more common logic in template by @daniel-hutao in <a href="https://github.com/devstream-io/devstream/pull/350">https://github.com/devstream-io/devstream/pull/350</a></li>
<li>ci: e2e-test from <code>actions</code> to <code>eks</code> by @daniel-hutao in <a href="https://github.com/devstream-io/devstream/pull/322">https://github.com/devstream-io/devstream/pull/322</a></li>
<li>refactor(cmd): keep all plugins dirname-in-cmd == plugin-name by @daniel-hutao in <a href="https://github.com/devstream-io/devstream/pull/352">https://github.com/devstream-io/devstream/pull/352</a></li>
<li>fix: change main_go_dir to <code>cmd/plugin/</code> by @imxw in <a href="https://github.com/devstream-io/devstream/pull/354">https://github.com/devstream-io/devstream/pull/354</a></li>
<li>chore: update .gitignore to ignore .vscode directory by @algobot76 in <a href="https://github.com/devstream-io/devstream/pull/353">https://github.com/devstream-io/devstream/pull/353</a></li>
<li>Refactor some Plugins to Be Consistent with The Plugin Template by @summingyu in <a href="https://github.com/devstream-io/devstream/pull/355">https://github.com/devstream-io/devstream/pull/355</a></li>
<li>docs: update readme with new logo by @IronCore864 in <a href="https://github.com/devstream-io/devstream/pull/356">https://github.com/devstream-io/devstream/pull/356</a></li>
<li>chore: revert the workflow name with &lsquo;main builder&rsquo; by @daniel-hutao in <a href="https://github.com/devstream-io/devstream/pull/357">https://github.com/devstream-io/devstream/pull/357</a></li>
<li>Refactor: Automatically build the makefile from the cmd directory by @summingyu in <a href="https://github.com/devstream-io/devstream/pull/358">https://github.com/devstream-io/devstream/pull/358</a></li>
<li>feat: trigger GitHub actions after some specific comments by @daniel-hutao in <a href="https://github.com/devstream-io/devstream/pull/362">https://github.com/devstream-io/devstream/pull/362</a></li>
<li>refactor(plugin): adapt for the plugin generation tool by @imxw in <a href="https://github.com/devstream-io/devstream/pull/359">https://github.com/devstream-io/devstream/pull/359</a></li>
<li>Refactor: be consistent with the plugin template by @summingyu in <a href="https://github.com/devstream-io/devstream/pull/363">https://github.com/devstream-io/devstream/pull/363</a></li>
<li>ci: improve makefile for multi-threaded build and readme by @IronCore864 in <a href="https://github.com/devstream-io/devstream/pull/366">https://github.com/devstream-io/devstream/pull/366</a></li>
<li>Automated Release script and workflow by @lfbdev in <a href="https://github.com/devstream-io/devstream/pull/360">https://github.com/devstream-io/devstream/pull/360</a></li>
<li>fix: GitHub actions error - API rate limit exceeded by @daniel-hutao in <a href="https://github.com/devstream-io/devstream/pull/368">https://github.com/devstream-io/devstream/pull/368</a></li>
<li>feat: remove plugin version from config by @IronCore864 in <a href="https://github.com/devstream-io/devstream/pull/369">https://github.com/devstream-io/devstream/pull/369</a></li>
<li>fix: install goimports at <code>main builder</code> &amp; <code>automated-release</code> workflow by @daniel-hutao in <a href="https://github.com/devstream-io/devstream/pull/370">https://github.com/devstream-io/devstream/pull/370</a></li>
<li>chore: remove the suffix <code>_plugin</code> with all plugin docs under <code>docs/plugins</code> by @daniel-hutao in <a href="https://github.com/devstream-io/devstream/pull/371">https://github.com/devstream-io/devstream/pull/371</a></li>
<li>fix: install goimports at e2t-test workflow job by @daniel-hutao in <a href="https://github.com/devstream-io/devstream/pull/373">https://github.com/devstream-io/devstream/pull/373</a></li>
<li>feat: prepare for v0.3.1 release by @IronCore864 in <a href="https://github.com/devstream-io/devstream/pull/375">https://github.com/devstream-io/devstream/pull/375</a></li>
<li>fix: fix the return error by @lfbdev in <a href="https://github.com/devstream-io/devstream/pull/376">https://github.com/devstream-io/devstream/pull/376</a></li>
</ul>
<h2 id="new-contributors">New Contributors</h2>
<ul>
<li>@algobot76 made their first contribution in <a href="https://github.com/devstream-io/devstream/pull/353">https://github.com/devstream-io/devstream/pull/353</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a href="https://github.com/devstream-io/devstream/compare/v0.3.0...v0.3.1">https://github.com/devstream-io/devstream/compare/v0.3.0...v0.3.1</a></p></description></item><item><title>Blog: Creating a DevStream (dtm) Plugin for Anything</title><link>https://www.devstream.io/blog/creating-a-devstream-dtm-plugin-for-anything/</link><pubDate>Tue, 29 Mar 2022 00:00:00 +0000</pubDate><guid>https://www.devstream.io/blog/creating-a-devstream-dtm-plugin-for-anything/</guid><description>
<img src="https://www.devstream.io/blog/creating-a-devstream-dtm-plugin-for-anything/featured-background_hubced0a870f76eab482d622a0dd2f964e_408122_640x0_resize_q75_catmullrom.jpg" width="640" height="427"/>
<p>Yes, the title of this post isn&rsquo;t bluffing: you can actually create a plugin for just about anything that takes your fancy.</p>
<blockquote>
<p>In my previous article, I walked you guys through DevStream&rsquo;s codebase.</p>
<p>If you haven&rsquo;t read it yet, here&rsquo;s a quick link for you:</p>
<p><a href="../hello-world">Hello, world!</a>.</p>
</blockquote>
<p>In this blog, we will walk you through the steps of creating a DevStream plugin from scratch with an example.</p>
<hr>
<div class="card rounded p-2 td-post-card mb-4 mt-4" style="max-width: 1210px">
<img class="card-img-top" src="https://www.devstream.io/blog/creating-a-devstream-dtm-plugin-for-anything/featured-background_hubced0a870f76eab482d622a0dd2f964e_408122_1200x800_fill_q75_catmullrom_smart1.jpg" width="1200" height="800">
<div class="card-body px-0 pt-2 pb-0">
<p class="card-text">
A cat outside a lake side Starbucks, Chengdu, China.
<small class="text-muted"><br/>Photo: Tiexin Guo / CC-BY-CA</small></p>
</div>
</div>
<hr>
<blockquote>
<p>English version: <a href="../creating-a-devstream-dtm-plugin-for-anything/">《Creating a DevStream (dtm) Plugin for Anything》</a>Author: @Tiexin Guo(郭铁心)</p>
<p>中文版：<a href="../../zh/blog/creating-a-devstream-dtm-plugin-for-anything/">《给 DevStream (dtm) 开发一个插件，整合一切你想要的功能》</a>作者：@胡涛(Daniel Hu)</p>
</blockquote>
<hr>
<h2 id="what-is-devstream">What is DevStream</h2>
<p>DevStream is an amazing tool that lets you install, update, manage, and integrate your DevOps tools quickly and flexibly.</p>
<p>Not to brag, but with DevStream, you can have your own DevOps toolchain that is specifically tailored to your need up and running in 5 minutes.</p>
<p>Don&rsquo;t believe it? Check out our <a href="https://docs.devstream.io/en/latest/">docs</a> and follow the quickstart guide!</p>
<h2 id="existing-plugins">Existing Plugins</h2>
<p>At the moment of publishing this article, we already support the following tools:</p>
<ul>
<li>Trello (including integration with GitHub)</li>
<li>Jira (integration with GitHub)</li>
<li>GitHub Repository bootstrapping (for Go)</li>
<li>GitHub Actions (for Go, Python, and Nodejs)</li>
<li>GitLab CI (for Go)</li>
<li>Jenkins (installation)</li>
<li>ArgoCD</li>
<li>ArgoCD Application (the deployment of your apps)</li>
<li>Prometheus + Grafana</li>
<li>DevLake</li>
<li>OpenLDAP</li>
</ul>
<p>We aim to have 50 plugins at the end of 2022!</p>
<p>Check out our <a href="https://github.com/devstream-io/devstream">README</a> for the latest status.</p>
<h2 id="why-would-i-want-to-create-a-devstream-plugin">Why Would I Want to Create a DevStream Plugin</h2>
<p>Wait. YOU ALREADY HAVE TONS OF PLUGINS! Why on earth would I want to create yet another one?</p>
<p>I agree with you.</p>
<p>Can I get a show of hands, who here has made a DevStream plugin before?</p>
<p>Very few, if any, I guess.</p>
<p>However (I know it&rsquo;s just a fancy &ldquo;but&rdquo;), there are, in fact, things that you want to build a plugin for:</p>
<ul>
<li>Maybe you are building a nice and shiny DevOps tool, and you want to be able to set it up quickly without any hassle. You would have to write some automation scripts for it anyway, right? DevStream got you covered.</li>
<li>Maybe you even need to integrate your tool with other tools to get the most out of it and you really don&rsquo;t want to reinvent a lot of wheels just to manage those boring integrations. Again, DevStream got you covered.</li>
<li>Maybe you have both internal tools and open-source tools whose installation and integration need to be automated. The open-source tools are fine, but how to manage those internal ones and integrate them? DevStream got you covered.</li>
</ul>
<p>Or, maybe you just want to learn Go&rsquo;s plugin, become a contributor, join our community and earn your certification (maybe a small present, too, who knows.) No problem.</p>
<p>No matter what your intention is and what thing you want to achieve, DevStream got you covered.</p>
<p>So hang tight, let&rsquo;s get started.</p>
<h2 id="design-a-local-file-plugin">Design: A &ldquo;Local File&rdquo; Plugin</h2>
<p>In this example, let&rsquo;s build something dum but simple, just to show you the process of creating a plugin.</p>
<p>We are creating a &ldquo;local file&rdquo; plugin. You specify the name and the content, and the plugin will create a local file for you. Let&rsquo;s decide how we are going to use this plugin:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#204a87;font-weight:bold">tools</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span>- <span style="color:#204a87;font-weight:bold">name</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">my-file</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#204a87;font-weight:bold">plugin</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">localfile</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#204a87;font-weight:bold">options</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#204a87;font-weight:bold">filename</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">foo.txt</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#204a87;font-weight:bold">content</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#4e9a06">&#34;hello, world&#34;</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span></code></pre></div><p>Basically:</p>
<ul>
<li>the name of the plugin: <code>localfile</code></li>
<li>options of the plugin:
<ul>
<li>filename</li>
<li>content</li>
</ul>
</li>
</ul>
<p>The plugin will create a file with desired content according to this piece of config.</p>
<h2 id="scaffolding-automagically">Scaffolding Automagically</h2>
<p>First, let&rsquo;s clone the DevStream repo and generate some scaffolding code:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>git clone [email protected]:devstream-io/devstream.git
</span></span><span style="display:flex;"><span><span style="color:#204a87">cd</span> stream
</span></span><span style="display:flex;"><span><span style="color:#8f5902;font-style:italic"># builds dtm locally to make sure it&#39;s using the same dependencies as your new plugin</span>
</span></span><span style="display:flex;"><span>make build-core
</span></span><span style="display:flex;"><span>./dtm develop create-plugin --name<span style="color:#ce5c00;font-weight:bold">=</span>localfile
</span></span></code></pre></div><p>There will be some useful output to guide you through the whole process.</p>
<p>Now, if we do a <code>git status</code>, we can see some new stuff are already created automagically:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>$ git status
</span></span><span style="display:flex;"><span>On branch main
</span></span><span style="display:flex;"><span>Your branch is up to date with <span style="color:#4e9a06">&#39;origin/main&#39;</span>.
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>Untracked files:
</span></span><span style="display:flex;"><span> <span style="color:#ce5c00;font-weight:bold">(</span>use <span style="color:#4e9a06">&#34;git add &lt;file&gt;...&#34;</span> to include in what will be committed<span style="color:#ce5c00;font-weight:bold">)</span>
</span></span><span style="display:flex;"><span> cmd/plugin/localfile/
</span></span><span style="display:flex;"><span> docs/plugins/localfile.md
</span></span><span style="display:flex;"><span> internal/pkg/plugin/localfile/
</span></span></code></pre></div><p>Let&rsquo;s have a quick recap of the directory structure:</p>
<h3 id="cmd">cmd/</h3>
<p><code>cmd/plugin/localfile/main.go</code> is the main entrance of your plugin. But here you don&rsquo;t need to do anything; nothing.</p>
<p><code>dtm</code> has already generated the code for you, including the interfaces that you must implement.</p>
<h3 id="docs">docs/</h3>
<p><code>docs/plugins/localfile.md</code> is the automatically generated documentation.</p>
<p>Yep, I know <code>dtm</code> is automagic, but it can&rsquo;t read your mind. I&rsquo;m afraid that you will have to write your own doc.</p>
<p>But hey, at least here you get a reminder that you need to create a doc :)</p>
<h3 id="internalpkg">internal/pkg/</h3>
<p><code>internal/pkg/plugin/localfile</code> has your plugin&rsquo;s main logic.</p>
<p>Here, we need to:</p>
<ul>
<li>define your input parameters (options);</li>
<li>implement the validation of the input parameters;</li>
<li>implement four mandatory interfaces.</li>
</ul>
<h2 id="core-concepts">Core Concepts</h2>
<h3 id="configstateresource">Config/State/Resource</h3>
<p>Before explaining interfaces and implementing them, let&rsquo;s have a look at how DevStream actually works:</p>
<ul>
<li><em>Config</em> is a list of tools, each of which has a name, plugin, options, etc.</li>
<li><em>State</em> is a map containing each tool&rsquo;s name, plugin, options, etc. It&rsquo;s used to store the result of <code>dtm</code>&rsquo;s last action.</li>
<li><em>Resource</em> is the tool that the plugin created. The <code>Read</code> interface returns a description of that resource, which should be the same as the <em>State</em> if nothing has been changed after <code>dtm</code>&rsquo;s last action.</li>
</ul>
<p>DevStream decides what to do based on your <em>Config</em>, the <em>State</em>, and the <em>Resource</em>&rsquo;s status. See the flowchart below:</p>
<p><img src="https://www.devstream.io/img/config_state_resource.png" alt=""></p>
<h3 id="interfaces">Interfaces</h3>
<p>A quick recap: each DevStream plugin must satisfy the following four interfaces:</p>
<ul>
<li><code>Create</code></li>
<li><code>Read</code></li>
<li><code>Update</code></li>
<li><code>Delete</code></li>
</ul>
<p>Return values:</p>
<ul>
<li><code>Create</code> and <code>Update</code> return two values <code>(map[string]interface{}, error)</code>. The first return value is considered as the <em>State</em>, which will be stored in DevStream&rsquo;s state file.</li>
<li><code>Read</code>&rsquo;s first return value is a description of the <em>Resource</em>, which should be the same as the <em>State</em> if nothing has changed.</li>
<li><code>Delete</code> returns <code>(true, nil)</code> if there is no error; otherwise it returns <code>(false, error)</code>.</li>
</ul>
<h2 id="coding">Coding</h2>
<h3 id="input-optionsvalidation">Input Options/Validation</h3>
<p>Now let&rsquo;s open <code>internal/pkg/plugin/localfile/options.go</code> and add options according to our design in the previous section.</p>
<p>The <code>internal/pkg/plugin/localfile/options.go</code> should look like:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#204a87;font-weight:bold">package</span> <span style="color:#000">localfile</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#8f5902;font-style:italic">// Options is the struct for configurations of the localfile plugin.
</span></span></span><span style="display:flex;"><span><span style="color:#8f5902;font-style:italic"></span><span style="color:#204a87;font-weight:bold">type</span> <span style="color:#000">Options</span> <span style="color:#204a87;font-weight:bold">struct</span> <span style="color:#000;font-weight:bold">{</span>
</span></span><span style="display:flex;"><span> <span style="color:#000">Filename</span> <span style="color:#204a87;font-weight:bold">string</span>
</span></span><span style="display:flex;"><span> <span style="color:#000">Content</span> <span style="color:#204a87;font-weight:bold">string</span>
</span></span><span style="display:flex;"><span><span style="color:#000;font-weight:bold">}</span>
</span></span></code></pre></div><p>Let&rsquo;s also implement the validation function of the input options.</p>
<p>Open <code>internal/pkg/plugin/localfile/validate.go</code> and change the logic to verify the options. It should look like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#204a87;font-weight:bold">package</span> <span style="color:#000">localfile</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#204a87;font-weight:bold">import</span> <span style="color:#4e9a06">&#34;fmt&#34;</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#8f5902;font-style:italic">// validate validates the options provided by the core.
</span></span></span><span style="display:flex;"><span><span style="color:#8f5902;font-style:italic"></span><span style="color:#204a87;font-weight:bold">func</span> <span style="color:#000">validate</span><span style="color:#000;font-weight:bold">(</span><span style="color:#000">options</span> <span style="color:#ce5c00;font-weight:bold">*</span><span style="color:#000">Options</span><span style="color:#000;font-weight:bold">)</span> <span style="color:#000;font-weight:bold">[]</span><span style="color:#204a87;font-weight:bold">error</span> <span style="color:#000;font-weight:bold">{</span>
</span></span><span style="display:flex;"><span> <span style="color:#000">res</span> <span style="color:#ce5c00;font-weight:bold">:=</span> <span style="color:#204a87">make</span><span style="color:#000;font-weight:bold">([]</span><span style="color:#204a87;font-weight:bold">error</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#0000cf;font-weight:bold">0</span><span style="color:#000;font-weight:bold">)</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">if</span> <span style="color:#000">options</span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">Filename</span> <span style="color:#ce5c00;font-weight:bold">==</span> <span style="color:#4e9a06">&#34;&#34;</span> <span style="color:#000;font-weight:bold">{</span>
</span></span><span style="display:flex;"><span> <span style="color:#000">res</span> <span style="color:#000;font-weight:bold">=</span> <span style="color:#204a87">append</span><span style="color:#000;font-weight:bold">(</span><span style="color:#000">res</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#000">fmt</span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">Errorf</span><span style="color:#000;font-weight:bold">(</span><span style="color:#4e9a06">&#34;filename does not exist&#34;</span><span style="color:#000;font-weight:bold">))</span>
</span></span><span style="display:flex;"><span> <span style="color:#000;font-weight:bold">}</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">return</span> <span style="color:#000">res</span>
</span></span><span style="display:flex;"><span><span style="color:#000;font-weight:bold">}</span>
</span></span></code></pre></div><h3 id="create">Create()</h3>
<p>We want to create the file based on the input options. So, let&rsquo;s do just that in the file <code>internal/pkg/plugin/localfile/create.go</code>.</p>
<p>OK, talk is cheap, show me the code:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#204a87;font-weight:bold">package</span> <span style="color:#000">localfile</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#204a87;font-weight:bold">import</span> <span style="color:#000;font-weight:bold">(</span>
</span></span><span style="display:flex;"><span> <span style="color:#4e9a06">&#34;fmt&#34;</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#4e9a06">&#34;github.com/mitchellh/mapstructure&#34;</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#4e9a06">&#34;github.com/devstream-io/devstream/pkg/util/log&#34;</span>
</span></span><span style="display:flex;"><span><span style="color:#000;font-weight:bold">)</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#204a87;font-weight:bold">func</span> <span style="color:#000">Create</span><span style="color:#000;font-weight:bold">(</span><span style="color:#000">options</span> <span style="color:#204a87;font-weight:bold">map</span><span style="color:#000;font-weight:bold">[</span><span style="color:#204a87;font-weight:bold">string</span><span style="color:#000;font-weight:bold">]</span><span style="color:#204a87;font-weight:bold">interface</span><span style="color:#000;font-weight:bold">{})</span> <span style="color:#000;font-weight:bold">(</span><span style="color:#204a87;font-weight:bold">map</span><span style="color:#000;font-weight:bold">[</span><span style="color:#204a87;font-weight:bold">string</span><span style="color:#000;font-weight:bold">]</span><span style="color:#204a87;font-weight:bold">interface</span><span style="color:#000;font-weight:bold">{},</span> <span style="color:#204a87;font-weight:bold">error</span><span style="color:#000;font-weight:bold">)</span> <span style="color:#000;font-weight:bold">{</span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">var</span> <span style="color:#000">opts</span> <span style="color:#000">Options</span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">if</span> <span style="color:#000">err</span> <span style="color:#ce5c00;font-weight:bold">:=</span> <span style="color:#000">mapstructure</span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">Decode</span><span style="color:#000;font-weight:bold">(</span><span style="color:#000">options</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#ce5c00;font-weight:bold">&amp;</span><span style="color:#000">opts</span><span style="color:#000;font-weight:bold">);</span> <span style="color:#000">err</span> <span style="color:#ce5c00;font-weight:bold">!=</span> <span style="color:#204a87;font-weight:bold">nil</span> <span style="color:#000;font-weight:bold">{</span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">return</span> <span style="color:#204a87;font-weight:bold">nil</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#000">err</span>
</span></span><span style="display:flex;"><span> <span style="color:#000;font-weight:bold">}</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">if</span> <span style="color:#000">errs</span> <span style="color:#ce5c00;font-weight:bold">:=</span> <span style="color:#000">validate</span><span style="color:#000;font-weight:bold">(</span><span style="color:#ce5c00;font-weight:bold">&amp;</span><span style="color:#000">opts</span><span style="color:#000;font-weight:bold">);</span> <span style="color:#204a87">len</span><span style="color:#000;font-weight:bold">(</span><span style="color:#000">errs</span><span style="color:#000;font-weight:bold">)</span> <span style="color:#ce5c00;font-weight:bold">!=</span> <span style="color:#0000cf;font-weight:bold">0</span> <span style="color:#000;font-weight:bold">{</span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">for</span> <span style="color:#000">_</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#000">e</span> <span style="color:#ce5c00;font-weight:bold">:=</span> <span style="color:#204a87;font-weight:bold">range</span> <span style="color:#000">errs</span> <span style="color:#000;font-weight:bold">{</span>
</span></span><span style="display:flex;"><span> <span style="color:#000">log</span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">Errorf</span><span style="color:#000;font-weight:bold">(</span><span style="color:#4e9a06">&#34;Options error: %s.&#34;</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#000">e</span><span style="color:#000;font-weight:bold">)</span>
</span></span><span style="display:flex;"><span> <span style="color:#000;font-weight:bold">}</span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">return</span> <span style="color:#204a87;font-weight:bold">nil</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#000">fmt</span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">Errorf</span><span style="color:#000;font-weight:bold">(</span><span style="color:#4e9a06">&#34;opts are illegal&#34;</span><span style="color:#000;font-weight:bold">)</span>
</span></span><span style="display:flex;"><span> <span style="color:#000;font-weight:bold">}</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#000">err</span> <span style="color:#ce5c00;font-weight:bold">:=</span> <span style="color:#000">writefile</span><span style="color:#000;font-weight:bold">(</span><span style="color:#000">opts</span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">Filename</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#000">opts</span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">Content</span><span style="color:#000;font-weight:bold">)</span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">if</span> <span style="color:#000">err</span> <span style="color:#ce5c00;font-weight:bold">!=</span> <span style="color:#204a87;font-weight:bold">nil</span> <span style="color:#000;font-weight:bold">{</span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">return</span> <span style="color:#204a87;font-weight:bold">nil</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#000">err</span>
</span></span><span style="display:flex;"><span> <span style="color:#000;font-weight:bold">}</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#000">status</span> <span style="color:#ce5c00;font-weight:bold">:=</span> <span style="color:#204a87;font-weight:bold">map</span><span style="color:#000;font-weight:bold">[</span><span style="color:#204a87;font-weight:bold">string</span><span style="color:#000;font-weight:bold">]</span><span style="color:#204a87;font-weight:bold">interface</span><span style="color:#000;font-weight:bold">{}{</span>
</span></span><span style="display:flex;"><span> <span style="color:#4e9a06">&#34;filename&#34;</span><span style="color:#000;font-weight:bold">:</span> <span style="color:#000">opts</span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">Filename</span><span style="color:#000;font-weight:bold">,</span>
</span></span><span style="display:flex;"><span> <span style="color:#4e9a06">&#34;content&#34;</span><span style="color:#000;font-weight:bold">:</span> <span style="color:#000">opts</span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">Content</span><span style="color:#000;font-weight:bold">,</span>
</span></span><span style="display:flex;"><span> <span style="color:#000;font-weight:bold">}</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">return</span> <span style="color:#000">status</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#204a87;font-weight:bold">nil</span>
</span></span><span style="display:flex;"><span><span style="color:#000;font-weight:bold">}</span>
</span></span></code></pre></div><p>If you compare the code above to the code generated by <code>dtm develop</code>, you will see that we basically did nothing here. We only filled some blanks which are marked by <code>dtm</code>&rsquo;s comment. The same is true for all the other interfaces.</p>
<p><em>A small tip: here, we can put the function <code>writefile</code> in <code>internal/pkg/plugin/localfile/localfile.go</code>, so that the code is better organized.</em></p>
<p>The <code>localfile.go</code> will look like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#204a87;font-weight:bold">package</span> <span style="color:#000">localfile</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#204a87;font-weight:bold">import</span> <span style="color:#4e9a06">&#34;os&#34;</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#204a87;font-weight:bold">func</span> <span style="color:#000">writefile</span><span style="color:#000;font-weight:bold">(</span><span style="color:#000">filename</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#000">content</span> <span style="color:#204a87;font-weight:bold">string</span><span style="color:#000;font-weight:bold">)</span> <span style="color:#204a87;font-weight:bold">error</span> <span style="color:#000;font-weight:bold">{</span>
</span></span><span style="display:flex;"><span> <span style="color:#000">f</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#000">err</span> <span style="color:#ce5c00;font-weight:bold">:=</span> <span style="color:#000">os</span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">Create</span><span style="color:#000;font-weight:bold">(</span><span style="color:#000">filename</span><span style="color:#000;font-weight:bold">)</span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">if</span> <span style="color:#000">err</span> <span style="color:#ce5c00;font-weight:bold">!=</span> <span style="color:#204a87;font-weight:bold">nil</span> <span style="color:#000;font-weight:bold">{</span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">return</span> <span style="color:#000">err</span>
</span></span><span style="display:flex;"><span> <span style="color:#000;font-weight:bold">}</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">defer</span> <span style="color:#000">f</span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">Close</span><span style="color:#000;font-weight:bold">()</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#000">_</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#000">err</span> <span style="color:#000;font-weight:bold">=</span> <span style="color:#000">f</span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">WriteString</span><span style="color:#000;font-weight:bold">(</span><span style="color:#000">content</span><span style="color:#000;font-weight:bold">)</span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">if</span> <span style="color:#000">err</span> <span style="color:#ce5c00;font-weight:bold">!=</span> <span style="color:#204a87;font-weight:bold">nil</span> <span style="color:#000;font-weight:bold">{</span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">return</span> <span style="color:#000">err</span>
</span></span><span style="display:flex;"><span> <span style="color:#000;font-weight:bold">}</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">return</span> <span style="color:#204a87;font-weight:bold">nil</span>
</span></span><span style="display:flex;"><span><span style="color:#000;font-weight:bold">}</span>
</span></span></code></pre></div><h3 id="read">Read()</h3>
<p>Let&rsquo;s also implement the <code>Read</code> interface.</p>
<p>The logic is simple: we want to try to see if the file exists or not, and if yes, what&rsquo;s the filename and content.</p>
<p><code>internal/pkg/plugin/localfile/read.go</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#204a87;font-weight:bold">package</span> <span style="color:#000">localfile</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#204a87;font-weight:bold">import</span> <span style="color:#000;font-weight:bold">(</span>
</span></span><span style="display:flex;"><span> <span style="color:#4e9a06">&#34;fmt&#34;</span>
</span></span><span style="display:flex;"><span> <span style="color:#4e9a06">&#34;os&#34;</span>
</span></span><span style="display:flex;"><span> <span style="color:#4e9a06">&#34;strings&#34;</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#4e9a06">&#34;github.com/mitchellh/mapstructure&#34;</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#4e9a06">&#34;github.com/devstream-io/devstream/pkg/util/log&#34;</span>
</span></span><span style="display:flex;"><span><span style="color:#000;font-weight:bold">)</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#204a87;font-weight:bold">func</span> <span style="color:#000">Read</span><span style="color:#000;font-weight:bold">(</span><span style="color:#000">options</span> <span style="color:#204a87;font-weight:bold">map</span><span style="color:#000;font-weight:bold">[</span><span style="color:#204a87;font-weight:bold">string</span><span style="color:#000;font-weight:bold">]</span><span style="color:#204a87;font-weight:bold">interface</span><span style="color:#000;font-weight:bold">{})</span> <span style="color:#000;font-weight:bold">(</span><span style="color:#204a87;font-weight:bold">map</span><span style="color:#000;font-weight:bold">[</span><span style="color:#204a87;font-weight:bold">string</span><span style="color:#000;font-weight:bold">]</span><span style="color:#204a87;font-weight:bold">interface</span><span style="color:#000;font-weight:bold">{},</span> <span style="color:#204a87;font-weight:bold">error</span><span style="color:#000;font-weight:bold">)</span> <span style="color:#000;font-weight:bold">{</span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">var</span> <span style="color:#000">opts</span> <span style="color:#000">Options</span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">if</span> <span style="color:#000">err</span> <span style="color:#ce5c00;font-weight:bold">:=</span> <span style="color:#000">mapstructure</span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">Decode</span><span style="color:#000;font-weight:bold">(</span><span style="color:#000">options</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#ce5c00;font-weight:bold">&amp;</span><span style="color:#000">opts</span><span style="color:#000;font-weight:bold">);</span> <span style="color:#000">err</span> <span style="color:#ce5c00;font-weight:bold">!=</span> <span style="color:#204a87;font-weight:bold">nil</span> <span style="color:#000;font-weight:bold">{</span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">return</span> <span style="color:#204a87;font-weight:bold">nil</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#000">err</span>
</span></span><span style="display:flex;"><span> <span style="color:#000;font-weight:bold">}</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">if</span> <span style="color:#000">errs</span> <span style="color:#ce5c00;font-weight:bold">:=</span> <span style="color:#000">validate</span><span style="color:#000;font-weight:bold">(</span><span style="color:#ce5c00;font-weight:bold">&amp;</span><span style="color:#000">opts</span><span style="color:#000;font-weight:bold">);</span> <span style="color:#204a87">len</span><span style="color:#000;font-weight:bold">(</span><span style="color:#000">errs</span><span style="color:#000;font-weight:bold">)</span> <span style="color:#ce5c00;font-weight:bold">!=</span> <span style="color:#0000cf;font-weight:bold">0</span> <span style="color:#000;font-weight:bold">{</span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">for</span> <span style="color:#000">_</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#000">e</span> <span style="color:#ce5c00;font-weight:bold">:=</span> <span style="color:#204a87;font-weight:bold">range</span> <span style="color:#000">errs</span> <span style="color:#000;font-weight:bold">{</span>
</span></span><span style="display:flex;"><span> <span style="color:#000">log</span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">Errorf</span><span style="color:#000;font-weight:bold">(</span><span style="color:#4e9a06">&#34;Options error: %s.&#34;</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#000">e</span><span style="color:#000;font-weight:bold">)</span>
</span></span><span style="display:flex;"><span> <span style="color:#000;font-weight:bold">}</span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">return</span> <span style="color:#204a87;font-weight:bold">nil</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#000">fmt</span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">Errorf</span><span style="color:#000;font-weight:bold">(</span><span style="color:#4e9a06">&#34;opts are illegal&#34;</span><span style="color:#000;font-weight:bold">)</span>
</span></span><span style="display:flex;"><span> <span style="color:#000;font-weight:bold">}</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#000">content</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#000">err</span> <span style="color:#ce5c00;font-weight:bold">:=</span> <span style="color:#000">os</span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">ReadFile</span><span style="color:#000;font-weight:bold">(</span><span style="color:#000">opts</span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">Filename</span><span style="color:#000;font-weight:bold">)</span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">if</span> <span style="color:#000">err</span> <span style="color:#ce5c00;font-weight:bold">!=</span> <span style="color:#204a87;font-weight:bold">nil</span> <span style="color:#000;font-weight:bold">{</span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">if</span> <span style="color:#000">strings</span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">Contains</span><span style="color:#000;font-weight:bold">(</span><span style="color:#000">err</span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">Error</span><span style="color:#000;font-weight:bold">(),</span> <span style="color:#4e9a06">&#34;no such file or directory&#34;</span><span style="color:#000;font-weight:bold">)</span> <span style="color:#000;font-weight:bold">{</span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">return</span> <span style="color:#204a87;font-weight:bold">nil</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#204a87;font-weight:bold">nil</span>
</span></span><span style="display:flex;"><span> <span style="color:#000;font-weight:bold">}</span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">return</span> <span style="color:#204a87;font-weight:bold">nil</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#000">err</span>
</span></span><span style="display:flex;"><span> <span style="color:#000;font-weight:bold">}</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#000">status</span> <span style="color:#ce5c00;font-weight:bold">:=</span> <span style="color:#204a87;font-weight:bold">map</span><span style="color:#000;font-weight:bold">[</span><span style="color:#204a87;font-weight:bold">string</span><span style="color:#000;font-weight:bold">]</span><span style="color:#204a87;font-weight:bold">interface</span><span style="color:#000;font-weight:bold">{}{</span>
</span></span><span style="display:flex;"><span> <span style="color:#4e9a06">&#34;filename&#34;</span><span style="color:#000;font-weight:bold">:</span> <span style="color:#000">opts</span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">Filename</span><span style="color:#000;font-weight:bold">,</span>
</span></span><span style="display:flex;"><span> <span style="color:#4e9a06">&#34;content&#34;</span><span style="color:#000;font-weight:bold">:</span> <span style="color:#204a87">string</span><span style="color:#000;font-weight:bold">(</span><span style="color:#000">content</span><span style="color:#000;font-weight:bold">),</span>
</span></span><span style="display:flex;"><span> <span style="color:#000;font-weight:bold">}</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">return</span> <span style="color:#000">status</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#204a87;font-weight:bold">nil</span>
</span></span><span style="display:flex;"><span><span style="color:#000;font-weight:bold">}</span>
</span></span></code></pre></div><p>On the return value:</p>
<ul>
<li>If the file doesn&rsquo;t exist, return nil, no error, because it means the resource hasn&rsquo;t been created yet (or deleted by somebody)</li>
<li>If the file exists, return the &ldquo;status&rdquo; of it and no error.</li>
<li>Otherwise, return nil and the error.</li>
</ul>
<h3 id="update">Update()</h3>
<p><code>Update</code> will be triggered if <code>Read</code> returns a different result than what&rsquo;s recorded in the <em>State</em>.</p>
<p>For the implementation, since we are updating a file, it&rsquo;s the same as <code>Create</code>, so we can actually reuse it here without duplicated code:</p>
<p><code>internal/pkg/plugin/localfile/update.go</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#204a87;font-weight:bold">package</span> <span style="color:#000">localfile</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#204a87;font-weight:bold">func</span> <span style="color:#000">Update</span><span style="color:#000;font-weight:bold">(</span><span style="color:#000">options</span> <span style="color:#204a87;font-weight:bold">map</span><span style="color:#000;font-weight:bold">[</span><span style="color:#204a87;font-weight:bold">string</span><span style="color:#000;font-weight:bold">]</span><span style="color:#204a87;font-weight:bold">interface</span><span style="color:#000;font-weight:bold">{})</span> <span style="color:#000;font-weight:bold">(</span><span style="color:#204a87;font-weight:bold">map</span><span style="color:#000;font-weight:bold">[</span><span style="color:#204a87;font-weight:bold">string</span><span style="color:#000;font-weight:bold">]</span><span style="color:#204a87;font-weight:bold">interface</span><span style="color:#000;font-weight:bold">{},</span> <span style="color:#204a87;font-weight:bold">error</span><span style="color:#000;font-weight:bold">)</span> <span style="color:#000;font-weight:bold">{</span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">return</span> <span style="color:#000">Create</span><span style="color:#000;font-weight:bold">(</span><span style="color:#000">options</span><span style="color:#000;font-weight:bold">)</span>
</span></span><span style="display:flex;"><span><span style="color:#000;font-weight:bold">}</span>
</span></span></code></pre></div><h3 id="delete">Delete()</h3>
<p>Last but not least, let&rsquo;s implement the <code>Delete</code> interface.</p>
<p><code>internal/pkg/plugin/localfile/delete.go</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#204a87;font-weight:bold">package</span> <span style="color:#000">localfile</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#204a87;font-weight:bold">import</span> <span style="color:#000;font-weight:bold">(</span>
</span></span><span style="display:flex;"><span> <span style="color:#4e9a06">&#34;fmt&#34;</span>
</span></span><span style="display:flex;"><span> <span style="color:#4e9a06">&#34;os&#34;</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#4e9a06">&#34;github.com/mitchellh/mapstructure&#34;</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#4e9a06">&#34;github.com/devstream-io/devstream/pkg/util/log&#34;</span>
</span></span><span style="display:flex;"><span><span style="color:#000;font-weight:bold">)</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#204a87;font-weight:bold">func</span> <span style="color:#000">Delete</span><span style="color:#000;font-weight:bold">(</span><span style="color:#000">options</span> <span style="color:#204a87;font-weight:bold">map</span><span style="color:#000;font-weight:bold">[</span><span style="color:#204a87;font-weight:bold">string</span><span style="color:#000;font-weight:bold">]</span><span style="color:#204a87;font-weight:bold">interface</span><span style="color:#000;font-weight:bold">{})</span> <span style="color:#000;font-weight:bold">(</span><span style="color:#204a87;font-weight:bold">bool</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#204a87;font-weight:bold">error</span><span style="color:#000;font-weight:bold">)</span> <span style="color:#000;font-weight:bold">{</span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">var</span> <span style="color:#000">opts</span> <span style="color:#000">Options</span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">if</span> <span style="color:#000">err</span> <span style="color:#ce5c00;font-weight:bold">:=</span> <span style="color:#000">mapstructure</span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">Decode</span><span style="color:#000;font-weight:bold">(</span><span style="color:#000">options</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#ce5c00;font-weight:bold">&amp;</span><span style="color:#000">opts</span><span style="color:#000;font-weight:bold">);</span> <span style="color:#000">err</span> <span style="color:#ce5c00;font-weight:bold">!=</span> <span style="color:#204a87;font-weight:bold">nil</span> <span style="color:#000;font-weight:bold">{</span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">return</span> <span style="color:#204a87;font-weight:bold">false</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#000">err</span>
</span></span><span style="display:flex;"><span> <span style="color:#000;font-weight:bold">}</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">if</span> <span style="color:#000">errs</span> <span style="color:#ce5c00;font-weight:bold">:=</span> <span style="color:#000">validate</span><span style="color:#000;font-weight:bold">(</span><span style="color:#ce5c00;font-weight:bold">&amp;</span><span style="color:#000">opts</span><span style="color:#000;font-weight:bold">);</span> <span style="color:#204a87">len</span><span style="color:#000;font-weight:bold">(</span><span style="color:#000">errs</span><span style="color:#000;font-weight:bold">)</span> <span style="color:#ce5c00;font-weight:bold">!=</span> <span style="color:#0000cf;font-weight:bold">0</span> <span style="color:#000;font-weight:bold">{</span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">for</span> <span style="color:#000">_</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#000">e</span> <span style="color:#ce5c00;font-weight:bold">:=</span> <span style="color:#204a87;font-weight:bold">range</span> <span style="color:#000">errs</span> <span style="color:#000;font-weight:bold">{</span>
</span></span><span style="display:flex;"><span> <span style="color:#000">log</span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">Errorf</span><span style="color:#000;font-weight:bold">(</span><span style="color:#4e9a06">&#34;Options error: %s.&#34;</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#000">e</span><span style="color:#000;font-weight:bold">)</span>
</span></span><span style="display:flex;"><span> <span style="color:#000;font-weight:bold">}</span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">return</span> <span style="color:#204a87;font-weight:bold">false</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#000">fmt</span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">Errorf</span><span style="color:#000;font-weight:bold">(</span><span style="color:#4e9a06">&#34;opts are illegal&#34;</span><span style="color:#000;font-weight:bold">)</span>
</span></span><span style="display:flex;"><span> <span style="color:#000;font-weight:bold">}</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#000">err</span> <span style="color:#ce5c00;font-weight:bold">:=</span> <span style="color:#000">os</span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">Remove</span><span style="color:#000;font-weight:bold">(</span><span style="color:#000">opts</span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">Filename</span><span style="color:#000;font-weight:bold">)</span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">if</span> <span style="color:#000">err</span> <span style="color:#ce5c00;font-weight:bold">!=</span> <span style="color:#204a87;font-weight:bold">nil</span> <span style="color:#000;font-weight:bold">{</span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">return</span> <span style="color:#204a87;font-weight:bold">false</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#000">err</span>
</span></span><span style="display:flex;"><span> <span style="color:#000;font-weight:bold">}</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span> <span style="color:#204a87;font-weight:bold">return</span> <span style="color:#204a87;font-weight:bold">true</span><span style="color:#000;font-weight:bold">,</span> <span style="color:#204a87;font-weight:bold">nil</span>
</span></span><span style="display:flex;"><span><span style="color:#000;font-weight:bold">}</span>
</span></span></code></pre></div><p>Just delete the file; nothing to look at here.</p>
<h2 id="build">Build</h2>
<p>Now that we finished coding, let&rsquo;s build our new plugin. Our Makefile now supports building a specific plugin only:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>make build-plugin.localfile
</span></span></code></pre></div><h2 id="test">Test</h2>
<p>First, let&rsquo;s create a config file <code>config-localfile-test.yaml</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#204a87;font-weight:bold">tools</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span>- <span style="color:#204a87;font-weight:bold">name</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">my-file</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#204a87;font-weight:bold">plugin</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">localfile</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#204a87;font-weight:bold">options</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#204a87;font-weight:bold">filename</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">foo.txt</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#204a87;font-weight:bold">content</span><span style="color:#000;font-weight:bold">:</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#4e9a06">&#34;hello, world&#34;</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span></code></pre></div><p>Now, let&rsquo;s <code>apply</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>$ ./dtm -y apply -f config-localfile-test.yaml
</span></span><span style="display:flex;"><span>2022-03-29 11:25:17 ℹ <span style="color:#ce5c00;font-weight:bold">[</span>INFO<span style="color:#ce5c00;font-weight:bold">]</span> Apply started.
</span></span><span style="display:flex;"><span>2022-03-29 11:25:17 ℹ <span style="color:#ce5c00;font-weight:bold">[</span>INFO<span style="color:#ce5c00;font-weight:bold">]</span> Using dir &lt;.devstream&gt; to store plugins.
</span></span><span style="display:flex;"><span>2022-03-29 11:25:17 ℹ <span style="color:#ce5c00;font-weight:bold">[</span>INFO<span style="color:#ce5c00;font-weight:bold">]</span> Tool my-file <span style="color:#ce5c00;font-weight:bold">(</span>localfile<span style="color:#ce5c00;font-weight:bold">)</span> found in config but doesn<span style="color:#a40000">&#39;</span>t exist in the state, will be created.
</span></span><span style="display:flex;"><span>2022-03-29 11:25:17 ℹ <span style="color:#ce5c00;font-weight:bold">[</span>INFO<span style="color:#ce5c00;font-weight:bold">]</span> Start executing the plan.
</span></span><span style="display:flex;"><span>2022-03-29 11:25:17 ℹ <span style="color:#ce5c00;font-weight:bold">[</span>INFO<span style="color:#ce5c00;font-weight:bold">]</span> Changes count: 1.
</span></span><span style="display:flex;"><span>2022-03-29 11:25:17 ℹ <span style="color:#ce5c00;font-weight:bold">[</span>INFO<span style="color:#ce5c00;font-weight:bold">]</span> -------------------- <span style="color:#ce5c00;font-weight:bold">[</span> Processing progress: 1/1. <span style="color:#ce5c00;font-weight:bold">]</span> --------------------
</span></span><span style="display:flex;"><span>2022-03-29 11:25:17 ℹ <span style="color:#ce5c00;font-weight:bold">[</span>INFO<span style="color:#ce5c00;font-weight:bold">]</span> Processing: my-file <span style="color:#ce5c00;font-weight:bold">(</span>localfile<span style="color:#ce5c00;font-weight:bold">)</span> -&gt; Create ...
</span></span><span style="display:flex;"><span>2022-03-29 11:25:17 ✔ <span style="color:#ce5c00;font-weight:bold">[</span>SUCCESS<span style="color:#ce5c00;font-weight:bold">]</span> Plugin my-file<span style="color:#ce5c00;font-weight:bold">(</span>localfile<span style="color:#ce5c00;font-weight:bold">)</span> Create <span style="color:#204a87;font-weight:bold">done</span>.
</span></span><span style="display:flex;"><span>2022-03-29 11:25:17 ℹ <span style="color:#ce5c00;font-weight:bold">[</span>INFO<span style="color:#ce5c00;font-weight:bold">]</span> -------------------- <span style="color:#ce5c00;font-weight:bold">[</span> Processing <span style="color:#204a87;font-weight:bold">done</span>. <span style="color:#ce5c00;font-weight:bold">]</span> --------------------
</span></span><span style="display:flex;"><span>2022-03-29 11:25:17 ✔ <span style="color:#ce5c00;font-weight:bold">[</span>SUCCESS<span style="color:#ce5c00;font-weight:bold">]</span> All plugins applied successfully.
</span></span><span style="display:flex;"><span>2022-03-29 11:25:17 ✔ <span style="color:#ce5c00;font-weight:bold">[</span>SUCCESS<span style="color:#ce5c00;font-weight:bold">]</span> Apply finished.
</span></span></code></pre></div><p>As we can see, the plugin has created our file successfully. To verify, you can open the &ldquo;foo.txt&rdquo; file to have a look.</p>
<p>If we <code>apply</code> again, nothing should happen, since the file is already created with the desired content:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>$ ./dtm -y apply -f config-localfile-test.yaml
</span></span><span style="display:flex;"><span>2022-03-29 11:30:43 ℹ <span style="color:#ce5c00;font-weight:bold">[</span>INFO<span style="color:#ce5c00;font-weight:bold">]</span> Apply started.
</span></span><span style="display:flex;"><span>2022-03-29 11:30:43 ℹ <span style="color:#ce5c00;font-weight:bold">[</span>INFO<span style="color:#ce5c00;font-weight:bold">]</span> Using dir &lt;.devstream&gt; to store plugins.
</span></span><span style="display:flex;"><span>2022-03-29 11:30:45 ℹ <span style="color:#ce5c00;font-weight:bold">[</span>INFO<span style="color:#ce5c00;font-weight:bold">]</span> No changes <span style="color:#204a87;font-weight:bold">done</span> since last apply. There is nothing to <span style="color:#204a87;font-weight:bold">do</span>.
</span></span><span style="display:flex;"><span>2022-03-29 11:30:45 ✔ <span style="color:#ce5c00;font-weight:bold">[</span>SUCCESS<span style="color:#ce5c00;font-weight:bold">]</span> Apply finished.
</span></span></code></pre></div><p>But, what if somebody changed the content of &ldquo;foo.txt&rdquo;? Let&rsquo;s experiment:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>$ <span style="color:#204a87">echo</span> <span style="color:#4e9a06">&#34;changed&#34;</span> &gt; foo.txt
</span></span><span style="display:flex;"><span>$ ./dtm -y apply -f config-localfile-test.yaml
</span></span><span style="display:flex;"><span>2022-03-29 11:26:40 ℹ <span style="color:#ce5c00;font-weight:bold">[</span>INFO<span style="color:#ce5c00;font-weight:bold">]</span> Apply started.
</span></span><span style="display:flex;"><span>2022-03-29 11:26:40 ℹ <span style="color:#ce5c00;font-weight:bold">[</span>INFO<span style="color:#ce5c00;font-weight:bold">]</span> Using dir &lt;.devstream&gt; to store plugins.
</span></span><span style="display:flex;"><span>2022-03-29 11:26:40 ℹ <span style="color:#ce5c00;font-weight:bold">[</span>INFO<span style="color:#ce5c00;font-weight:bold">]</span> Tool my-file <span style="color:#ce5c00;font-weight:bold">(</span>localfile<span style="color:#ce5c00;font-weight:bold">)</span> drifted from the state, will be updated.
</span></span><span style="display:flex;"><span>2022-03-29 11:26:40 ℹ <span style="color:#ce5c00;font-weight:bold">[</span>INFO<span style="color:#ce5c00;font-weight:bold">]</span> Start executing the plan.
</span></span><span style="display:flex;"><span>2022-03-29 11:26:40 ℹ <span style="color:#ce5c00;font-weight:bold">[</span>INFO<span style="color:#ce5c00;font-weight:bold">]</span> Changes count: 1.
</span></span><span style="display:flex;"><span>2022-03-29 11:26:40 ℹ <span style="color:#ce5c00;font-weight:bold">[</span>INFO<span style="color:#ce5c00;font-weight:bold">]</span> -------------------- <span style="color:#ce5c00;font-weight:bold">[</span> Processing progress: 1/1. <span style="color:#ce5c00;font-weight:bold">]</span> --------------------
</span></span><span style="display:flex;"><span>2022-03-29 11:26:40 ℹ <span style="color:#ce5c00;font-weight:bold">[</span>INFO<span style="color:#ce5c00;font-weight:bold">]</span> Processing: my-file <span style="color:#ce5c00;font-weight:bold">(</span>localfile<span style="color:#ce5c00;font-weight:bold">)</span> -&gt; Update ...
</span></span><span style="display:flex;"><span>2022-03-29 11:26:40 ✔ <span style="color:#ce5c00;font-weight:bold">[</span>SUCCESS<span style="color:#ce5c00;font-weight:bold">]</span> Plugin my-file<span style="color:#ce5c00;font-weight:bold">(</span>localfile<span style="color:#ce5c00;font-weight:bold">)</span> Update <span style="color:#204a87;font-weight:bold">done</span>.