Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Implement Graceful shutdown #21

Merged
merged 34 commits into from
Oct 30, 2024
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
a4e70ff
docs: Added more documentation to functions and objects in the project
werniq Oct 21, 2024
bb9b964
fix: Repaired dockerfile and added commands in Makefile to build/push…
werniq Oct 21, 2024
14cda9c
feat: Added Taskfile for the owner to compare it with Makefile
werniq Oct 21, 2024
097300a
fix: Removed redundant clock package, changed all calls to time.Now()
werniq Oct 21, 2024
292d214
Completely removed clock package
werniq Oct 21, 2024
4f24899
refactor: Adding more descriptive error messages, and renaming vars
werniq Oct 21, 2024
06ba9e1
Made visualizer and proxy server ports as flags, to allow user specif…
werniq Oct 21, 2024
317e5c7
Handling errors during certificate loading.
werniq Oct 21, 2024
9cb3ba9
Fix: removed redundant commans at the end of line.
werniq Oct 21, 2024
1ec0180
Handlded error in proxy/main.go
werniq Oct 21, 2024
617c0bd
Merge pull request #1 from werniq/feature/documentation
werniq Oct 21, 2024
2b7a753
Merge pull request #2 from werniq/feature/docs-and-clean-code
werniq Oct 22, 2024
581314b
Implemented graceful shutdown in collector and proxy
werniq Oct 22, 2024
d4397db
Implemented graceful shutdown in server/server.go
werniq Oct 22, 2024
fc9ddfb
Merge pull request #3 from werniq/feature/graceful-shutdown
werniq Oct 22, 2024
3ebae2c
fix: resolved merge conflicts
werniq Oct 22, 2024
2bc082c
Merge branch 'main' of https://github.com/werniq/waffle
werniq Oct 22, 2024
b9f1dc6
fix: Fixed CI/CD
werniq Oct 22, 2024
19336b4
Merge branch 'feature/graceful-shutdown' of https://github.com/werniq…
werniq Oct 22, 2024
7173f7e
fix: Repaired github action, and building docker image instead of pro…
werniq Oct 22, 2024
9dc854a
fix: Commented unused code
werniq Oct 22, 2024
ea9ac7d
Running tests only in internal package instead of ./...
werniq Oct 22, 2024
df6733f
fix: Running tests in every folder in internal pkg
werniq Oct 22, 2024
f57c01d
fix: Repairing Github Pipeline; Added pcap installation
werniq Oct 22, 2024
b6c965d
fix: Repairing Github Pipeline; Added sudo to pcap install
werniq Oct 22, 2024
7a79fa6
Testing everything in waffle folder, since pcap error is resolved.
werniq Oct 22, 2024
0ebb7d3
changed username in docker image to YOUR_USERNAME
werniq Oct 24, 2024
44cc1e9
Removed unused packages from project.
werniq Oct 24, 2024
2f1d9ac
Repaired taskfile and added description to all tasks
werniq Oct 24, 2024
9751d02
fix: Building application binary instead of docker, since we should l…
werniq Oct 24, 2024
042fa0e
fixed task name from dockerBuild to build
werniq Oct 24, 2024
a0725ea
fix: Changed output path in build task
werniq Oct 24, 2024
d1eaa5f
Changed flag name to avoid duplication and added better logging.
werniq Oct 24, 2024
2418a11
Merge branch 'main' into main
werniq Oct 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 1 addition & 8 deletions cmd/collector/main.go
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
package main

import (
"context"
"flag"
"log"
"time"
Expand All @@ -20,12 +19,6 @@ func main() {
)
flag.StringVar(&networkInterface, "i", defaultNetworkInterfaceDescription, "Identification of the interface")

// question: Why do we need context here? It is not used in collector.Run, except of ctx.Done, but since it is not
// context.WithTimeout (as example) it can not be closed in any way.
// Same in c.serializer.SerializePackets(ctx, packetsChan), it can not be closed there as well.
// Why not just to remove it?
ctx := context.Background()

log.Println("starting collector")

packetSerializer := packet.NewMemoryPacketSerializer(time.Minute * 5)
Expand All @@ -41,7 +34,7 @@ func main() {
packet.NewWindowsNetworkInterfaceProvider(networkInterface),
packetSerializer)

if err := collector.Run(ctx); err != nil {
if err := collector.Run(); err != nil {
log.Fatalln("Error during running collector: ", err.Error())
}
}
27 changes: 24 additions & 3 deletions cmd/proxy/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@ import (
_ "embed"
"flag"
"log"
"os"
"os/signal"
"syscall"
"time"
"waffle/cmd/proxy/server"
)

Expand All @@ -23,8 +27,25 @@ func main() {
flag.StringVar(&visualizeServerPort, "p", "8081", "Port for server to listen on")
flag.StringVar(&proxyServerPort, "p", "8081", "Port for server to listen on")

ctx := context.Background()
if err := server.Run(ctx, proxyServerPort, visualizeServerPort, yamlConfigBytes, certificates); err != nil {
log.Fatalln(err)
quit := make(chan os.Signal)

signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)

go func() {
if err := server.Run(proxyServerPort, visualizeServerPort, yamlConfigBytes, certificates); err != nil {
log.Fatalln(err)
}
}()

log.Println("Server started on :8080")
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it must be modified to the correct ports

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean like this:

image


<-quit
log.Println("Shutdown signal received, shutting down gracefully...")

ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()

if err := server.Shutdown(ctx); err != nil {
log.Fatalf("Server forced to shutdown: %s\n", err)
}
}
44 changes: 36 additions & 8 deletions cmd/proxy/server/server.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,13 @@ package server
import (
"context"
"embed"
"errors"
"fmt"
"log"
"net/http"
"os"
"os/signal"
"syscall"
"time"
"waffle/internal/visualize"

Expand Down Expand Up @@ -42,10 +45,7 @@ import (
// If the proxy server fails to start, the function logs a fatal error.
//
// The function returns nil upon normal completion.
func Run(ctx context.Context, proxyServerPort, visualizeServerPort string, yamlConfigBytes []byte, certificates embed.FS) error {
_, cancel := signal.NotifyContext(ctx, os.Interrupt)
defer cancel()

func Run(proxyServerPort, visualizeServerPort string, yamlConfigBytes []byte, certificates embed.FS) error {
_, err := config.LoadEnvironmentConfig()
if err != nil {
log.Fatal(err.Error())
Expand Down Expand Up @@ -80,7 +80,6 @@ func Run(ctx context.Context, proxyServerPort, visualizeServerPort string, yamlC
)

defender := guard.NewDefenseCoordinator([]guard.Defender{&guard.XSS{}})

limiter := ratelimit.NewInMemoryLimiter(time.Minute * 5)

visualizeServerPort = fmt.Sprintf(":%s", visualizeServerPort)
Expand All @@ -95,18 +94,47 @@ func Run(ctx context.Context, proxyServerPort, visualizeServerPort string, yamlC
)

proxyServerPort = fmt.Sprintf(":%s", proxyServerPort)

proxyServer := proxy.NewServer(proxyServerPort, certificateProvider, guardHandler)

log.Printf("Starting Waffle Proxy on port %s 🚀\n", proxyServerPort)

if err := proxyServer.Start(); err != nil {
log.Fatal(err.Error())
signalChan := make(chan os.Signal, 1)
signal.Notify(signalChan, syscall.SIGINT, syscall.SIGTERM)

ctx, cancel := context.WithCancel(context.Background())
defer cancel()

go func() {
if err := proxyServer.Start(); err != nil && !errors.Is(err, http.ErrServerClosed) {
log.Fatalf("Proxy server encountered an error: %v\n", err)
}
}()

select {
case sig := <-signalChan:
log.Printf("Received shutdown signal: %s, shutting down gracefully...", sig)
cancel()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As you can see here, when we receive signal into signalChan, we call cancel func, which closes our context, and 'graceful-shutdowning' process begins.

case <-ctx.Done():
log.Println("Context canceled, shutting down...")
}

shutdownCtx, shutdownCancel := context.WithTimeout(context.Background(), 10*time.Second)
defer shutdownCancel()

if err := proxyServer.Shutdown(shutdownCtx); err != nil {
log.Printf("Error during server shutdown: %v", err)
} else {
log.Println("Proxy server shut down gracefully.")
}

return nil
}

// Shutdown is a function to be called while 'gracefully shutdowning' the server.
func Shutdown(ctx context.Context) error {
return nil
}

// loadLocalCustomCACerts reads the local custom CA certificates from the embedded file system.
// It reads the CA certificate file (ca.crt) located in the ".cert" directory and returns it as a slice of byte slices.
// This CA certificate is used for establishing trust during TLS/SSL handshakes.
Expand Down
12 changes: 7 additions & 5 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,29 +4,31 @@ go 1.22.0

require (
github.com/Netflix/go-env v0.0.0-20220526054621-78278af1949d
github.com/cdipaolo/goml v0.0.0-20220715001353-00e0c845ae1c
github.com/corazawaf/libinjection-go v0.1.3
github.com/emirpasic/gods v1.18.1
github.com/goccy/go-yaml v1.11.3
Copy link
Contributor Author

@werniq werniq Oct 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had to run go mod tidy here, seems we have some unused imports left.

As an idea, we can add these to our CI/CD pipeline, and check if go mod tidy + git diff will return true, meaning that contributor should clean deps first.

github.com/google/gopacket v1.1.19
github.com/google/uuid v1.6.0
github.com/malaschitz/randomForest v0.0.0-20240228214944-c64ffe1648e0
github.com/patrickmn/go-cache v2.1.0+incompatible
github.com/stretchr/testify v1.9.0
golang.org/x/sys v0.17.0
nhooyr.io/websocket v1.8.10
)

require (
github.com/cdipaolo/goml v0.0.0-20220715001353-00e0c845ae1c // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/fatih/color v1.16.0 // indirect
github.com/google/gopacket v1.1.19 // indirect
github.com/malaschitz/randomForest v0.0.0-20240228214944-c64ffe1648e0 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/sjwhitworth/golearn v0.0.0-20221228163002-74ae077eafb2 // indirect
github.com/stretchr/objx v0.5.2 // indirect
golang.org/x/crypto v0.14.0 // indirect
golang.org/x/sys v0.17.0 // indirect
golang.org/x/net v0.10.0 // indirect
golang.org/x/text v0.13.0 // indirect
golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028 // indirect
gonum.org/v1/gonum v0.9.3 // indirect
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
Loading
Loading