Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding WATER reverse listener #622

Merged
merged 30 commits into from
Jul 24, 2024
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
8815ecc
feat: adding WATER dependencies
WendelHime Jun 17, 2024
48a5011
feat: WATER reverse listener along with wasm directory for local test…
WendelHime Jun 17, 2024
7c8a42a
feat: adding WATER listener and parameters
WendelHime Jun 17, 2024
312b6b7
feat: updating reverse connection handler to copy connection pipe so …
WendelHime Jun 17, 2024
41d5cf5
feat: updating reverse connection handler to handlr client/target err…
WendelHime Jun 17, 2024
f37f21a
chore: making water reverse handler to accept connection and send it …
WendelHime Jun 17, 2024
a340009
Merge branch 'main' of github.com:getlantern/http-proxy into feat/1437
WendelHime Jun 18, 2024
5d88f5c
fix: receiving WASM as base64 and decoding before using it
WendelHime Jun 24, 2024
ab0da0f
feat: moving wasm files to testdata directory inside water package
WendelHime Jun 26, 2024
78c4417
fix: removing local listeners from water and just use the generated w…
WendelHime Jun 26, 2024
89c8823
fix: renaming function calls to refer to WATER listener instead of sp…
WendelHime Jun 26, 2024
4fb3a82
fix: renaming reverse files to listener
WendelHime Jun 26, 2024
80ac30a
fix: update WATER version to 0.7.0-alpha for supporting transport v1
WendelHime Jun 26, 2024
828ecdf
fix: adding reverse_v1.wasm
WendelHime Jun 26, 2024
c167102
chore: removing unused WASM files
WendelHime Jun 26, 2024
cad117e
feat: update WATER package for using transport v1 and wrap listener
WendelHime Jun 26, 2024
ff58803
feat: use v1 NewListenerWithContext instead of directly calling from …
WendelHime Jun 26, 2024
a850e2d
fix: updating test for dialing to the base listener instead of direct…
WendelHime Jun 26, 2024
0a353f4
fix: updating go.mod required version to 1.22 (just to see if it pass…
WendelHime Jun 26, 2024
59e856d
Revert "fix: updating go.mod required version to 1.22 (just to see if…
WendelHime Jun 26, 2024
8a52fd2
fix: keep using water instead of directly referring to v1
WendelHime Jun 26, 2024
1b44bc4
fix: removing wrapped listener and starting WATER listener directly w…
WendelHime Jun 27, 2024
091fc5f
fix: re-wrap water listener as a multiplexed transport
WendelHime Jun 28, 2024
b06da11
chore: removing old version water
WendelHime Jul 2, 2024
e61f6f5
feat: overriding WATER logger
WendelHime Jul 2, 2024
be75ded
fix: removing golog.Logger reference since it's a interface
WendelHime Jul 2, 2024
690ec0d
fix: removing golog.Logger reference when creating slog logger
WendelHime Jul 2, 2024
ccbc458
feat: adding WATER transport field
WendelHime Jul 2, 2024
533717a
fix: adding missing transport parameter
WendelHime Jul 2, 2024
d25d660
chore: removing multiplex from water
WendelHime Jul 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 10 additions & 4 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -48,9 +48,8 @@ require (
github.com/hashicorp/golang-lru v0.5.4
github.com/mitchellh/panicwrap v1.0.0
github.com/op/go-logging v0.0.0-20160315200505-970db520ece7
github.com/oschwald/geoip2-golang v1.8.0
github.com/prometheus/client_golang v1.19.1
github.com/refraction-networking/utls v1.3.3
github.com/refraction-networking/water v0.7.0-alpha
github.com/siddontang/go v0.0.0-20180604090527-bdc77568d726
github.com/spaolacci/murmur3 v1.1.0
github.com/stretchr/testify v1.8.4
Expand Down Expand Up @@ -80,6 +79,7 @@ require (
github.com/aead/ecdh v0.2.0 // indirect
github.com/andybalholm/brotli v1.0.5 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/blang/vfs v1.0.0 // indirect
github.com/cenkalti/backoff/v4 v4.2.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
Expand All @@ -89,6 +89,7 @@ require (
github.com/dvyukov/go-fuzz v0.0.0-20210429054444-fca39067bc72 // indirect
github.com/felixge/httpsnoop v1.0.3 // indirect
github.com/gaukas/godicttls v0.0.4 // indirect
github.com/gaukas/wazerofs v0.1.0 // indirect
github.com/getlantern/algeneva v0.0.0-20240222191137-2b4e88234f59 // indirect
github.com/getlantern/bufconn v0.0.0-20190625204133-a08544339f8d // indirect
github.com/getlantern/byteexec v0.0.0-20220903142956-e6ed20032cfd // indirect
Expand Down Expand Up @@ -131,6 +132,7 @@ require (
github.com/mmcloughlin/avo v0.0.0-20200803215136-443f81d77104 // indirect
github.com/nwaples/rardecode v1.1.2 // indirect
github.com/onsi/ginkgo/v2 v2.12.0 // indirect
github.com/oschwald/geoip2-golang v1.8.0 // indirect
github.com/oschwald/maxminddb-golang v1.10.0 // indirect
github.com/oxtoacart/bpool v0.0.0-20190530202638-03653db5a59c // indirect
github.com/pierrec/lz4/v4 v4.1.12 // indirect
Expand All @@ -152,6 +154,7 @@ require (
github.com/pion/webrtc/v3 v3.2.6 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/prometheus/client_golang v1.19.1 // indirect
github.com/prometheus/client_model v0.5.0 // indirect
github.com/prometheus/common v0.48.0 // indirect
github.com/prometheus/procfs v0.12.0 // indirect
Expand All @@ -162,6 +165,7 @@ require (
github.com/songgao/water v0.0.0-20200317203138-2b4b6d7c09d8 // indirect
github.com/templexxx/cpu v0.0.8 // indirect
github.com/templexxx/xorsimd v0.4.1 // indirect
github.com/tetratelabs/wazero v1.7.1 // indirect
github.com/ti-mo/conntrack v0.3.0 // indirect
github.com/ti-mo/netfilter v0.3.1 // indirect
github.com/tjfoc/gmsm v1.3.2 // indirect
Expand All @@ -180,13 +184,13 @@ require (
go.uber.org/multierr v1.8.0 // indirect
go.uber.org/zap v1.21.0 // indirect
golang.org/x/crypto v0.18.0 // indirect
golang.org/x/exp v0.0.0-20240110193028-0dcbfd608b1e // indirect
golang.org/x/exp v0.0.0-20240119083558-1b970713d09a // indirect
golang.org/x/mod v0.14.0 // indirect
golang.org/x/oauth2 v0.16.0 // indirect
golang.org/x/sync v0.6.0 // indirect
golang.org/x/sys v0.17.0 // indirect
golang.org/x/text v0.14.0 // indirect
golang.org/x/tools v0.16.0 // indirect
golang.org/x/tools v0.17.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20231002182017-d307bd883b97 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20231002182017-d307bd883b97 // indirect
Expand All @@ -199,3 +203,5 @@ require (

// Waiting on https://github.com/mitchellh/panicwrap/pull/27 to be merged upstream
replace github.com/mitchellh/panicwrap v1.0.0 => github.com/getlantern/panicwrap v0.0.0-20200707191944-9ba45baf8e51

replace github.com/tetratelabs/wazero => github.com/refraction-networking/wazero v1.7.1-w
18 changes: 14 additions & 4 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@ github.com/benbjohnson/clock v1.1.0 h1:Q92kusRqC1XV2MjkWETPvjJVqKetz1OzxZB7mHJLj
github.com/benbjohnson/clock v1.1.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/blang/vfs v1.0.0 h1:AUZUgulCDzbaNjTRWEP45X7m/J10brAptZpSRKRZBZc=
github.com/blang/vfs v1.0.0/go.mod h1:jjuNUc/IKcRNNWC9NUCvz4fR9PZLPIKxEygtPs/4tSI=
github.com/cenkalti/backoff/v4 v4.2.1 h1:y4OZtCnogmCPw98Zjyt5a6+QwPLGkiQsYW5oUqylYbM=
github.com/cenkalti/backoff/v4 v4.2.1/go.mod h1:Y3VNntkOUPxTVeUxJ/G5vcM//AlwfmyYozVcomhLiZE=
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
Expand Down Expand Up @@ -74,6 +76,8 @@ github.com/fsnotify/fsnotify v1.6.0 h1:n+5WquG0fcWoWp6xPWfHdbskMCQaFnG6PfBrh1Ky4
github.com/fsnotify/fsnotify v1.6.0/go.mod h1:sl3t1tCWJFWoRz9R8WJCbQihKKwmorjAbSClcnxKAGw=
github.com/gaukas/godicttls v0.0.4 h1:NlRaXb3J6hAnTmWdsEKb9bcSBD6BvcIjdGdeb0zfXbk=
github.com/gaukas/godicttls v0.0.4/go.mod h1:l6EenT4TLWgTdwslVb4sEMOCf7Bv0JAK67deKr9/NCI=
github.com/gaukas/wazerofs v0.1.0 h1:wIkW1bAxSnpaaVkQ5LOb1tm1BXdVap3eKjJpVWIqt2E=
github.com/gaukas/wazerofs v0.1.0/go.mod h1:+JECB9Fwt0taPqSgHckG9lmT3tcoVK+9VJozTsq9UlI=
github.com/getlantern/algeneva v0.0.0-20240222191137-2b4e88234f59 h1:uWNy0b1Wtpsd4n64Kat+fRjvPCBwM2Nykwt71LupJAQ=
github.com/getlantern/algeneva v0.0.0-20240222191137-2b4e88234f59/go.mod h1:PrNR8tMXO26YNs8K9653XCUH7u2Kv4OdfFC3Ke1GsX0=
github.com/getlantern/broflake v0.0.0-20231117182649-7d46643a6f87 h1:9nUdYJp3TCKE/jTwI0mymz8FPT7jXHBRvRy2rXlKFDo=
Expand Down Expand Up @@ -423,6 +427,12 @@ github.com/quic-go/quic-go v0.40.0 h1:GYd1iznlKm7dpHD7pOVpUvItgMPo/jrMgDWZhMCecq
github.com/quic-go/quic-go v0.40.0/go.mod h1:PeN7kuVJ4xZbxSv/4OX6S1USOX8MJvydwpTx31vx60c=
github.com/refraction-networking/utls v1.3.3 h1:f/TBLX7KBciRyFH3bwupp+CE4fzoYKCirhdRcC490sw=
github.com/refraction-networking/utls v1.3.3/go.mod h1:DlecWW1LMlMJu+9qpzzQqdHDT/C2LAe03EdpLUz/RL8=
github.com/refraction-networking/water v0.6.4 h1:7E0nFdSyVXMZ5bz7GPHz1ALBQDUiyHbiBfPZ4eSBmIM=
github.com/refraction-networking/water v0.6.4/go.mod h1:UDjvBH4nYp0UkrJm9GJ1G96cpELH31/BIjoTiWkYoME=
github.com/refraction-networking/water v0.7.0-alpha h1:yLMAvIi0lHHD2mvwx63GLnptpjvnmBCWFvTHTWLykLk=
github.com/refraction-networking/water v0.7.0-alpha/go.mod h1:VPQENA6E5+2V5RlX/yaBlTCKeFQ25j49DUc/pmmwkXE=
github.com/refraction-networking/wazero v1.7.1-w h1:z7Ty5PsMkJEDBCsn3ELUjceQGBT0FMVGldOSpDK3giQ=
github.com/refraction-networking/wazero v1.7.1-w/go.mod h1:ytl6Zuh20R/eROuyDaGPkp82O9C/DJfXAwJfQ3X6/7Y=
github.com/riobard/go-bloom v0.0.0-20200614022211-cdc8013cb5b3 h1:f/FNXud6gA3MNr8meMVVGxhp+QBTqY91tM8HjEuMjGg=
github.com/riobard/go-bloom v0.0.0-20200614022211-cdc8013cb5b3/go.mod h1:HgjTstvQsPGkxUsCd2KWxErBblirPizecHcpD3ffK+s=
github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ=
Expand Down Expand Up @@ -550,8 +560,8 @@ golang.org/x/crypto v0.9.0/go.mod h1:yrmDGqONDYtNj3tH8X9dzUun2m2lzPa9ngI6/RUPGR0
golang.org/x/crypto v0.18.0 h1:PGVlW0xEltQnzFZ55hkuX5+KLyrMYhHld1YHO4AKcdc=
golang.org/x/crypto v0.18.0/go.mod h1:R0j02AL6hcrfOiy9T4ZYp/rcWeMxM3L6QYxlOuEG1mg=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20240110193028-0dcbfd608b1e h1:723BNChdd0c2Wk6WOE320qGBiPtYx0F0Bbm1kriShfE=
golang.org/x/exp v0.0.0-20240110193028-0dcbfd608b1e/go.mod h1:iRJReGqOEeBhDZGkGbynYwcHlctCvnjTYIamk7uXpHI=
golang.org/x/exp v0.0.0-20240119083558-1b970713d09a h1:Q8/wZp0KX97QFTc2ywcOE0YRjZPVIx+MXInMzdvQqcA=
golang.org/x/exp v0.0.0-20240119083558-1b970713d09a/go.mod h1:idGWGoKP1toJGkd5/ig9ZLuPcZBC3ewk7SzmH0uou08=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
Expand Down Expand Up @@ -677,8 +687,8 @@ golang.org/x/tools v0.0.0-20201224043029-2b0845dc783e/go.mod h1:emZCQorbCU4vsT4f
golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/tools v0.16.0 h1:GO788SKMRunPIBCXiQyo2AaexLstOrVhuAL5YwsckQM=
golang.org/x/tools v0.16.0/go.mod h1:kYVVN6I1mBNoB1OX+noeBjbRk4IUEPa7JJ+TJMEooJ0=
golang.org/x/tools v0.17.0 h1:FvmRgNOcs3kOa+T20R1uhfP9F6HgG2mfxDv1vrx1Htc=
golang.org/x/tools v0.17.0/go.mod h1:xsh6VxdV005rRVaS6SSAf9oiAqljS7UZUacMZ8Bnsps=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
Expand Down
5 changes: 5 additions & 0 deletions http-proxy/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,9 @@ var (

algenevaAddr = flag.String("algeneva-addr", "", "Address at which to listen for algenAddr connections.")

waterAddr = flag.String("water-addr", "", "Address at which to listen for WATER connections.")
waterWASM = flag.String("water-wasm", "", "Base64 encoded WASM for WATER")

track = flag.String("track", "", "The track this proxy is running on")
)

Expand Down Expand Up @@ -469,6 +472,8 @@ func main() {
BroflakeCert: os.Getenv("BROFLAKE_CERT"),
BroflakeKey: os.Getenv("BROFLAKE_KEY"),
AlgenevaAddr: *algenevaAddr,
WaterAddr: *waterAddr,
WaterWASM: *waterWASM,
}
if *maxmindLicenseKey != "" {
log.Debug("Will use Maxmind for geolocating clients")
Expand Down
25 changes: 25 additions & 0 deletions http_proxy.go
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ import (
"github.com/getlantern/http-proxy-lantern/v2/tlslistener"
"github.com/getlantern/http-proxy-lantern/v2/tlsmasq"
"github.com/getlantern/http-proxy-lantern/v2/tokenfilter"
"github.com/getlantern/http-proxy-lantern/v2/water"
"github.com/getlantern/http-proxy-lantern/v2/wss"

algeneva "github.com/getlantern/lantern-algeneva"
Expand Down Expand Up @@ -184,6 +185,9 @@ type Proxy struct {

AlgenevaAddr string

WaterAddr string
WaterWASM string

throttleConfig throttle.Config
instrument instrument.Instrument
}
Expand Down Expand Up @@ -649,6 +653,8 @@ func (p *Proxy) buildOTELOpts(endpoint string, includeProxyName bool) *otel.Opts
opts.Addr = p.BroflakeAddr
} else if p.AlgenevaAddr != "" {
opts.Addr = p.AlgenevaAddr
} else if p.WaterAddr != "" {
opts.Addr = p.WaterAddr
}
if includeProxyName {
opts.ProxyName = proxyName
Expand Down Expand Up @@ -960,6 +966,25 @@ func (p *Proxy) listenAlgeneva(baseListen func(string) (net.Listener, error)) li
}
}

func (p *Proxy) listenWATER(baseListen func(string) (net.Listener, error)) listenerBuilderFN {
return func(addr string) (net.Listener, error) {
ctx := context.Background()

base, err := baseListen(addr)
if err != nil {
return nil, err
}

waterListener, err := water.NewWATERListener(ctx, base, p.WaterWASM)
if err != nil {
return nil, err
}

log.Debugf("Listening for water at %v", waterListener.Addr())
return waterListener, nil
}
}

func (p *Proxy) setupPacketForward() error {
if runtime.GOOS != "linux" {
log.Debugf("Ignoring packet forward on %v", runtime.GOOS)
Expand Down
1 change: 1 addition & 0 deletions protoListeners.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,5 +29,6 @@ func getProtoListenersArgs(p *Proxy) []protoListenerArgs {
p.ShadowsocksMultiplexAddr,
p.wrapMultiplexing(p.listenShadowsocks),
},
{"water", p.WaterAddr, p.wrapMultiplexing(p.listenWATER(p.listenTCP))},
}
}
35 changes: 35 additions & 0 deletions water/listener.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
package water

import (
"context"
"encoding/base64"
"net"

"github.com/getlantern/golog"
"github.com/refraction-networking/water"
_ "github.com/refraction-networking/water/transport/v1"
)

var log = golog.LoggerFor("water")

func NewWATERListener(ctx context.Context, baseListener net.Listener, wasm string) (net.Listener, error) {
decodedWASM, err := base64.StdEncoding.DecodeString(wasm)
if err != nil {
log.Errorf("failed to decode WASM base64: %v", err)
return nil, err
}

cfg := &water.Config{
TransportModuleBin: decodedWASM,
NetworkListener: baseListener,
ModuleConfigFactory: water.NewWazeroModuleConfigFactory(),
}

waterListener, err := water.NewListenerWithContext(ctx, cfg)
if err != nil {
log.Errorf("error creating water listener: %v", err)
return nil, err
}

return waterListener, nil
}
92 changes: 92 additions & 0 deletions water/listener_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
package water

import (
"bytes"
"context"
"embed"
"encoding/base64"
"io"
"net"
"testing"

"github.com/refraction-networking/water"
_ "github.com/refraction-networking/water/transport/v1"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)

//go:embed testdata/*
var testData embed.FS

func TestWATERListener(t *testing.T) {
f, err := testData.Open("testdata/reverse_v1.wasm")
require.Nil(t, err)

wasm, err := io.ReadAll(f)
require.Nil(t, err)

b64WASM := base64.StdEncoding.EncodeToString(wasm)

ctx := context.Background()

cfg := &water.Config{
TransportModuleBin: wasm,
}

l0, err := net.ListenTCP("tcp", &net.TCPAddr{IP: net.ParseIP("127.0.0.1"), Port: 0})
require.Nil(t, err, "ListenTCP failed: %v", err)

ll, err := NewWATERListener(ctx, l0, b64WASM)
require.Nil(t, err)

messageRequest := "hello"
expectedResponse := "world"
// running listener
go func() {
for {
var conn net.Conn
conn, err = ll.Accept()
if err != nil {
t.Error(err)
return
}

go func() {
if conn == nil {
log.Error("nil connection")
return
}
buf := make([]byte, 2*len(messageRequest))
n, err := conn.Read(buf)
if err != nil {
log.Errorf("error reading: %v", err)
return
}

buf = buf[:n]
if !bytes.Equal(buf, []byte(messageRequest)) {
log.Errorf("unexpected request %v %v", buf, messageRequest)
return
}
conn.Write([]byte(expectedResponse))
}()
}
}()

dialer, err := water.NewDialerWithContext(ctx, cfg)
require.Nil(t, err)

conn, err := dialer.DialContext(ctx, "tcp", l0.Addr().String())
require.Nil(t, err)
defer conn.Close()

n, err := conn.Write([]byte(messageRequest))
assert.Nil(t, err)
assert.Equal(t, len(messageRequest), n)

buf := make([]byte, 1024)
n, err = conn.Read(buf)
assert.Nil(t, err)
assert.Equal(t, len(expectedResponse), n)
assert.Equal(t, expectedResponse, string(buf[:n]))
}
Binary file added water/testdata/reverse_v1.wasm
Binary file not shown.
Loading