Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] nil pointer in GetDialedIP #5799

Open
1 task done
com0t opened this issue Nov 6, 2024 · 7 comments
Open
1 task done

[BUG] nil pointer in GetDialedIP #5799

com0t opened this issue Nov 6, 2024 · 7 comments
Assignees
Labels
Investigation Something to Investigate Type: Bug Inconsistencies or issues which will cause an issue or problem for users or implementors.

Comments

@com0t
Copy link

com0t commented Nov 6, 2024

Is there an existing issue for this?

  • I have searched the existing issues.

Current Behavior

i test code and get error

goroutine 19990 [running]:
github.com/projectdiscovery/fastdialer/fastdialer.(*Dialer).GetDialedIP(0xc01a26fd08?, {0xc01a26fd08?, 0x3831260?})
        /home/labs/go/pkg/mod/github.com/projectdiscovery/[email protected]/fastdialer/dialer.go:307 +0x13
github.com/projectdiscovery/nuclei/v3/pkg/protocols/http.(*Request).executeRequest(0xc034698000, 0xc02741d380, 0xc02070e960, 0xc02741d6e0, 0x0, 0xc01bf598d8?, 0x1)
        /home/labs/go/pkg/mod/github.com/projectdiscovery/nuclei/[email protected]/pkg/protocols/http/request.go:847 +0x382f
github.com/projectdiscovery/nuclei/v3/pkg/protocols/http.(*Request).ExecuteWithResults.func1({0xc0330caa00, 0x48}, 0xc02741d860, 0xc02741d0b0)
        /home/labs/go/pkg/mod/github.com/projectdiscovery/nuclei/[email protected]/pkg/protocols/http/request.go:517 +0x613
github.com/projectdiscovery/nuclei/v3/pkg/protocols/http.(*Request).ExecuteWithResults(0xc034698000, 0xc02741d380, 0xc02741d0b0, 0xc02741d6e0, 0xc02741d770)
        /home/labs/go/pkg/mod/github.com/projectdiscovery/nuclei/[email protected]/pkg/protocols/http/request.go:589 +0x3d5
github.com/projectdiscovery/nuclei/v3/pkg/tmplexec/generic.(*Generic).ExecuteWithResults(0xc0343ff1d0, 0xc01b77c140)
        /home/labs/go/pkg/mod/github.com/projectdiscovery/nuclei/[email protected]/pkg/tmplexec/generic/exec.go:61 +0x2e3
github.com/projectdiscovery/nuclei/v3/pkg/tmplexec.(*TemplateExecuter).Execute(0xc03468b440, 0xc01b77c140)
        /home/labs/go/pkg/mod/github.com/projectdiscovery/nuclei/[email protected]/pkg/tmplexec/exec.go:212 +0x41c
github.com/projectdiscovery/nuclei/v3/pkg/core.(*Engine).executeTemplateWithTargets.func2.1(0x88fca5?, 0x0?, 0xc001326680)
        /home/labs/go/pkg/mod/github.com/projectdiscovery/nuclei/[email protected]/pkg/core/executors.go:139 +0x1e6
created by github.com/projectdiscovery/nuclei/v3/pkg/core.(*Engine).executeTemplateWithTargets.func2 in goroutine 20120
        /home/labs/go/pkg/mod/github.com/projectdiscovery/nuclei/[email protected]/pkg/core/executors.go:115 +0x54e
exit status 2

Expected Behavior

no error

Steps To Reproduce

package govulner

import (
	"context"
	"log"

	nuclei "github.com/projectdiscovery/nuclei/v3/lib"
)

func scanWithNuclei(ctx context.Context, targets []string) {
	ne, err := nuclei.NewNucleiEngineCtx(ctx,
		nuclei.WithTemplateFilters(nuclei.TemplateFilters{Severity: "low"}),
	)
	if err != nil {
		panic(err)
	}
	ne.LoadTargets(targets, false)
	err = ne.ExecuteWithCallback(nil)
	if err != nil {
		panic(err)
	}
	defer ne.Close()
}

func Test(ctx context.Context) {
	targets := []string{
		"hackerone.com",
		"tesla.com",
	}

	for _, target := range targets {
		log.Println("Nuclei Target: ", target)
		go func() {
			scanWithNuclei(ctx, []string{target})
		}()
	}
}

Relevant log output

No response

Environment

  • OS: Ubuntu 24.04.1 LTS
  • Nuclei: v3.3.5
  • Go: go version go1.23.2 linux/amd64

Anything else?

No response

@com0t com0t added the Type: Bug Inconsistencies or issues which will cause an issue or problem for users or implementors. label Nov 6, 2024
@com0t com0t changed the title [BUG] ... [BUG] nil pointer in GetDialedIP Nov 7, 2024
@dogancanbakir dogancanbakir self-assigned this Nov 8, 2024
@dogancanbakir
Copy link
Member

Couldn't repro this, made a little update to your code

$ go run .
2024/11/08 14:46:09 Nuclei Target:  http://scanme.sh
2024/11/08 14:46:09 Nuclei Target:  http://honey.scanme.sh
[INF] Templates clustered: 1636 (Reduced 1537 Requests)
[INF] Templates clustered: 1636 (Reduced 1537 Requests)
[CVE-2015-6477] http://honey.scanme.sh/login
...
package main

import (
	"context"
	"log"
	"sync"

	nuclei "github.com/projectdiscovery/nuclei/v3/lib"
)

func scanWithNuclei(ctx context.Context, target string) {
	ne, err := nuclei.NewNucleiEngineCtx(ctx)
	if err != nil {
		panic(err)
	}
	ne.LoadTargets([]string{target}, false)
	ne.Options().Verbose = true
	err = ne.ExecuteWithCallback(nil)
	if err != nil {
		panic(err)
	}
	defer ne.Close()
}

func main() {
	targets := []string{
		"http://scanme.sh",
		"http://honey.scanme.sh",
	}

	var wg sync.WaitGroup

	for _, target := range targets {
		log.Println("Nuclei Target: ", target)
		wg.Add(1)
		go func(t string) {
			scanWithNuclei(context.Background(), t)
		}(target)
	}

	wg.Wait()
}

@dogancanbakir dogancanbakir added the Investigation Something to Investigate label Nov 8, 2024
@dogancanbakir
Copy link
Member

I've replicated the issue and will investigate it further!

@com0t
Copy link
Author

com0t commented Nov 12, 2024

You can try the code below

package main

import (
	"context"
	"log"
	"os"
	"os/signal"

	nuclei "github.com/projectdiscovery/nuclei/v3/lib"
)

func scanWithNuclei(ctx context.Context, targets []string) {
	ne, err := nuclei.NewNucleiEngineCtx(ctx,
		nuclei.WithTemplateFilters(nuclei.TemplateFilters{Severity: "low"}),
	)
	if err != nil {
		panic(err)
	}
	ne.LoadTargets(targets, false)
	err = ne.ExecuteWithCallback(nil)
	if err != nil {
		panic(err)
	}
	defer ne.Close()
}

func Test(ctx context.Context) {
	targets := []string{
		"hackerone.com",
		"tesla.com",
	}

	for _, target := range targets {
		log.Println("Nuclei Target: ", target)
		go func() {
			scanWithNuclei(ctx, []string{target})
		}()
	}
}

func main() {
	ctx, cancel := context.WithCancel(context.Background())
	defer cancel()

	sigChan := make(chan os.Signal, 1)
	signal.Notify(sigChan, os.Interrupt)

	Test(ctx)

	<-sigChan
	log.Println("Shutting down gracefully...")
	cancel()
	log.Println("Consumer closed.")
}

@com0t
Copy link
Author

com0t commented Nov 12, 2024

I used your code with input and got an error. I think because the input does not contain http or https, I get an error when checking the low template. Try going this direction to see if you discover anything new

Input

targets := []string{
		"hackerone.com",
		"tesla.com",
}

Err

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xca7638]

goroutine 327145 [running]:
github.com/projectdiscovery/fastdialer/fastdialer.(*Dialer).DialTLS(0x0, {0x337f340, 0xc00bd59ab0}, {0x28f3676, 0x3}, {0xc01e992b10, 0xd})
        /root/go/pkg/mod/github.com/projectdiscovery/[email protected]/fastdialer/dialer.go:212 +0x38
github.com/projectdiscovery/nuclei/v3/pkg/protocols/http/httpclientpool.wrappedGet.func1({0x337f340, 0xc00bd59ab0}, {0x28f3676, 0x3}, {0xc01e992b10, 0xd})
        /root/go/pkg/mod/github.com/projectdiscovery/nuclei/[email protected]/pkg/protocols/http/httpclientpool/clientpool.go:261 +0xee
net/http.(*Transport).customDialTLS(0x41b274?, {0x337f340?, 0xc00bd59ab0?}, {0x28f3676?, 0xc00987c610?}, {0xc01e992b10?, 0x41a9d7?})
        /usr/local/go/src/net/http/transport.go:1348 +0x4a
net/http.(*Transport).dialConn(0xc004dde500, {0x337f340, 0xc00bd59ab0}, {{}, 0x0, {0x28f977b, 0x5}, {0xc01e992b10, 0xd}, 0x0})
        /usr/local/go/src/net/http/transport.go:1623 +0x3df
net/http.(*Transport).dialConnFor(0xc004dde500, 0xc00d511a20)
        /usr/local/go/src/net/http/transport.go:1484 +0xca
created by net/http.(*Transport).queueForDial in goroutine 327140
        /usr/local/go/src/net/http/transport.go:1449 +0x3c9
exit status 2

@com0t
Copy link
Author

com0t commented Nov 14, 2024

Couldn't repro this, made a little update to your code

$ go run .
2024/11/08 14:46:09 Nuclei Target: http://scanme.sh
2024/11/08 14:46:09 Nuclei Target: http://honey.scanme.sh
[INF] Templates clustered: 1636 (Reduced 1537 Requests)
[INF] Templates clustered: 1636 (Reduced 1537 Requests)
[CVE-2015-6477] http://honey.scanme.sh/login
...
package main

import (
"context"
"log"
"sync"

nuclei "github.com/projectdiscovery/nuclei/v3/lib"
)

func scanWithNuclei(ctx context.Context, target string) {
ne, err := nuclei.NewNucleiEngineCtx(ctx)
if err != nil {
panic(err)
}
ne.LoadTargets([]string{target}, false)
ne.Options().Verbose = true
err = ne.ExecuteWithCallback(nil)
if err != nil {
panic(err)
}
defer ne.Close()
}

func main() {
targets := []string{
"http://scanme.sh",
"http://honey.scanme.sh",
}

var wg sync.WaitGroup

for _, target := range targets {
log.Println("Nuclei Target: ", target)
wg.Add(1)
go func(t string) {
scanWithNuclei(context.Background(), t)
}(target)
}

wg.Wait()
}

Hi, I ran your code and still got nil pointer error. Please check again and wait for the code to run all templates

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xce3241]

goroutine 302901 [running]:
github.com/projectdiscovery/fastdialer/fastdialer.(*Dialer).dial(0x0, {0x32b5130?, 0xc04f5bcaf0?}, 0xc02dc58500)
        /home/labs/go/pkg/mod/github.com/projectdiscovery/[email protected]/fastdialer/dialer_private.go:66 +0x61
github.com/projectdiscovery/fastdialer/fastdialer.(*Dialer).Dial(0x0, {0x32b5130, 0xc04f5bcaf0}, {0x291909c, 0x3}, {0xc0405d6150, 0x12})
        /home/labs/go/pkg/mod/github.com/projectdiscovery/[email protected]/fastdialer/dialer.go:198 +0xb7
net/http.(*Transport).dial(0x160?, {0x32b5130?, 0xc04f5bcaf0?}, {0x291909c?, 0x726926?}, {0xc0405d6150?, 0xc01cdb3308?})
        /snap/go/10743/src/net/http/transport.go:1226 +0xd2
net/http.(*Transport).dialConn(0xc033188780, {0x32b5130, 0xc04f5bcaf0}, {{}, 0x0, {0xc0457160f0, 0x4}, {0xc0405d6150, 0x12}, 0x0})
        /snap/go/10743/src/net/http/transport.go:1728 +0x7e5
net/http.(*Transport).dialConnFor(0xc033188780, 0xc03238f4a0)
        /snap/go/10743/src/net/http/transport.go:1563 +0xb8
net/http.(*Transport).startDialConnForLocked.func1()
        /snap/go/10743/src/net/http/transport.go:1545 +0x35
created by net/http.(*Transport).startDialConnForLocked in goroutine 197030
        /snap/go/10743/src/net/http/transport.go:1544 +0x117
exit status 2

@Autumn-27
Copy link

#5747

I encountered the same problem. Through brief debugging, I found that if it is called multiple times, it seems that some global variables are closed in advance. My previous solution was to create a global manager and then close it after all target calls are completed, but it may cause memory to increase. I don't know the specific reason.

@Autumn-27
Copy link

In addition, in this scenario, multiple calls to NewNucleiEngineCtx will load all templates multiple times, resulting in serious memory consumption. I observed that the Parser in the code stores the parsed templates in the cache when loading the templates. Is it possible to provide a method in the SDK to modify the Parser so that all nuclei instances can share a parsing module?Image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Investigation Something to Investigate Type: Bug Inconsistencies or issues which will cause an issue or problem for users or implementors.
Projects
None yet
Development

No branches or pull requests

3 participants