enhance trace output, addressing #45

README.md

@@ -1,5 +1,6 @@
 
 # xgo
+
 [![Go Reference](https://pkg.go.dev/badge/github.com/xhd2015/xgo.svg)](https://pkg.go.dev/github.com/xhd2015/xgo)
 [![Go Report Card](https://goreportcard.com/badge/github.com/xhd2015/xgo)](https://goreportcard.com/report/github.com/xhd2015/xgo)
 [![Go Coverage](https://img.shields.io/badge/Coverage-82.6%25-brightgreen)](https://github.com/xhd2015/xgo/actions)
@@ -10,7 +11,7 @@
 
 Enable function Trap in `go`, provide tools like Mock and Trace to help go developers write unit test and debug both easier and faster.
 
-`xgo` works as a preprocessor for `go run`,`go build`, and `go test`(see our [blog](https://blog.xhd2015.xyz/posts/xgo-monkey-patching-in-go-using-toolexec/)).
+`xgo` works as a preprocessor for `go run`,`go build`, and `go test`(see our [blog](https://blog.xhd2015.xyz/posts/xgo-monkey-patching-in-go-using-toolexec)).
 
 It **preprocess** the source code and IR(Intermediate Representation) before invoking `go`, adding missing abilities to go program by cooperating with(or hacking) the go compiler.
 
@@ -21,6 +22,8 @@ These abilities include:
 
 See [Quick Start](#quick-start) and [Documentation](./doc) for more details.
 
+> *By the way, I promise you this is an interesting project.*
+
 # Installation
 ```sh
 go install github.com/xhd2015/xgo/cmd/xgo@latest
@@ -155,7 +158,6 @@ import (
 func init() {
     trap.AddInterceptor(&trap.Interceptor{
         Pre: func(ctx context.Context, f *core.FuncInfo, args core.Object, results core.Object) (interface{}, error) {
-            trap.Skip()
             if f.Name == "A" {
                 fmt.Printf("trap A\n")
                 return nil, nil
@@ -421,11 +423,22 @@ By default, Trace will write traces to a temp directory under current working di
 - `XGO_TRACE_OUTPUT=<dir>`: traces will be written to `<dir>`,
 - `XGO_TRACE_OUTPUT=off`: turn off trace.
 
+# Concurrent safety
+I know you guys from other monkey patching library suffer from the unsafety implied by these frameworks.
+
+But I guarantee you mocking in xgo is builtin concurrent safe. That means, you can run multiple tests concurrently as long as you like.
+
+Why? when you run a test, you setup some mock, these mocks will only affect the goroutine test running the test. And these mocks get cleared when the goroutine ends, no matter the test passed or failed.
+
+Want to know why? Stay tuned, we are working on internal documentation.
+
 # Implementation Details
 > Working in progress...
 
 See [Issue#7](https://github.com/xhd2015/xgo/issues/7) for more details.
 
+This blog has a basic explanation: https://blog.xhd2015.xyz/posts/xgo-monkey-patching-in-go-using-toolexec
+
 # Why `xgo`?
 The reason is simple: **NO** interface.
 
@@ -435,7 +448,7 @@ Extracting interface just for mocking is never an option to me. To the domain of
 
 Monkey patching simply does the right thing for the problem. But existing library are bad at compatibility.
 
-So I created `xgo`, so I hope `xgo` will also take over other solutions to the mocking problem.
+So I created `xgo`, and hope it will finally take over other solutions to the mocking problem.
 
 # Comparing `xgo` with `monkey`
 The project [bouk/monkey](https://github.com/bouk/monkey), was initially created by bouk, as described in his blog https://bou.ke/blog/monkey-patching-in-go.

README_zh_cn.md

@@ -9,7 +9,7 @@
 
 允许对`go`的函数进行拦截, 并提供Mock和Trace等工具帮助开发者编写测试和快速调试。
 
-`xgo`作为一个预处理器工作在`go run`,`go build`,和`go test`之上(查看[blog](https://blog.xhd2015.xyz/zh/posts/xgo-monkey-patching-in-go-using-toolexec/))。
+`xgo`作为一个预处理器工作在`go run`,`go build`,和`go test`之上(查看[blog](https://blog.xhd2015.xyz/zh/posts/xgo-monkey-patching-in-go-using-toolexec))。
 
 `xgo`对源代码和IR(中间码)进行预处理之后, 再调用`go`进行后续的编译工作。通过这种方式, `xgo`实现了一些在`go`中缺乏的能力。
 
@@ -20,6 +20,8 @@
 
 更多细节, 参见[快速开始](#快速开始)和[文档](./doc)。
 
+> *顺便说一下, 我可以向你保证这是一个有趣的项目。*
+
 # 安装
 ```sh
 go install github.com/xhd2015/xgo/cmd/xgo@latest
@@ -150,7 +152,6 @@ import (
 func init() {
     trap.AddInterceptor(&trap.Interceptor{
         Pre: func(ctx context.Context, f *core.FuncInfo, args core.Object, results core.Object) (interface{}, error) {
-            trap.Skip()
             if f.Name == "A" {
                 fmt.Printf("trap A\n")
                 return nil, nil
@@ -414,12 +415,23 @@ XGO_TRACE_OUTPUT=stdout xgo run ./
 - `XGO_TRACE_OUTPUT=<dir>`: 堆栈记录被写入到`<dir>`目录下,
 - `XGO_TRACE_OUTPUT=off`: 关闭堆栈记录收集。
 
+# 并发安全
+我知道大部分人认为Monkey Patching不是并发安全的，但那是现有的库的实现方式决定的。
+
+我可以向你保证，在xgo中进行Monkey Patching是并发安全的，也就意味着，你可以同时并行跑所有的测试用例。
+
+为什么? 因为当你设置mock时，只有当前的goroutine受影响，并且在goroutine退出后清除，不管当前测试失败还是成功。
+
+想知道真正的原因吗? 我们正在整理内部实现的文档，尽请期待。
+
 # 实现原理
 
 > 仍在整理中...
 
 参见[Issue#7](https://github.com/xhd2015/xgo/issues/7)
 
+这个博客作了一些简单的解释: https://blog.xhd2015.xyz/zh/posts/xgo-monkey-patching-in-go-using-toolexec
+
 # 为何使用`xgo`?
 原因很简单: **避免**interface.
 

cmd/xgo/main.go

@@ -287,7 +287,7 @@ func handleBuild(cmd string, args []string) error {
 			return err
 		}
 
-		if resetInstrument || revisionChanged {
+		if resetInstrument {
 			logDebug("revision changed, reset %s", instrumentDir)
 			err := os.RemoveAll(instrumentDir)
 			if err != nil {

cmd/xgo/patch.go

@@ -167,7 +167,7 @@ func syncGoroot(goroot string, instrumentGoroot string, fullSyncRecordFile strin
 		// need copy, delete target dst dir first
 		// TODO: use git worktree add if .git exists
 		err = filecopy.NewOptions().
-			Concurrent(10).
+			Concurrent(2). // 10 is too much
 			CopyReplaceDir(goroot, instrumentGoroot)
 		if err != nil {
 			return err

cmd/xgo/patch_runtime.go

@@ -15,15 +15,44 @@ import (
 var xgoAutoGenRegisterFuncHelper = _FilePath{"src", "runtime", "__xgo_autogen_register_func_helper.go"}
 var xgoTrap = _FilePath{"src", "runtime", "xgo_trap.go"}
 var runtimeProc = _FilePath{"src", "runtime", "proc.go"}
-var testingFile = _FilePath{"src", "testing", "testing.go"}
 var runtimeTime _FilePath = _FilePath{"src", "runtime", "time.go"}
 var timeSleep _FilePath = _FilePath{"src", "time", "sleep.go"}
 
+var testingFilePatch = &FilePatch{
+	FilePath: _FilePath{"src", "testing", "testing.go"},
+	Patches: []*Patch{
+		{
+			Mark:         "declare_testing_callback_v2",
+			InsertIndex:  0,
+			InsertBefore: true,
+			Anchors: []string{
+				"func tRunner(t *T, fn func",
+				"{",
+				"\n",
+			},
+			Content: patch.TestingCallbackDeclarations,
+		},
+		{
+			Mark:         "call_testing_callback_v2",
+			InsertIndex:  4,
+			InsertBefore: true,
+			Anchors: []string{
+				"func tRunner(t *T, fn func",
+				"{",
+				"\n",
+				`t.start = time.Now()`,
+				"fn(t",
+			},
+			Content: patch.TestingStart,
+		},
+	},
+}
+
 var runtimeFiles = []_FilePath{
 	xgoAutoGenRegisterFuncHelper,
 	xgoTrap,
 	runtimeProc,
-	testingFile,
+	testingFilePatch.FilePath,
 	runtimeTime,
 	timeSleep,
 }
@@ -136,22 +165,7 @@ func patchRuntimeProc(goroot string) error {
 }
 
 func patchRuntimeTesting(goroot string) error {
-	testingFile := filepath.Join(goroot, filepath.Join(testingFile...))
-	return editFile(testingFile, func(content string) (string, error) {
-		// func tRunner(t *T, fn func(t *T)) {
-		anchor := []string{"func tRunner(t *T", "{", "\n"}
-		content = addContentBefore(content,
-			"/*<begin declare_testing_callback>*/", "/*<end declare_testing_callback>*/",
-			anchor,
-			patch.TestingCallbackDeclarations,
-		)
-		content = addContentAfter(content,
-			"/*<begin call_testing_callback>*/", "/*<end call_testing_callback>*/",
-			anchor,
-			patch.TestingStart,
-		)
-		return content, nil
-	})
+	return testingFilePatch.Apply(goroot, nil)
 }
 
 // only required if need to mock time.Sleep

cmd/xgo/version.go

@@ -2,9 +2,9 @@ package main
 
 import "fmt"
 
-const VERSION = "1.0.19"
-const REVISION = "75c04e25cd9ccc811d6893fc0c0c02df889cad66+1"
-const NUMBER = 171
+const VERSION = "1.0.20"
+const REVISION = "7ab84d83e8d847f0c2307a44866705581ba5cbbe+1"
+const NUMBER = 172
 
 func getRevision() string {
 	revSuffix := ""

patch/ctxt/ctx.go

@@ -13,6 +13,8 @@ const XgoRuntimeCorePkg = XgoModule + "/runtime/core"
 var XgoMainModule = os.Getenv("XGO_MAIN_MODULE")
 var XgoCompilePkgDataDir = os.Getenv("XGO_COMPILE_PKG_DATA_DIR")
 
+const XgoLinkTrapVarForGenerated = "__xgo_link_trap_var_for_generated"
+
 func SkipPackageTrap() bool {
 	pkgPath := GetPkgPath()
 	if pkgPath == "" {
@@ -56,68 +58,9 @@ func SkipPackageTrap() bool {
 	return false
 }
 
-var stdWhitelist = map[string]map[string]bool{
-	// "runtime": map[string]bool{
-	// "timeSleep": true,
-	// },
-	"os": map[string]bool{
-		// starts with Get
-		"OpenFile":  true,
-		"ReadFile":  true,
-		"WriteFile": true,
-	},
-	"io": map[string]bool{
-		"ReadAll": true,
-	},
-	"io/ioutil": map[string]bool{
-		"ReadAll":  true,
-		"ReadFile": true,
-		"ReadDir":  true,
-	},
-	"time": map[string]bool{
-		"Now": true,
-		// time.Sleep is special:
-		//  if trapped like normal functions
-		//    runtime/time.go:178:6: ns escapes to heap, not allowed in runtime
-		// there are special handling of this, see cmd/xgo/patch_runtime patchRuntimeTime
-		"Sleep":       true, // NOTE: time.Sleep links to runtime.timeSleep
-		"NewTicker":   true,
-		"Time.Format": true,
-	},
-	"os/exec": map[string]bool{
-		"Command":       true,
-		"(*Cmd).Run":    true,
-		"(*Cmd).Output": true,
-		"(*Cmd).Start":  true,
-	},
-	"net/http": map[string]bool{
-		"Get":  true,
-		"Head": true,
-		"Post": true,
-		// Sever
-		"Serve":           true,
-		"Handle":          true,
-		"(*Client).Do":    true,
-		"(*Server).Close": true,
-	},
-	"net": map[string]bool{
-		// starts with Dial
-	},
-}
-
 func AllowPkgFuncTrap(pkgPath string, isStd bool, funcName string) bool {
 	if isStd {
-		if stdWhitelist[pkgPath][funcName] {
-			return true
-		}
-		switch pkgPath {
-		case "os":
-			return strings.HasPrefix(funcName, "Get")
-		case "net":
-			return strings.HasPrefix(funcName, "Dial")
-		}
-		// by default block all
-		return false
+		return allowStdFunc(pkgPath, funcName)
 	}
 
 	return true

patch/ctxt/stdlib.go

@@ -0,0 +1,69 @@
+package ctxt
+
+import "strings"
+
+var stdWhitelist = map[string]map[string]bool{
+	// "runtime": map[string]bool{
+	// "timeSleep": true,
+	// },
+	"os": map[string]bool{
+		// starts with Get
+		"OpenFile":  true,
+		"ReadFile":  true,
+		"WriteFile": true,
+	},
+	"io": map[string]bool{
+		"ReadAll": true,
+	},
+	"io/ioutil": map[string]bool{
+		"ReadAll":  true,
+		"ReadFile": true,
+		"ReadDir":  true,
+	},
+	"time": map[string]bool{
+		"Now": true,
+		// time.Sleep is special:
+		//  if trapped like normal functions
+		//    runtime/time.go:178:6: ns escapes to heap, not allowed in runtime
+		// there are special handling of this, see cmd/xgo/patch_runtime patchRuntimeTime
+		"Sleep":       true, // NOTE: time.Sleep links to runtime.timeSleep
+		"NewTicker":   true,
+		"Time.Format": true,
+	},
+	"os/exec": map[string]bool{
+		"Command":       true,
+		"(*Cmd).Run":    true,
+		"(*Cmd).Output": true,
+		"(*Cmd).Start":  true,
+	},
+	"net/http": map[string]bool{
+		"Get":  true,
+		"Head": true,
+		"Post": true,
+		// Sever
+		"Serve":           true,
+		"Handle":          true,
+		"(*Client).Do":    true,
+		"(*Server).Close": true,
+	},
+	"net": map[string]bool{
+		// starts with Dial
+	},
+	"encoding/json": map[string]bool{
+		"newTypeEncoder": true,
+	},
+}
+
+func allowStdFunc(pkgPath string, funcName string) bool {
+	if stdWhitelist[pkgPath][funcName] {
+		return true
+	}
+	switch pkgPath {
+	case "os":
+		return strings.HasPrefix(funcName, "Get")
+	case "net":
+		return strings.HasPrefix(funcName, "Dial")
+	}
+	// by default block all
+	return false
+}

patch/link_name.go

@@ -58,7 +58,7 @@ func isLinkValid(fnName string, targetName string, pkgPath string) bool {
 	if disableXgoLink {
 		return false
 	}
-	safeGenerated := (fnName == xgo_syntax.XgoLinkGeneratedRegisterFunc || fnName == xgo_syntax.XgoLinkTrapForGenerated)
+	safeGenerated := (fnName == xgo_syntax.XgoLinkGeneratedRegisterFunc || fnName == xgo_syntax.XgoLinkTrapForGenerated || fnName == xgo_ctxt.XgoLinkTrapVarForGenerated)
 	if safeGenerated {
 		// generated by xgo on the fly for every instrumented package
 		return true