Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reporter package #116

Merged
merged 18 commits into from
Nov 27, 2023
Merged

Reporter package #116

merged 18 commits into from
Nov 27, 2023

Conversation

amircybersec
Copy link
Contributor

@amircybersec amircybersec commented Oct 24, 2023

@fortuna I refactored the send report functionality with random sampling into a separate reporter package to be imported when needed in other apps. Since I am using this logic in both outline-connectivity and outline-connectivity-app, it would be repetitive to have the code block in both places. I recommend dropping the other PR and get this PR merged first:

#112

I can then do another PR for refactored main.go for for both connectivity apps that uses this package for reporting.

x/reporter/reporter.go Outdated Show resolved Hide resolved
x/reporter/reporter.go Outdated Show resolved Hide resolved
@amircybersec amircybersec requested a review from fortuna October 26, 2023 17:26
x/reporter/reporter.go Outdated Show resolved Hide resolved
x/reporter/reporter.go Outdated Show resolved Hide resolved
x/reporter/reporter.go Outdated Show resolved Hide resolved
x/reporter/reporter.go Outdated Show resolved Hide resolved
return true
}
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: WriteCollector that takes a io.Writer

x/reporter/reporter.go Outdated Show resolved Hide resolved
x/reporter/reporter.go Outdated Show resolved Hide resolved
x/reporter/reporter.go Outdated Show resolved Hide resolved
x/reporter/reporter.go Outdated Show resolved Hide resolved
@amircybersec amircybersec requested a review from fortuna November 21, 2023 19:31
Comment on lines 78 to 79
log.Printf("Error encoding JSON: %s\n", err)
return err
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
log.Printf("Error encoding JSON: %s\n", err)
return err
return fmt.Errorf("failed to marshal JSON: %w", err)

"time"
)

var debugLog log.Logger = *log.New(io.Discard, "", 0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should remove all logging from this library.

Suggested change
var debugLog log.Logger = *log.New(io.Discard, "", 0)

if random < samplingRate {
err := c.collector.Collect(ctx, report)
if err != nil {
log.Printf("Error collecting report: %v", err)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
log.Printf("Error collecting report: %v", err)

)

var debugLog log.Logger = *log.New(io.Discard, "", 0)
var httpClient = &http.Client{}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pass the client to the RemoteReporter

Suggested change
var httpClient = &http.Client{}

}

type RemoteCollector struct {
collectorEndpoint *url.URL
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
collectorEndpoint *url.URL
httpClient *http.Client
collectorEndpoint *url.URL

}
}

type RetryCollector struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All public types and function must have comments.

}
}

type RetryCollector struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
type RetryCollector struct {
// RetryCollector is a [Collector] that retries in case of [BadRequestErr].
type RetryCollector struct {

Comment on lines 34 to 41
type BadRequestErr struct {
StatusCode int
Message string
}

func (e BadRequestErr) Error() string {
return e.Message
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
type BadRequestErr struct {
StatusCode int
Message string
}
func (e BadRequestErr) Error() string {
return e.Message
}
type BadRequestError {
error
}
func (e BadRequestError) Unwrap() error {
return e.error
}

Then you can do:

  return BadRequestError{err}

debugLog.Printf("Error reading the HTTP response body: %s\n", err)
return err
}
if resp.StatusCode >= 400 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if resp.StatusCode >= 400 {
if 400 <= resp.StatusCode && resp.StatusCode < 500 {

Comment on lines 209 to 212
return BadRequestErr{
StatusCode: resp.StatusCode,
Message: fmt.Sprintf("HTTP request failed with status code %d", resp.StatusCode),
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return BadRequestErr{
StatusCode: resp.StatusCode,
Message: fmt.Sprintf("HTTP request failed with status code %d", resp.StatusCode),
}
return BadRequestErr{err}

var debugLog log.Logger = *log.New(io.Discard, "", 0)
var httpClient = &http.Client{}

type BadRequestErr struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems the standard library uses the Err prefix for error constants and the Error suffix for type, so we should use BadRequestError here.

@amircybersec amircybersec requested a review from fortuna November 27, 2023 04:52
@fortuna fortuna marked this pull request as ready for review November 27, 2023 17:07
Comment on lines 140 to 164
// MultiCollector represents a collector that combines multiple collectors.
type MultiCollector struct {
collectors []Collector
}

// Collect implements [Collector] interface on [MultiCollector] type.
// It collects the report using all the provided collectors in the [MultiCollector].
// It returns an error if all collectors fail to collect the report.
func (c *MultiCollector) Collect(ctx context.Context, report Report) error {
success := false
for i := range c.collectors {
err := c.collectors[i].Collect(ctx, report)
if err != nil {
success = success || false
} else {
success = success || true
}
}
if success {
// At least one collector succeeded
return nil
}
return errors.New("all collectors failed")
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's drop this for now, since the semantics can be confusing and it's not clear we will use it.

Suggested change
// MultiCollector represents a collector that combines multiple collectors.
type MultiCollector struct {
collectors []Collector
}
// Collect implements [Collector] interface on [MultiCollector] type.
// It collects the report using all the provided collectors in the [MultiCollector].
// It returns an error if all collectors fail to collect the report.
func (c *MultiCollector) Collect(ctx context.Context, report Report) error {
success := false
for i := range c.collectors {
err := c.collectors[i].Collect(ctx, report)
if err != nil {
success = success || false
} else {
success = success || true
}
}
if success {
// At least one collector succeeded
return nil
}
return errors.New("all collectors failed")
}

if errors.As(err, &e) {
break
} else {
time.Sleep(c.waitBetweenRetry)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fortuna fortuna merged commit 4182a61 into Jigsaw-Code:main Nov 27, 2023
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants