Add intercepter and writer #2
Conversation
nicolasgere
force-pushed
the
nicolas/test-intercept
branch
from
9328e2d to
2733e74
Compare
| ) | ||
|
|
||
| func DataToRawLine(info *RecorderInfo, path string) (b []byte, err error) { | ||
| d := map[string]string{ |
fs/layer/recorder.go
Outdated
| r.LayerSha = layerDigest | ||
| r.ImageTag, ok = labels["containerd.io/snapshot/remote/stargz.reference"] | ||
| if !ok { | ||
| return r, fmt.Errorf("containerd.io/snapshot/remote/stargz.reference not found") |
There was a problem hiding this comment.
nit - maybe these can be ErrReferenceNotFound
fs/layer/recorder.go
Outdated
| // Create a singleton with a sync.Once to avoid concurrent creation of the same object. | ||
| sync.OnceFunc(func() { | ||
| var err error | ||
| b10Writer, err = NewBufferedWriter("/var/b10/log", 1024*1024, 5*time.Second, 10000) |
There was a problem hiding this comment.
parameterize these? Do we need to worry about log file rotation?
There was a problem hiding this comment.
I need to check more in detail how does fluentd does log rotation
| ) | ||
|
|
||
| // Buffered writer that writes to a file in a buffered manner, it need to be a singleton | ||
| type BufferedWriter struct { |
There was a problem hiding this comment.
Is this functionality handled by a logger? Would using a structured logger like zero log make this code simpler? Seems like flushing, singleton, mutual exclusion semantics are handled by loggers.
There was a problem hiding this comment.
I can check which logger they use, and if it provide the functionality we need
| @@ -214,6 +214,9 @@ func (r *reader) Clone(sr *io.SectionReader) (metadata.Reader, error) { | |||
| decompressor: r.decompressor, | |||
| }, nil | |||
| } | |||
| func (r *reader) GetName(id uint32) string { | |||
fs/layer/node.go
Outdated
| @@ -352,6 +356,7 @@ func (n *node) Open(ctx context.Context, flags uint32) (fh fusefs.FileHandle, fu | |||
| ra: ra, | |||
| fd: -1, | |||
| } | |||
| n.recorder.Append(n.fs.r.Metadata().GetName(n.id)) | |||
There was a problem hiding this comment.
Can you add a comment on why do it here as opposed to in OpenFile?
Add an interceptor at the fuse level, it would push a message to a buffered chan, to avoid as much as possible having any impact. The writer which listen to the chan, will write every 1mb OR 5 seconds