Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make hasher work with symlinks #83

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
* `hasher` will no longer create a directory if a non-existing one is passed as an argument.
* trivrost will no longer attempt to repeat range requests to a host after it has failed to conformly respond while displaying the confusing message `Taking longer than usual: HTTP Status 200` and will now fail immediately in such cases instead.
* trivrost will no longer fail to comply with HTTP 2 strictly using lower-case HTTP Header names. This had been caused by methods of `http.Header` still being oriented around HTTP 1 canonical header names due to Go's backwards compatibility promise.
* hasher now supports symlinks within hashed directories.

## 1.4.6 (2021-01-25)
### Fixes
Expand Down
36 changes: 27 additions & 9 deletions pkg/launcher/hashing/scan.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import (
"io/ioutil"
"os"
"path/filepath"
"strings"

"github.com/setlog/trivrost/pkg/launcher/config"
log "github.com/sirupsen/logrus"
Expand Down Expand Up @@ -61,24 +62,41 @@ func mustHashRelatively(ctx context.Context, readDir readDirFunc, readFile readF
}

func mustHashDir(ctx context.Context, readDir readDirFunc, readFile readFileFunc, stat statFunc, hashFilePath string) config.FileInfoMap {
fm := make(config.FileInfoMap)
for _, info := range mustReadDir(readDir, hashFilePath) {
if info.IsDir() {
fm.Join(mustHashDir(ctx, readDir, readFile, stat, filepath.Join(hashFilePath, info.Name())))
fileMap := make(config.FileInfoMap)
for _, curPathInfo := range mustReadDir(readDir, hashFilePath) {
curPath := filepath.Join(hashFilePath, curPathInfo.Name())
resolvedPath := evaluateSoftLink(curPath)
if curPath != resolvedPath {
log.Warnf("File \"%s\"->\"%s\" is a symlink, will be treated as a regular file/dir.", curPath, resolvedPath)
curPath = resolvedPath
}
if !strings.HasPrefix(curPath, hashFilePath) {
panic(fmt.Errorf("hashing '%s' outside hash directory is not allowed", curPath))
}
curPathInfo, _ = stat(curPath)
if curPathInfo.IsDir() {
fileMap.Join(mustHashDir(ctx, readDir, readFile, stat, curPath))
} else {
filePath := filepath.Join(hashFilePath, info.Name())
sha, size, err := calculateSha256(ctx, filePath, readFile)
sha, size, err := calculateSha256(ctx, curPath, readFile)
if err != nil {
panic(fmt.Errorf("failed hashing file \"%s\": %w", hashFilePath, err))
}
fm[filePath] = &config.FileInfo{SHA256: sha, Size: size}
fileMap[curPath] = &config.FileInfo{SHA256: sha, Size: size}
}
}
return fm
return fileMap
}

func evaluateSoftLink(filePath string) string {
evaluatedName, err := filepath.EvalSymlinks(filePath)
if err != nil {
panic(err)
}
return evaluatedName
}

func mustReadDir(readDir readDirFunc, directoryPath string) []os.FileInfo {
infos, err := readDir(directoryPath)
infos, err := readDir(evaluateSoftLink(directoryPath))
if err != nil {
if os.IsNotExist(err) {
return nil
Expand Down