Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: reconquest/orgalorg
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 1.0.1
Choose a base ref
...
head repository: reconquest/orgalorg
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: master
Choose a head ref
  • 13 commits
  • 22 files changed
  • 5 contributors

Commits on Jun 24, 2021

  1. Copy the full SHA
    4a19883 View commit details
  2. Copy the full SHA
    376244e View commit details
  3. update dependencies

    kovetskiy committed Jun 24, 2021
    Copy the full SHA
    c51061e View commit details

Commits on Aug 20, 2021

  1. Copy the full SHA
    3d3ae44 View commit details

Commits on Aug 23, 2021

  1. Merge pull request #33 from AlekSi/upgrade-sys-go117

    Update x/sys to support go 1.17
    kovetskiy authored Aug 23, 2021
    Copy the full SHA
    8258475 View commit details

Commits on Dec 20, 2021

  1. Add Windows support, including openssh and pageant agents; closes #19 (

    …#35)
    
    * Add Windows support, including openssh and pageant agents; closes #19
    
    * Wrapped README.md; removed useless var
    
    * Removed --no-preserve-uid --no-preserve-gid for windows
    
    * Fix var name in the late hour :)
    
    Co-authored-by: Cristian Chiru <cristian.chiru@revomatico.com>
    cr1cr1 and Cristian Chiru authored Dec 20, 2021
    Copy the full SHA
    6608ee9 View commit details

Commits on Dec 24, 2021

  1. Copy the full SHA
    5024122 View commit details

Commits on Jul 11, 2024

  1. update modules

    Signed-off-by: Egor Kovetskiy <e.kovetskiy@gmail.com>
    kovetskiy committed Jul 11, 2024
    Copy the full SHA
    3921f3f View commit details
  2. update readme

    Signed-off-by: Egor Kovetskiy <e.kovetskiy@gmail.com>
    kovetskiy committed Jul 11, 2024
    Copy the full SHA
    6c73516 View commit details

Commits on Aug 16, 2024

  1. fix: update workflow files to go mod updates (#43)

    Signed-off-by: Rui Chen <rui@chenrui.dev>
    chenrui333 authored Aug 16, 2024
    Copy the full SHA
    898c8e9 View commit details

Commits on Aug 19, 2024

  1. update modules

    Signed-off-by: Egor Kovetskiy <e.kovetskiy@gmail.com>
    kovetskiy committed Aug 19, 2024
    Copy the full SHA
    8ec2ab6 View commit details

Commits on Aug 21, 2024

  1. tidy up go modules

    Signed-off-by: Egor Kovetskiy <e.kovetskiy@gmail.com>
    kovetskiy committed Aug 21, 2024
    Copy the full SHA
    806aaa2 View commit details
  2. remove rm-dist flag

    Signed-off-by: Egor Kovetskiy <e.kovetskiy@gmail.com>
    kovetskiy committed Aug 21, 2024
    Copy the full SHA
    17aad35 View commit details
21 changes: 6 additions & 15 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -6,28 +6,19 @@ jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Set up Go
uses: actions/setup-go@v2
uses: actions/setup-go@v5
with:
go-version: 1.15
go-version-file: "go.mod"
id: go

- name: Check out code into the Go module directory
uses: actions/checkout@v2

- name: Get dependencies
run: |
go get -v -t -d ./...
if [ -f Gopkg.toml ]; then
curl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh
dep ensure
fi
- name: Build
run: go build -mod=readonly -v .
run: go build -v .

- name: Test
run: go test -mod=readonly -v .
run: go test -v .

- name: Upload build artifact
uses: actions/upload-artifact@v2
15 changes: 5 additions & 10 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -9,21 +9,16 @@ jobs:
goreleaser:
runs-on: ubuntu-latest
steps:
-
name: Checkout
uses: actions/checkout@v2
with:
fetch-depth: 0
-
name: Set up Go
uses: actions/setup-go@v2
- uses: actions/checkout@v4

- uses: actions/setup-go@v5
with:
go-version: 1.15
go-version-file: "go.mod"
-
name: Run GoReleaser
uses: goreleaser/goreleaser-action@v2
with:
version: latest
args: release --rm-dist
args: release
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
/lorg
/.last-testcase
/orgalorg
/orgalorg.exe
/.cover
/coverage
6 changes: 3 additions & 3 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[submodule "vendor/github.com/reconquest/import.bash"]
path = vendor/github.com/reconquest/import.bash
url = https://github.com/reconquest/import.bash
[submodule "vendor.bash/github.com/reconquest/import.bash"]
path = vendor.bash/github.com/reconquest/import.bash
url = https://github.com/reconquest/import.bash
1 change: 1 addition & 0 deletions .goreleaser.yml
Original file line number Diff line number Diff line change
@@ -6,6 +6,7 @@ builds:
goos:
- linux
- darwin
- windows
goarch:
- amd64
flags:
24 changes: 11 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
@@ -30,6 +30,8 @@
fail on any error or try to complete, no matter of what.

* Interactive password authentication as well as SSH public key authentication.
Will use ssh-agent if present. On Windows, orgalorg can connect to
**pageant** or **openssh agent**.

* Ability to run commands through `sudo`.

@@ -39,26 +41,26 @@

# Installation

## go get
## go install

```bash
go get github.com/reconquest/orgalorg
go install github.com/reconquest/orgalorg@latest
```

# Alternatives

* ansible: intended to apply complex DSL-based scenarios of actions;
* ansible: intended to apply complex DSL-based scenarios of actions;
orgalorg aimed only on running commands and synchronizing files in parallel.
orgalorg can accept target hosts list on stdin and can provide realtime
output from commands, which ansible can't do (like running `tail -f`).
orgalorg also uses same argument semantic as `ssh`:
orgalorg also uses same argument semantic as `ssh`:
`orgalorg ... -C tail -f '/var/log/*.log'` will do exactly the same.

* clusterssh / cssh: will open number of xterm terminals to all nodes.
* clusterssh / cssh: will open number of xterm terminals to all nodes.
orgalorg intended to use in batch mode, no GUI is assumed. orgalorg, however,
can be used in interactive mode (see example section below).

* pssh: buggy, uses binary ssh, which is not resource efficient.
* pssh: buggy, uses binary ssh, which is not resource efficient.
orgalorg uses native SSH protocol implementation, so safe and fast to use
on thousand of nodes.

@@ -67,7 +69,7 @@ go get github.com/reconquest/orgalorg
# Example usages

`-o <host>...` in later examples will mean any supported combination of
host-specification arguments, like
host-specification arguments, like
`-o node1.example.com -o node2.example.com`.

## Evaluating command on hosts in parallel
@@ -170,7 +172,6 @@ So, orgalorg expected to work with third-party synchronization tool, that
will do actual files relocation and can be quite intricate, **but orgalorg can
work without that tool and perform simple files sync (more on this later)**.


## Global Cluster Lock

Before doing anything else orgalorg will perform global cluster lock. That lock
@@ -191,7 +192,6 @@ sync procedure.
User can stop there by using `--lock` or `-L` flag, effectively transform
orgalorg to the distributed locking tool.


## File Upload

Files will be sent from local node to the amount of specified nodes.
@@ -209,7 +209,8 @@ directory and then exit.

orgalorg preserves all file attributes while transfer as well as user and group
IDs. That behaviour can be changed by using `--no-preserve-uid` and
`--no-preseve-gid` command line options.
`--no-preseve-gid` command line options. These flags are ignored when orgalorg
is ran from Windows.

By default, orgalorg will keep source file paths as is, creating same directory
layout on the target nodes. E.g., if orgalorg told to upload file `a` while
@@ -224,7 +225,6 @@ remote nodes, `--sudo` or `-x` can be used to elevate to root before uploading
files. It makes possible to login to the remote nodes under normal user and
rewrite system files.


## Synchronization Tool

After file upload orgalorg will execute synchronization tool
@@ -245,7 +245,6 @@ to that program by using `--stdin` or `-i` flag.
Tool can accept number of arguments, which can be specified by using `-g` or
`--arg` flags.


# Synchronization Protocol

orgalorg will communicate with given sync tool using special sync protocol,
@@ -264,7 +263,6 @@ given prefix will be printed as is, untouched.

Communication begins from the hello message.


## Protocol

### HELLO
94 changes: 0 additions & 94 deletions archive.go
Original file line number Diff line number Diff line change
@@ -2,12 +2,10 @@ package main

import (
"archive/tar"
"fmt"
"io"
"os"
"path/filepath"
"sync"
"syscall"

"github.com/reconquest/hierr-go"
"github.com/reconquest/lineflushwriter-go"
@@ -154,98 +152,6 @@ func archiveFilesToWriter(
return nil
}

func writeFileToArchive(
fileName string,
stream io.Writer,
archive *tar.Writer,
workDir string,
preserveUID, preserveGID bool,
) error {
fileInfo, err := os.Stat(fileName)
if err != nil {
return hierr.Errorf(
err,
`can't stat file for archiving: '%s`, fileName,
)
}

// avoid tar warnings about leading slash
tarFileName := fileName
if tarFileName[0] == '/' {
tarFileName = tarFileName[1:]

fileName, err = filepath.Rel(workDir, fileName)
if err != nil {
return hierr.Errorf(
err,
`can't make relative path from: '%s'`,
fileName,
)
}
}

header := &tar.Header{
Name: tarFileName,
Mode: int64(fileInfo.Sys().(*syscall.Stat_t).Mode),
Size: fileInfo.Size(),

ModTime: fileInfo.ModTime(),
}

if preserveUID {
header.Uid = int(fileInfo.Sys().(*syscall.Stat_t).Uid)
}

if preserveGID {
header.Gid = int(fileInfo.Sys().(*syscall.Stat_t).Gid)
}

tracef(
hierr.Errorf(
fmt.Sprintf(
"size: %d bytes; mode: %o; uid/gid: %d/%d; modtime: %s",
header.Size,
header.Mode,
header.Uid,
header.Gid,
header.ModTime,
),
`local file: %s; remote file: %s`,
fileName,
tarFileName,
).Error(),
)

err = archive.WriteHeader(header)

if err != nil {
return hierr.Errorf(
err,
`can't write tar header for fileName: '%s'`, fileName,
)
}

fileToArchive, err := os.Open(fileName)
if err != nil {
return hierr.Errorf(
err,
`can't open fileName for reading: '%s'`,
fileName,
)
}

_, err = io.Copy(stream, fileToArchive)
if err != nil {
return hierr.Errorf(
err,
`can't copy file to the archive: '%s'`,
fileName,
)
}

return nil
}

func getFilesList(relative bool, sources ...string) ([]file, error) {
files := []file{}

Loading