Skip to content

Commit

Permalink
Revamp operations syntax in favor of op_name(path, args*) (#1)
Browse files Browse the repository at this point in the history
* Syntax should be like op_name(path, args*)
* Support for multiple operations in one go
  • Loading branch information
can3p authored Apr 14, 2024
1 parent 4118730 commit a28f27a
Show file tree
Hide file tree
Showing 11 changed files with 2,127 additions and 63 deletions.
118 changes: 84 additions & 34 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,48 @@
# Sackmesser - your cli to modify json/yaml on the fly
# Sackmesser - your cli to modify JSON/yaml on the fly

__Warning: sackmesser is a prototype at the moment, do not expect the api to be stable
__Warning: sackmesser is a prototype at the moment, do not expect the api to be stable__

Remember all those times where you had to save json in order to find and update a single field?
Remember all those times when you had to save JSON in order to find and update a single field?
Or worse, where you had to count spaces in yaml? Fear no more, sackmesser will take care of that.

## Capabilities

* Input and output formats are disconnected
* Support: set field, delete field
* Supports mutation only, you cannot query JSON with `sackmesser`
* Input and output formats are disconnected, both yaml and JSON are supported
* Operations: set field, delete field
* Supports multiple operations in one go

## Operations

Operations have a signature like `op_name(path, args*)` where

* path is dot delimited, you could use quotes in case field names contain spaces, dots, commas, etc:

```
echo '{ "a" : { "test prop": 1 } }' | sackmesser mod 'set(a."test prop", "test")'
{
"a": {
"test prop": "test"
}
}
```

* Zero or more arguments are required for an operation. An argument could be one of
- a number
- `null`
- A string, you could use single, double quotes and backticks as quotes to minimize escaping
- A JSON value

Available operations:

| Operation | Description |
| ------------- | ------------- |
| set(path, value) | assign field to a particular value |
| del(path) | delete a key |

## Examples:

### If you just want to convert json to yaml or back
### If you just want to convert JSON to yaml or back

```
$ echo '{ "a":1, "prop": { "b": [1,2,3] } }' | sackmesser mod --output-format yaml
Expand All @@ -27,29 +57,48 @@ prop:
### Set a field with a value

```
$ echo '{ "a":1 }' | sackmesser mod '.prop' '{ "test": 123 }'
$ echo '{ "a":1 }' | sackmesser mod 'set(prop, `{ "test": 123 }`'
{
"a": 1,
"prop": "{ \"test\": 123 }"
}
```

### Set a field with a value that will be parse as a json first
Please note that you can use three types of quotes for strings - double quotes, single quotes, and backticks

```
$ echo '{ "a":1 }' | sackmesser mod -j '.prop' '{ "test": 123 }'
$ echo '{ "a":1 }' | sackmesser mod "set(prop, 'value')"
{
"a": 1,
"prop": "value"
}
```

```
$ echo '{ "a":1 }' | sackmesser mod "set(prop, `value`)"
{
"a": 1,
"prop": "value"
}
```

### Set a field with a value that will be parsed as a JSON first

```
$ echo '{ "a":1 }' | sackmesser mod 'set(prop, { "test": 123 }'
{
"a": 1,
"prop": {
"test": 123
}
}%
}
```

You can always spit out a different format if you want!

```
$ echo '{ "a":1 }' | sackmesser mod --output-format yaml -j '.prop' '{ "test": 123 }'
$ echo '{ "a":1 }' | sackmesser mod --output-format yaml 'set(prop, { "test": 123 }'
{
a: 1
prop:
test: 123
Expand All @@ -58,19 +107,31 @@ prop:
### Delete a field

```
echo '{ "a":1, "deleteme": "please" }' | sackmesser mod -d '.deleteme'
echo '{ "a":1, "deleteme": "please" }' | sackmesser mod 'del(deleteme)'
{
"a": 1
}
```

### Chain commands

You can supply as many commands as you like if needed

```
echo '{ "a":1, "deleteme": "please" }' | sackmesser mod 'set(b, "test")' 'del(deleteme)'
{
"a": 1,
"b "test",
}
```

See [TODO](#TODO) section for possible changes

## Installation

### Install Script

Download `sackmesser` and install into a local bin directory.
Download `sackmesser` and install it into a local bin directory.

#### MacOS, Linux, WSL

Expand All @@ -86,10 +147,10 @@ Specific version:
curl -L https://raw.githubusercontent.com/can3p/sackmesser/master/generated/install.sh | sh -s 0.0.4
```

The script will install the binary into `$HOME/bin` folder by default, you can override this by setting
The script will install the binary into the `$HOME/bin` folder by default, you can override this by setting
`$CUSTOM_INSTALL` environment variable

### Manual download
### Manual Download

Get the archive that fits your system from the [Releases](https://github.com/can3p/sackmesser/releases) page and
extract the binary into a folder that is mentioned in your `$PATH` variable.
Expand All @@ -100,29 +161,18 @@ The project has been scaffolded with the help of [kleiner](https://github.com/ca

## TODO

Current syntax sucks, because

- It's only possible to specify one operation
- No aggregations, merges
- sackmesser will not create a chain of nested objects for you if you give a path that includes non existing fields. Ideally this should work: `echo '{}' | sackmesser mod 'a.b.c.d' 123`

Parsers are not perfect as well

- Indentation settings are hardcoded
- Ideally the updated json should be formatted identically to the input except modified fields, unless specifically asked for

And in general

- More operations
- Some tests will be helpful

Some dream scenarios:
Or something like this, suggestions are welcome!

## Prior art

- `echo { "a": 1 } | sackmesser 'inc(.a)' -> { "a": 2 }`
- `echo { "a": 1 } | sackmesser 'inc(.a)' '.b = true' -> { "a": 2, "b": true }`
- `echo { "a": [1,2,3] } | sackmesser '.len = len(.a)' -> { "a": [1,2,3], len: 3 }`
- `echo { "props": [ { "field": "value1 }, { "field2": "value1 } ] } | sackmesser '.props[].index = index()' -> { "props": [ { "index": 0, "field": "value1 }, { "index": 1, "field2": "value1 } ] }`
There are awesome alternatives to `sackmesser`, which should be considered as well!

Or somethings like this, suggestions welcome!
* [jq](https://jqlang.github.io/jq/) - legendary json processor. Compared to `sackmesser` has infinite capabilities
heavily skewed towards reading the data, however, mutation is also possible, `jq` works with JSON only
* [jj](https://github.com/tidwall/jj) - this tool is optimized for speed and supports JSON lines. Compared to `sackmesser` it only supports one operation at a time and is optimized for speed

## License

Expand Down
39 changes: 12 additions & 27 deletions cmd/mod.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,23 +4,20 @@ Copyright © 2024 Dmitrii Petrov <[email protected]>
package cmd

import (
"encoding/json"
"fmt"
"io"
"os"
"strings"

"github.com/can3p/sackmesser/pkg/cobrahelpers"
"github.com/can3p/sackmesser/pkg/operations"
"github.com/can3p/sackmesser/pkg/traverse/simplejson"
"github.com/can3p/sackmesser/pkg/traverse/simpleyaml"
"github.com/can3p/sackmesser/pkg/traverse/types"
"github.com/pkg/errors"
"github.com/spf13/cobra"
)

func ModCommand() *cobra.Command {
var deleteField bool
var jsonValue bool
var inputFormat string
var outputFormat string

Expand Down Expand Up @@ -52,32 +49,22 @@ func ModCommand() *cobra.Command {
}

if len(args) > 0 {
path := strings.Split(strings.TrimLeft(args[0], "."), ".")
ops := []*operations.OpInstance{}
parser := operations.NewParser()

if err != nil {
return err
}

if deleteField {
if err := operations.Delete(root, path); err != nil {
return err
}
} else {
for _, arg := range args {
op, err := parser.Parse(arg)

if len(args) < 2 {
panic("set operation requires at least two arguments")
if err != nil {
return errors.Wrapf(err, "Invalid operation: [%s]", arg)
}

var val any = args[1]

if jsonValue {
if err := json.Unmarshal([]byte(args[1]), &val); err != nil {
return err
}
}
ops = append(ops, op)
}

if err := operations.Set(root, path, val); err != nil {
return err
for _, op := range ops {
if err := op.Apply(root); err != nil {
return errors.Wrapf(err, "failed to apply operation: %s", op.String())
}
}
}
Expand Down Expand Up @@ -105,8 +92,6 @@ func ModCommand() *cobra.Command {
},
}

modCmd.Flags().BoolVarP(&deleteField, "delete", "d", false, "delete field from a given path")
modCmd.Flags().BoolVarP(&jsonValue, "json", "j", false, "parse value as json")
modCmd.Flags().Var(cobrahelpers.NewEnumFlag(&inputFormat, "json", "json", "yaml"), "input-format", `input format: json or yaml`)
modCmd.Flags().Var(cobrahelpers.NewEnumFlag(&outputFormat, "json", "json", "yaml"), "output-format", `input format: json or yaml`)

Expand Down
4 changes: 4 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,14 @@ require (

require (
aead.dev/minisign v0.2.0 // indirect
github.com/alecthomas/assert/v2 v2.8.1 // indirect
github.com/alecthomas/participle/v2 v2.1.1 // indirect
github.com/alecthomas/repr v0.4.0 // indirect
github.com/fatih/color v1.16.0 // indirect
github.com/go-resty/resty/v2 v2.11.0 // indirect
github.com/google/go-github/v57 v57.0.0 // indirect
github.com/google/go-querystring v1.1.0 // indirect
github.com/hexops/gotextdiff v1.0.3 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/kr/text v0.2.0 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
Expand Down
8 changes: 8 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
aead.dev/minisign v0.2.0 h1:kAWrq/hBRu4AARY6AlciO83xhNnW9UaC8YipS2uhLPk=
aead.dev/minisign v0.2.0/go.mod h1:zdq6LdSd9TbuSxchxwhpA9zEb9YXcVGoE8JakuiGaIQ=
github.com/alecthomas/assert/v2 v2.8.1 h1:YCxnYR6jjpfnEK5AK5SysALKdUEBPGH4Y7As6tBnDw0=
github.com/alecthomas/assert/v2 v2.8.1/go.mod h1:Bze95FyfUr7x34QZrjL+XP+0qgp/zg8yS+TtBj1WA3k=
github.com/alecthomas/participle/v2 v2.1.1 h1:hrjKESvSqGHzRb4yW1ciisFJ4p3MGYih6icjJvbsmV8=
github.com/alecthomas/participle/v2 v2.1.1/go.mod h1:Y1+hAs8DHPmc3YUFzqllV+eSQ9ljPTk0ZkPMtEdAx2c=
github.com/alecthomas/repr v0.4.0 h1:GhI2A8MACjfegCPVq9f1FLvIBS+DrQ2KQBFZP1iFzXc=
github.com/alecthomas/repr v0.4.0/go.mod h1:Fr0507jx4eOXV7AlPV6AVZLYrLIuIeSOWtW57eE/O/4=
github.com/can3p/kleiner v0.0.11 h1:TEFwzK0ZbJ9+KTik897Kb41fXpBlmJrcMyOM2zINDHU=
github.com/can3p/kleiner v0.0.11/go.mod h1:qX/0Iu5n92m3ChytdnGaon7VXw4bCyOoBiy4H6HQ8gM=
github.com/cpuguy83/go-md2man/v2 v2.0.3/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
Expand All @@ -17,6 +23,8 @@ github.com/google/go-github/v57 v57.0.0 h1:L+Y3UPTY8ALM8x+TV0lg+IEBI+upibemtBD8Q
github.com/google/go-github/v57 v57.0.0/go.mod h1:s0omdnye0hvK/ecLvpsGfJMiRt85PimQh4oygmLIxHw=
github.com/google/go-querystring v1.1.0 h1:AnCroh3fv4ZBgVIf1Iwtovgjaw/GiKJo8M8yD/fhyJ8=
github.com/google/go-querystring v1.1.0/go.mod h1:Kcdr2DB4koayq7X8pmAG4sNG59So17icRSOU623lUBU=
github.com/hexops/gotextdiff v1.0.3 h1:gitA9+qJrrTCsiCl7+kh75nPqQt1cx4ZkudSTLoUqJM=
github.com/hexops/gotextdiff v1.0.3/go.mod h1:pSWU5MAI3yDq+fZBTazCSJysOMbxWL1BSow5/V2vxeg=
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
Expand Down
2 changes: 1 addition & 1 deletion pkg/operations/delete.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import (
"github.com/can3p/sackmesser/pkg/traverse/types"
)

func Delete(root types.Node, path []string) error {
func Delete(root types.Node, path []string, args ...any) error {
if len(path) == 1 {
return root.DeleteField(path[0])
}
Expand Down
Loading

0 comments on commit a28f27a

Please sign in to comment.