API query error: unsupported Unicode escape sequence (SQLSTATE 22P05)
#13711
Comments
agilgur5
changed the title
unsupported Unicode escape sequence (SQLSTATE 22P05) errors unsupported Unicode escape sequence (SQLSTATE 22P05)
|
My guess is that one of your workflows has non-ascii characters in it that are upsetting sqllite. It's unclear if they're actually an invalid unicode escape sequence. Possibly in something like the description of the workflow. Can you look at the workflows in the cluster and work out which one(s) it might be and attach (a redacted version) of them. Ideally in a zip or tar file so as not to lose any bad escape characters via copy and pasting into github. |
|
any idea @jiachengxu @linzhengen @MasonM https://www.postgresql.org/message-id/368156.1677514339%40sss.pgh.pa.us mentions JSONB is stricter than JSON |
|
@tooptoop4 Good find, but the JSONB migration (#13779) wasn't in 3.5.11, so unless @yonirab has done the migration manually, that's not the cause. My first thought was that this is a regression from #12912, since that changed the However, it appears that was released as v3.5.6. |
|
i was able to reproduce this on master with postgres db and below workflow (click submit after creating the cron): in my case, since its a new installation, the workflow would run but fail to get archived: as you can see the \x00 from the workflow is being converted to \u0000 in the json. probably by so i think few things would need to happen:
|
|
We can do the marshaling like this: package main
import (
"encoding/json"
"fmt"
)
type RawString string
type Exa struct {
Name RawString `json:"name"`
}
func (r RawString) MarshalJSON() ([]byte, error) {
return []byte(r), nil
}
func (r *RawString) UnmarshalJSON(m []byte) error {
*r = RawString(m)
return nil
}
func main() {
x := `{"name": "\uC548"}`
e := Exa{}
err := json.Unmarshal([]byte(x), &e)
if err != nil {
fmt.Println("ERROR", err)
return
}
fmt.Println(e.Name)
fmt.Printf("%+v\n", e)
b, err := json.Marshal(e)
fmt.Println(string(b))
}This basically prevents any parsing of escaped chars. We will need to update the db. We won't need to reconvert though, I believe this got inserted as a NULL byte, if we fix this in the database it should return correctly as well. |
|
That being said, this feels like a validation error, we shouldn't accept @tooptoop4 's workflow here. Having a null byte in your workflow is obviously wrong? It feels like this should be prevented via the Unmarshal method really, we shouldn't really ever been able to parse this workflow. |
|
Here you go: some postgres functions rely upon The solution is to quote before insertion into postgres, unquote after read. |
|
Please run UPDATE argo_archived_workflows
SET workflow = replace(workflow::text, '\u0000', '')::json
WHERE strpos(workflow::text, '\u0000') > 0;@Joibel I think I'd like to have jsonb support in 3.5.x. This is (arguably a bug)/restriction on the postgres side, we should not be attempting to mutate workflows to stuff this into the database, I would like to reject these workflows ideally, the jsonb validation in postgres should do so. |
|
that update would break retries since the character was in the original workflow |
|
I'm not sure that jsonb would help in this case?
This could also just work with the strings being presented as |
More to prevent this workflow from going in, but I think we should accept these workflows now.
Yeah solution I was thinking of would be to use |
Pre-requisites
:latestimage tag (i.e.quay.io/argoproj/workflow-controller:latest) and can confirm the issue still exists on:latest. If not, I have explained why, in detail, in my description below.What happened? What did you expect to happen?
After upgrading from v3.5.6 to v3.5.11, we see the following error in the Argo UI whenever trying to list
SucceededorFailedworkflows:argo-server logs show the following:
Note also the following:
Our DB for archived workflows is Postgres 9.6.
We have the following defined in
workflow-controller-configmapVersion(s)
v3.5.11
Paste a minimal workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.
This is in our production environment immediately after updating to 3.5.11
Logs from the workflow controller
Logs from in your workflow's wait container
The text was updated successfully, but these errors were encountered: