Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: modify deprecated functions #6052

Merged
merged 2 commits into from
Dec 6, 2024

Conversation

machichima
Copy link
Contributor

@machichima machichima commented Nov 27, 2024

Tracking issue

Why are the changes needed?

All functions in pluginmachinery/utils/marshal_utils.go are marked as deprecated and should be changed. See:

// Deprecated: Use flytestdlib/utils.UnmarshalStructToPb instead.
func UnmarshalStruct(structObj *structpb.Struct, msg proto.Message) error {
if structObj == nil {
return fmt.Errorf("nil Struct Object passed")
}
jsonObj, err := jsonPbMarshaler.MarshalToString(structObj)
if err != nil {
return err
}
if err = jsonPbUnmarshaler.Unmarshal(strings.NewReader(jsonObj), msg); err != nil {
return err
}
return nil
}
// Deprecated: Use flytestdlib/utils.MarshalPbToStruct instead.
func MarshalStruct(in proto.Message, out *structpb.Struct) error {
if out == nil {
return fmt.Errorf("nil Struct Object passed")
}
jsonObj, err := jsonPbMarshaler.MarshalToString(in)
if err != nil {
return err
}
if err = jsonpb.UnmarshalString(jsonObj, out); err != nil {
return err
}
return nil
}
// Deprecated: Use flytestdlib/utils.MarshalToString instead.
func MarshalToString(msg proto.Message) (string, error) {
return jsonPbMarshaler.MarshalToString(msg)
}
// Deprecated: Use flytestdlib/utils.MarshalObjToStruct instead.
// Don't use this if input is a proto Message.
func MarshalObjToStruct(input interface{}) (*structpb.Struct, error) {
b, err := json.Marshal(input)
if err != nil {
return nil, err
}
// Turn JSON into a protobuf struct
structObj := &structpb.Struct{}
if err := jsonpb.UnmarshalString(string(b), structObj); err != nil {
return nil, err
}
return structObj, nil
}
// Deprecated: Use flytestdlib/utils.UnmarshalStructToObj instead.
// Don't use this if the unmarshalled obj is a proto message.
func UnmarshalStructToObj(structObj *structpb.Struct, obj interface{}) error {
if structObj == nil {
return fmt.Errorf("nil Struct Object passed")
}
jsonObj, err := json.Marshal(structObj)
if err != nil {
return err
}
if err = json.Unmarshal(jsonObj, obj); err != nil {
return err
}
return nil
}

Above function is deprecated and suggested using following function instead:

// UnmarshalStructToPb unmarshals a proto struct into a proto message using jsonPb marshaler.
func UnmarshalStructToPb(structObj *structpb.Struct, msg proto.Message) error {
if structObj == nil {
return fmt.Errorf("nil Struct object passed")
}
if msg == nil {
return fmt.Errorf("nil proto.Message object passed")
}
jsonObj, err := jsonPbMarshaler.MarshalToString(structObj)
if err != nil {
return errors.WithMessage(err, "Failed to marshal strcutObj input")
}
if err = UnmarshalStringToPb(jsonObj, msg); err != nil {
return errors.WithMessage(err, "Failed to unmarshal json obj into proto")
}
return nil
}
// MarshalPbToStruct marshals a proto message into proto Struct using jsonPb marshaler.
func MarshalPbToStruct(in proto.Message) (out *structpb.Struct, err error) {
if in == nil {
return nil, fmt.Errorf("nil proto message passed")
}
var buf bytes.Buffer
if err := jsonPbMarshaler.Marshal(&buf, in); err != nil {
return nil, errors.WithMessage(err, "Failed to marshal input proto message")
}
out = &structpb.Struct{}
if err = UnmarshalBytesToPb(buf.Bytes(), out); err != nil {
return nil, errors.WithMessage(err, "Failed to unmarshal json object into struct")
}
return out, nil
}
// MarshalPbToString marshals a proto message using jsonPb marshaler to string.
func MarshalPbToString(msg proto.Message) (string, error) {
return jsonPbMarshaler.MarshalToString(msg)
}
// UnmarshalStringToPb unmarshals a string to a proto message
func UnmarshalStringToPb(s string, msg proto.Message) error {
return jsonPbUnmarshaler.Unmarshal(strings.NewReader(s), msg)
}
// MarshalPbToBytes marshals a proto message to a byte slice
func MarshalPbToBytes(msg proto.Message) ([]byte, error) {
var buf bytes.Buffer
err := jsonPbMarshaler.Marshal(&buf, msg)
if err != nil {
return nil, err
}
return buf.Bytes(), err
}
// UnmarshalBytesToPb unmarshals a byte slice to a proto message
func UnmarshalBytesToPb(b []byte, msg proto.Message) error {
return jsonPbUnmarshaler.Unmarshal(bytes.NewReader(b), msg)
}
// MarshalObjToStruct marshals obj into a struct. Will use jsonPb if input is a proto message, otherwise, it'll use json
// marshaler.
func MarshalObjToStruct(input interface{}) (*structpb.Struct, error) {
if p, casted := input.(proto.Message); casted {
return MarshalPbToStruct(p)
}
b, err := json.Marshal(input)
if err != nil {
return nil, errors.WithMessage(err, "Failed to marshal input proto message")
}
// Turn JSON into a protobuf struct
structObj := &structpb.Struct{}
if err := UnmarshalBytesToPb(b, structObj); err != nil {
return nil, errors.WithMessage(err, "Failed to unmarshal json object into struct")
}
return structObj, nil
}
// UnmarshalStructToObj unmarshals a struct to the passed obj. Don't use this if the unmarshalled obj is a proto message.
func UnmarshalStructToObj(structObj *structpb.Struct, obj interface{}) error {
if structObj == nil {
return fmt.Errorf("nil Struct Object passed")
}
jsonObj, err := json.Marshal(structObj)
if err != nil {
return err
}
if err = json.Unmarshal(jsonObj, obj); err != nil {
return err
}
return nil
}

What changes were proposed in this pull request?

Based on the suggestion of the deprecated function's docstring, we should use functions in flytestdlib/utils.marshal_utils.go instead of pluginmachinery/utils/marshal_utils.go. Therefor, I rename the original utils import to pluginsUtils, and add flytestdlib/utils as new utils import. See below:

pluginsUtils "github.com/flyteorg/flyte/flyteplugins/go/tasks/pluginmachinery/utils"
utils "github.com/flyteorg/flyte/flytestdlib/utils"

and

"github.com/flyteorg/flyte/flytestdlib/utils"

Then modify the functions based on this new import. Only functions in marshal_utils.go are using new utils module flytestdlib/utils, others still use the original pluginmachinery/utils model.

This PR only modifies the ray.go and ray_test.go files currently.

How was this patch tested?

Unit tests

Setup process

Screenshots

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

Docs link

Use suggested function from the deprecated function's docs

Signed-off-by: machichima <[email protected]>
Copy link

codecov bot commented Nov 27, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 37.10%. Comparing base (6f187ae) to head (eed55d2).
Report is 23 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6052      +/-   ##
==========================================
+ Coverage   37.05%   37.10%   +0.04%     
==========================================
  Files        1316     1318       +2     
  Lines      132247   132403     +156     
==========================================
+ Hits        48998    49122     +124     
- Misses      78987    79009      +22     
- Partials     4262     4272      +10     
Flag Coverage Δ
unittests-datacatalog 51.58% <ø> (ø)
unittests-flyteadmin 54.10% <ø> (ø)
unittests-flytecopilot 30.99% <ø> (+8.76%) ⬆️
unittests-flytectl 62.29% <ø> (-0.18%) ⬇️
unittests-flyteidl 7.23% <ø> (-0.01%) ⬇️
unittests-flyteplugins 53.82% <100.00%> (+0.09%) ⬆️
unittests-flytepropeller 42.60% <ø> (-0.03%) ⬇️
unittests-flytestdlib 57.59% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Future-Outlier Future-Outlier self-assigned this Nov 29, 2024
@machichima machichima changed the title fix: modify deprecated functions [WIP] fix: modify deprecated functions Dec 1, 2024
@machichima machichima changed the title [WIP] fix: modify deprecated functions fix: modify deprecated functions Dec 5, 2024
Signed-off-by: Future-Outlier <[email protected]>
@Future-Outlier Future-Outlier enabled auto-merge (squash) December 6, 2024 16:05
@Future-Outlier Future-Outlier merged commit 9dc21f4 into flyteorg:master Dec 6, 2024
52 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

2 participants