Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate CacheFileFormat to System.Text.Json #6081

Draft
wants to merge 11 commits into
base: dev
Choose a base branch
from

Conversation

Nigusu-Allehu
Copy link
Contributor

@Nigusu-Allehu Nigusu-Allehu commented Oct 3, 2024

Bug

Fixes: NuGet/Home#13059

Description

This PR makes sure we use System.Text.Json to read the cache file in CacheFileFormat.

PR Checklist

  • Meaningful title, helpful description and a linked NuGet/Home issue
  • Added tests
  • Link to an issue or pull request to update docs if this PR changes settings, environment variables, new feature, etc.

@Nigusu-Allehu Nigusu-Allehu requested a review from a team as a code owner October 3, 2024 23:04
@Nigusu-Allehu Nigusu-Allehu marked this pull request as draft October 3, 2024 23:04
@Nigusu-Allehu Nigusu-Allehu self-assigned this Oct 3, 2024
@Nigusu-Allehu Nigusu-Allehu marked this pull request as ready for review October 5, 2024 01:44
Copy link
Member

@nkolev92 nkolev92 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a few comments, but these are really conversations, stylistic ideas.

Once we're closed on this, I think this is ready to :shipit:

@Nigusu-Allehu Nigusu-Allehu force-pushed the dev-nyenework-migrate-to-system-text-json branch from 012c80f to 2834cc5 Compare October 15, 2024 21:15
@microsoft-github-policy-service microsoft-github-policy-service bot added the Status:No recent activity PRs that have not had any recent activity and will be closed if the label is not removed label Oct 23, 2024
{
internal class CacheFileProperties
{
internal const string VersionProperty = "version";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only place these properties are used are in CacheFile, in JsonPropertyName attributes. I think we're better off just putting the strings directly in the attribute (for example [JsonPropertyName("version")]), rather than having this class.

@microsoft-github-policy-service microsoft-github-policy-service bot removed the Status:No recent activity PRs that have not had any recent activity and will be closed if the label is not removed label Oct 23, 2024
@Nigusu-Allehu Nigusu-Allehu marked this pull request as draft October 29, 2024 22:00
{
public override IAssetsLogMessage Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
{
return JsonSerializer.Deserialize<AssetsLogMessage>(ref reader, options);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding https://github.com/NuGet/NuGet.Client/pull/6081/files#r1825078556, we lose about 30% perf needing to go though an additional converter:

code
using System.Text;
using System.Text.Json;
using System.Text.Json.Serialization;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

[MemoryDiagnoser]
public class Program
{
    private static void Main(string[] args)
    {
        BenchmarkRunner.Run<Program>();
    }

    private static JsonSerializerOptions options1;
    private static JsonSerializerOptions options2;

    [GlobalSetup]
    public void Setup()
    {
        options1 = new JsonSerializerOptions();
        options1.Converters.Add(new ThingConverter1());

        options2 = new JsonSerializerOptions();
        options2.Converters.Add(new ThingConverter2());
    }

    private byte[] json = Encoding.UTF8.GetBytes("{\"Value\":\"something\"}");

    [Benchmark]
    public IThing? JsonSerializerDeserialize()
    {
        return JsonSerializer.Deserialize<IThing>(json, options1);
    }

    [Benchmark]
    public IThing? JsonConverterRead()
    {
        return JsonSerializer.Deserialize<IThing>(json, options2);
    }

    [Benchmark(Baseline = true)]
    public Thing? Baseline()
    {
        return JsonSerializer.Deserialize<Thing>(json);
    }

    public interface IThing
    {
        public string Value { get; }
    }

    public class Thing : IThing
    {
        public required string Value { get; init; }
    }

    private class ThingConverter1 : JsonConverter<IThing>
    {
        public override IThing? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
        {
            return JsonSerializer.Deserialize<Thing>(ref reader, options);
        }

        public override void Write(Utf8JsonWriter writer, IThing value, JsonSerializerOptions options)
        {
            throw new NotImplementedException();
        }
    }

    private class ThingConverter2 : JsonConverter<IThing>
    {
        public override IThing? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
        {
            JsonConverter<Thing> converter = (JsonConverter<Thing>)options.GetConverter(typeof(Thing));
            return converter.Read(ref reader, typeof(Thing), options);
        }

        public override void Write(Utf8JsonWriter writer, IThing value, JsonSerializerOptions options)
        {
            throw new NotImplementedException();
        }
    }
}
Method Mean Error StdDev Ratio RatioSD Gen0 Allocated Alloc Ratio
JsonSerializerDeserialize 189.38 ns 1.185 ns 1.050 ns 2.00 0.02 0.0076 128 B 1.00
JsonConverterRead 123.56 ns 0.691 ns 0.647 ns 1.31 0.01 0.0076 128 B 1.00
Baseline 94.51 ns 0.602 ns 0.563 ns 1.00 0.01 0.0076 128 B 1.00

We lose even more perf calling JsonSerializer.Deserialize from the converter, rather than my suggestion above, getting the converter directly from options, and calling the converter.

It depends on how much we want to microoptimize restore. 30 nanoseconds per project, on my machine, but nanoseconds are super small. Is it worth the public API breaking changes for it? Probably not. But I still think that IAssetsLogMessage doesn't have a reason to exist.

But I think the additional 60 nanoseconds lost by using JsonSerializer.Deserialize is not worth the 1 line of code it saves over getting the converter directly.

public int EndColumnNumber { get; set; }

public AssetsLogMessage() { }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure that System.Text.Json can still call the constructor if it's internal. If this class had #nullable enable, then you'd see the problem with adding this constructor: it doesn't ensure that types the rest of our code expects to always be non-null, is in fact non-null. Therefore, we'd have to make Message a string?, and now everything that processes AssetsLogMessages has to handle when Message is null.

Technically, there's still the problem even if this constructor is internal or private, but at least we can suppress the nullable warning explaining why it happens. Still, I'd prefer if it could be avoided. Can we somehow use [JsonConstructor] on the LogLevel, NuGetLogCode, string, string overload? I don't have experience with trying to tell the JsonSerializer to set some values via the constructor, and different values via properties.

For much the same reasons, a good API is one you can't use wrong, so we don't want the rest of NuGet.Client being able to construct instances where message is null. If we need this constructor for JSON deserialization, then we should limit it's potential misuse.


if (logLevel == LogLevel.Warning)
{
WarningLevel = WarningLevel.Severe; // setting default to Severe as 0 implies show no warnings
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this logLevel == LogLevel.Warnings condition in order to set WarningLevel's default looks like a behaviour change from the old code, and it's not intuitive to me why the condition is needed.

@@ -106,5 +127,25 @@ public override int GetHashCode()

return combiner.CombinedHash;
}

private class ToStringConverter<T> : JsonConverter<T>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@microsoft-github-policy-service microsoft-github-policy-service bot added the Status:No recent activity PRs that have not had any recent activity and will be closed if the label is not removed label Nov 9, 2024

This PR has been automatically marked as stale because it has no activity for 7 days. It will be closed if no further activity occurs within another 7 days of this comment. If it is closed, you may reopen it anytime when you're ready again, as long as you don't delete the branch.

@dotnet-policy-service dotnet-policy-service bot removed the Status:No recent activity PRs that have not had any recent activity and will be closed if the label is not removed label Nov 9, 2024
@microsoft-github-policy-service microsoft-github-policy-service bot added the Status:No recent activity PRs that have not had any recent activity and will be closed if the label is not removed label Nov 18, 2024

This PR has been automatically marked as stale because it has no activity for 7 days. It will be closed if no further activity occurs within another 7 days of this comment. If it is closed, you may reopen it anytime when you're ready again, as long as you don't delete the branch.

@dotnet-policy-service dotnet-policy-service bot removed the Status:No recent activity PRs that have not had any recent activity and will be closed if the label is not removed label Nov 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Use System.Text.Json to read the cache file in CacheFileFormat
3 participants