performance plot

[siwei-li]

No description provided.

[jettjaniak]

it's good, let's continue working on it!

1. make plotly plot instead - I'm not sure if using ipywidgets will be the best solution in this case, as plotly has some built-in support for dropdowns, see original issue description
2. change the input format to a tuple of (err_low, value, err_high) instead of list of all values - someone else will take care of computing this efficiently on the dataset scale
3. make it look somewhat like that, with colors as arguments with reasonable defaults; loss is log scale

4. see code comments

[siwei-li]

I have changed the plotting by using Plotly, the only thing I'm not sure about is this coloring thing @jettjaniak

3. make it look somewhat like that, with colors as arguments with reasonable defaults; loss is log scale

[jaidhyani]

lgtm, nice work

[jaidhyani]

Two things to follow up on:

1. I think we want the loss (y axis) to be log scale.

2. We should make color an argument of visualize_per_token_category (it can default to "purple")

[siwei-li]

Yeah I just thought that we already have data as in logprobs, so the loss calculated is already in log

1. I think we want the loss (y axis) to be log scale.

[jaidhyani]

Yeah, I think you're right. Jett, thoughts? You mentioned wanting a log scale in an earlier comment, were you just referring to charting logprob?

…

On Mon, Feb 19, 2024, 6:01 PM Siwei Li ***@***.***> wrote: Yeah I just thought that we already have data as in logprobs, so the loss calculated is already in log 1. I think we want the loss (y axis) to be log scale. — Reply to this email directly, view it on GitHub <#35 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAC5BY5AC5BBIH5EEQNXG4TYUP7WVAVCNFSM6AAAAABDJMJ7RCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJTGM3DSNZSG4> . You are receiving this because your review was requested.Message ID: ***@***.***>

addressed

[jettjaniak]

just had a meeting with Sil, a few small bits to address and we can merge

[siwei-li]

Todos:

• take log/linear kwarg options
• coloring kwargs fo the graph
• making dots larger