Interpretation of WandB "step"

[rhine3]

How can I interpret the “step” listed on WandB? It seems that WandB increments the step counter every time a call to wandb.log is made, but I can’t quite tell where that’s happening in OpenSoundscape.

Also, could the “step” counter correspond to different things on the same graph, potentially? E.g. if I plot training AND validation metrics on the same graph, could “step” correspond to subsequent batches for the training metrics, and epochs for the validation metrics?

[sammlapp]

Good question, the meaning of the 'step' is ambiguous when using WandB because, as you mention, step is incremented each time data is logged to wandb (documentation%20and%20then%20wandb.))

Because of this, it can be hard to interpret the step of wandb graphs, especially across runs with different batch sizes or number of batches per epoch. In the future, we might want to log in a way that allows another variable such as 'epoch' to be used as the x axis on wandb visualization plots, but I haven't quite figured out how to do this when I tried (not very hard) in the past. It's probably not too hard to do.

With regard to mismatch, since the step is global, you wouldn't have something like validation steps 1, 3, 6 corresponding to training steps 1, 50, 100 - I'm not sure if that's what you meant here. But you could have mismatches between runs as mentioned above.

OpenSoundscape is not explicit about where internal logging to wandb happens, but currently logging occurs during each training bach, and once during validation, if you are using opensoundscape.ml.CNN.train(). If you need to see all the places where logging occurs, search the codebase on github like this:
https://github.com/search?q=repo%3Akitzeslab%2Fopensoundscape+wandb_session.log%28&type=code

Sorry for a rambling answer, let me know if you have follow up q's

[rhine3]

Thanks Sam, that's a good explanation! Yes, that answers my question RE: mismatch--I was mostly wondering if wandb kept different global steps for "validation" and "training," but your answers clears that up.