exp run --reset: clarify expected behavior

[daavoo]

Bug Report

Description

If the user has a pipeline with a stage using checkpoints and runs dvc exp run --reset without changing the deps, the checkpoints stage runs again.

Reproduce

1. Add checkpoint stage
2. dvc exp run
3. dvc exp run --reset

Expected

I would expect step 3 to don't run the checkpoint stage unless something changes in the workspace.

Environment information

Output of dvc doctor:

$ dvc doctor
DVC version: 2.6.3 (pip)
---------------------------------
Platform: Python 3.9.6 on macOS-10.16-x86_64-i386-64bit
Supports:
        http (requests = 2.26.0),
        https (requests = 2.26.0)
Cache types: reflink, hardlink, symlink
Cache directory: apfs on /dev/disk1s1s1
Caches: local
Remotes: None
Workspace directory: apfs on /dev/disk1s1s1
Repo: dvc, git

[pmrowla]

This is currently intended behavior, it's not really a bug. Doing exp run --reset is treated similarly to doing dvc repro --force, and will reproduce your entire pipeline from the initial checkpoint state.

It looks like the discussions regarding how to handle ending checkpoint experiments, and whether or not to change the current "infinite run" approach are still ongoing (#6104), are we sure that this change is going to be the desired behavior?

@dberenbaum

[daavoo]

I see. I guess that at least we should clarify this --force similarity in the docs, as it seems to be missing from https://dvc.org/doc/command-reference/exp/run#options

[jorgeorpinel]

So is this discussion still ongoing or does it summarize as iterative/dvc.org#2745 ?

[dberenbaum]

In a related issue, when looking at the upcoming blog post from @flippedcoder on transfer learning in iterative/dvc.org#2656, I noticed that the queued experiments didn't seem to start over (see the last table in the post). I would have expected each queued experiment to start from step 0 and not have a parenthetical modified checkpoint revision.

[karajan1001]

reset here is to reset the epochs to 0, so it is reasonable to restart the training.