Sustainability of dependencies

[IgnatBeresnev]

Background

Dokka has a number of dependencies used for analyzing projects. Half of these dependencies must be updated every Kotlin release (like the Kotlin compiler and KGP). Unfortunately, not all of these dependencies (or their API) are stable or even meant to be used externally.

This past year, almost every single release, Dokka's maintainers had to spend a lot of time on resolving compatibility issues and transitive bugs that arise due to the updates. This leads to maintainers being frustrated and having to waste their time on fighting the dependencies rather than improving the product. And it leads to Dokka's users being frustrated because their projects also (as a consequence) start experiencing issues or get simply broken (#2977, #3081, #3038).

Goals

The main goal of this epic is to get Dokka's dependencies under control, to stabilize them as much as possible (or migrate to the stable API), and to identify upstream problems as soon as they are introduced, not months later.

More specifically,

• Improve predictability of maintenance. Dependency updates should not take weeks.
• Improve stability of Dokka from the user's perspective. Updating Kotlin in a user project should not cause Dokka to fail all of a sudden.
• Add Dokka to Kotlin's aggregate builds, so that compatibility problems introduced to any of Dokka's dependencies (like the compiler or KGP) will fail the aggregate build.
• Research if it's possible to include Dokka into the EAP program.

Dependencies

Evaluation of Dokka's hard dependencies:

• Kotlin Gradle Plugin (KGP):
  • Must be updated to the latest after every Kotlin release.
  • Dokka relies on it for project autoconfiguration (resolve platforms, source sets, classpath, etc).
  • Dokka uses some semi-internal / undocumented API, which tends to be unstable or gets deprecated/changed often.
  • At the moment, the biggest source of pain.
• K1:
  • Dokka depends heavily on the org.jetbrains.kotlin:kotlin-compiler artifact.
  • Dokka depends heavily on org.jetbrains.kotlin:idea and related artifacts (not anymore, addressed in Get rid of the dependency on Kotlin IDE plugins #2889)
  • Still used and needs to be updated every release until Dokka migrates to K2.
  • Less problems than with KGP, but occasional API changes and smaller underlying behaviour changes that are hard to spot.
• K2:
  • Stable compiler API is in the works, and at the moment it's not urgent as K2 analysis hasn't even been released.
  • To avoid the same mistake as with K1, we are starting to work on this in advance, see [K2] Use only stable compiler API #3141
• IntelliJ platform:
  • Multiple artifacts from the com.jetbrains.intellij package.
  • Used for Java source code analysis, as well as for some Kotlin parsing.
  • Needs to be updated only when kotlin-compiler also updates it, as parts of it are used and bundled in the compiler, and our versions need to match. Happens rarely.
  • Mostly stable, haven't had many major problems with it.
• Markdown:
  • org.jetbrains:markdown artifact.
  • Used for parsing module/package documentation (plain Markdown files), and some other minor things.
  • Mostly stable, haven't had to update it frequently.
• JPS (removed as part of Get rid of the dependency on Kotlin IDE plugins #2889)
  • An internal build tool for IntelliJ IDEs
  • Completely unrelated, not used and not needed by Dokka
  • Came as a transitive dependency with IDE artifacts

Subtasks

• Migrate Gradle runner's classpath resolution to stable API #3121
• [K2] Use only stable compiler API #3141

[IgnatBeresnev]

First iteration

In the first iteration, the problems with KGP should be addressed. A subtask has been created with more information: #3121

Internal discussion summary:

• We expect that with K2 around the corner, we'll be able to stop using K1 and thus get rid of the dependency on it, so it looks like we don't have to do anything about it now, and it will get resolved by itself over time.
  • Possible problems: if K2 analysis intentionally doesn't support a subset of projects, we might have to keep using K1 for longer, in which case it might continue being a problem. Need to discuss this with the Kotlin compiler team and see what they have to say.
• It's too early to do anything about K2, will come back to it after Kotlin 2.0.
• IntelliJ and Markdown dependencies do not bring much trouble, so they can be dealth with later.
• It's too early to even think about adding Dokka into Kotlin's aggregate build - need to get some of the items above resolved first. This is most likely a very long-term / end goal.