How to do a Kotlin style suspend?

[TwoClocks]

I did a project using kotlin's coroutines, and enjoyed it. I'm exploring giving scala a try and looking to use the same concurency style. I'm trying to figure out how to replicate the below kotlin code using dotty-cps-async, using a single-threaded executor. I can't seem to find an example in the docs of how to do an actual "suspend" that I can "resume" from outside of an async context.

My scala is still ok-ish, so it's likely an example/explanation exists in the docs, but it's too obtuse for my current level of understanding of the language.

Thanks in advance

lateinit var context: Continuation<Unit>

    suspend {
        val extra="extra"
        println("before suspend $extra")
        suspendCoroutine<Unit> { context = it }
        println("after suspend $extra")
    }.startCoroutine(
        object : Continuation<Unit> {
            override val context: CoroutineContext = EmptyCoroutineContext
            // called when a coroutine ends. do nothing.
            override fun resumeWith(result: Result<Unit>) {
                result.onFailure { ex : Throwable -> throw ex }
            }
        }
    )

    println("kick it")
    context.resume(Unit)

[rssh]

For direct work with continuations in Scala you should look at the gears project (https://github.com/lampepfl/gears )

In dotty-cps-async, you can define continuation monad (or use libraries that support continuations, like IO), and you can represent monadic computations in direct style using a compiler plugin see direct context encoding ), but you can't call resume continuation outside async context.

By the way, if I remember correctly, the situation in Kotlin is the same: you can call the suspend function only from the runBlocking argument or another suspend function.

Some basic examples with user-defined continuation monad can be found in tests: https://github.com/rssh/dotty-cps-async/blob/master/shared/src/test/scala/cpstest/CpsContT.scala.

[rssh]

In the previous reply, I forgot to mention CpsRuntimeAwait: i.e. if you can define monad for which CpsRuntimeAwait is available, then you can call suspend outside async block. In JVM it is possible to write CpsRuntimeAwait for simple task with continuations. Example is https://github.com/rssh/dotty-cps-async/blob/master/jvm-loom-addon/src/test/scala/cpsloomtest/TestPE.scala

You don't even need a monadic interface and dotty-cps-async for runtime continuations. (Just in suspend spawn, another virtual thread will submit a wake-up notification token to the top-level caller and put the current thread to sleep). In gears, similar mechanics is implemented in https://github.com/lampepfl/gears/blob/main/jvm/src/main/scala/async/VThreadSupport.scala.

[TwoClocks]

gears looks interesting, but I can't use loom. I'm using some exotic IO libraries via foreign function, as AFAICT loom doesn't offer a way for mere mortals to suspend. Also I'm just not of a fan of stack-full coroutines. I just want the compiler to make my callback based IO API to not look like callbacks. I think kotlin nailed this.

In kotlin, you can only suspend from a coroutine context, but you can resume from anywhere. The example snippet does this. This is the behavior I'm trying to reproduce in Scala.

[rssh]

In kotlin, you can only suspend from a coroutine context, but you can resume from anywhere.

But you can't receive results. In such a way, Scala offers the same capabilities. (i.e., you can call a method on monad, but you can't read the result outside monad).

The direct use of Continuation in your example is problematic. If you use Continuation, then you need Continuation support from the JVM (which, in fact, is Loom) or some monadic continuation interface (IO, ContT, etc.).

If only you need to run FFI in a separate thread, then using Continuation is overkill. For FFI calls you can create Future with custom execution context, which evaluates in the dedicated thread and has interface like:

object MyStrangeIO {
    val  dedicatedExecutionContext = ExecutionContext.fromExecutor(Executors.newSingleThreadExecuto))]

    def doSomething(a:A): Future[B] =
       //.  spawn future on dedicated thread
      given ExecutionContext = dedicatedExecutionContext
      Future{
          // FFI call here
      } 

}

Than you can use it with async/await or direct context encoding or monadic DSL outside async context/

async[Future]. {
    val b: B = doSomething(a).await
}

[TwoClocks]

The direct use of Continuation in your example is problematic. If you use Continuation, then you need Continuation support from the JVM (which, in fact, is Loom) or some monadic continuation interface (IO, ContT, etc.).

The kotlin code is single threaded and requires no support from the VM, nor does it use any libraries. The kotlin compiler implemented CPS. You can run it in the playground here : Kotlin Example. It transforms the coroutine into a function w/ a switch statement. One 'case' clause for each suspension point. A 'suspend' is effectively a 'return'.

Thread and context switches are expensive, I'd like to avoid them unless required. You should not need threading (virtual or otherwise) to do CPS. If you want to thread-pool fine, but CPS should work fine single threaded.

[rssh]

Then, if you really want continuations (which are overkill for FFI) then CpsContT example (in the first reply) is the maximum approximation 'from the ground' (i.e., you should build a monad with continuation support and execution runtime for it).

There are also libraries where continuations are already implemented (see, for example, https://typelevel.org/catseffect/docs/typeclasses/async ), but as I remember, all such libraries, which I know use thread-pool runtime in JVM by default. (From the other side, context switching is minimized.)

About threads and context switches are expensive: it's about OS threads. Green threads (like loom) were developed for cheap spawning and context-switching. What you are trying to do is to recreate 'green threading` over one carrier thread. Interesting, in what scenario difference between 'thread-pool green threading' and 'single-threaded green threading'. will be enough to have practical sense?

[rssh]

Yet one interesting question: What design is good for single-threaded concurrency and does not require continuations (and therefore is relatively simple)? I think it's a reactor model: i.e., one single-threaded executor, code in monads being chunks submitted for this executor. Two variants exist of such design: pure (where the task submitted to the executor when expressions are evaluated. We can name it's pure because it's pure functional) and impure (where the task submitted to the executor when expressions are created). Also note, that it will be hard to call async 'thread-pool' based task inside single-thread executor, because you should check when such task are done manually. Also, this is historically close to one of the first monadic concurrency implementations: A poor man’s concurrency monad

We have no exactly such examples but most close is: pure. impure. Note, that this is test examples, which are not devoted to real use, unoptimized at all, etc.... Building your concurrency from scratch is quite a deep rabbit hole.

[TwoClocks]

Building your concurrency from scratch is quite a deep rabbit hole.

Agreed. But with the requirement of this exotic native networking tech, and a preference for single-threaded-ness, it's forcing an exploration of the topic. Or go back to the 90s and use callbacks.

During this thread, I found the kotlin compiler dev's docs on code generation of continnuations. You might find them interesting if you haven't seen them already.

Thank you for being so generous with your time. I appreciate it.