Adding draft extension for host-provided scratch memory

[jatinchowdhury18]

No description provided.

[abique]

Hi @jatinchowdhury18

I have some questions with the interface.

1. Is this scratch memory thread-local?
2. Can you get the scratch memory pointer from the thread pool? If so is it shared amongst the all the tasks, or each one will use the current thread local one?

[defiantnerd]

Hi @jatinchowdhury18

I have some questions with the interface.

1. Is this scratch memory thread-local?
2. Can you get the scratch memory pointer from the thread pool? If so is it shared amongst the all the tasks, or each one will use the current thread local one?

The only constraint is that the memory is valid for a plugin during it's process call. Like the event queue or the audio buffers. No further lifetime guarantees are given.

[abique]

Hi @jatinchowdhury18
I have some questions with the interface.

1. Is this scratch memory thread-local?
2. Can you get the scratch memory pointer from the thread pool? If so is it shared amongst the all the tasks, or each one will use the current thread local one?

The only constraint is that the memory is valid for a plugin during it's process call. Like the event queue or the audio buffers. No further lifetime guarantees are given.

Then consider this:

• each voice needs 10K of scratch buffer
• you process 32 voices in parallel

If you have a single scratch buffer for process(), you can't process your voice in parallel, or you need 32 * 10K.
If you have a thread local scratch that you can query from the thread pool, then you can process your voices in parallel with a scratch buffer of 10K.

I believe this needs to be clarified in the spec.

[defiantnerd]

Then consider this:

• each voice needs 10K of scratch buffer
• you process 32 voices in parallel

If you have a single scratch buffer for process(), you can't process your voice in parallel, or you need 32 * 10K. If you have a thread local scratch that you can query from the thread pool, then you can process your voices in parallel with a scratch buffer of 10K.

I believe this needs to be clarified in the spec.

You can only request/register one buffer per instance with a predefined size during activation. The host signals in the return value of request_size if you can access that.

In the process call you just call auto *scratchmem = ext_scratch_memory->access(host); and if scratchmem is not a null you can just use it during the process call.

But there is only one buffer and the size does not change as long as the plugin is active.

[baconpaul]

Then consider this:

• each voice needs 10K of scratch buffer
• you process 32 voices in parallel

If you have a single scratch buffer for process(), you can't process your voice in parallel, or you need 32 * 10K. If you have a thread local scratch that you can query from the thread pool, then you can process your voices in parallel with a scratch buffer of 10K.
I believe this needs to be clarified in the spec.

You can only request/register one buffer per instance with a predefined size during activation. The host signals in the return value of request_size if you can access that.

In the process call you just call auto *scratchmem = ext_scratch_memory->access(host); and if scratchmem is not a null you can just use it during the process call.

But there is only one buffer and the size does not change as long as the plugin is active.

I think Alex's point is if you use the thread pool extension to schedule jobs, those thread pool local jobs will be effectively parallel and running under process.

Can a thread-pool extension job access memory? If so is it distinct per thread or is it a single memory location?

My guess is: The thread pool and memory scratch extensions need some careful co-consideration. And the patterns where people use the scratch memory will also require scratch-per-thread-voice not scratch-per-process-block in those cases.

[jatinchowdhury18]

Yeah, thanks for bringing this up, I hadn't considered the inter-operation of scratch-memory and the thread-pool extension.

I think the simplest solution would be to have the plugin request the total amount of scratch memory that it needs across all possible threads. Since thread_pool.request_exec is blocking, I think it should be safe for the thread pool jobs to use the scratch memory, provided the plugin makes sure that each thread is using an independent "chunk" of the memory. So a simple example would look like:

struct My_Plugin
{
    size_t scratch_mem_per_voice = 10'000;
    size_t num_voices = 32;
    char* scratch_memory_data = nullptr;

    void activate(...) {
        scratch_memory_ext.pre_reserve(host, scratch_mem_per_voice * num_voices);
    }

    void process(...) {
        // Get all the scratch memory here
        scratch_memory_data = scratch_memory_ext.access(host);
        thread_pool_ext.request_exec(host, num_voices);
    }

    void thread_pool_callback(uint32_t task_index) {
        // Get a partition of the scratch memory for this voice to use
        char* this_voice_scratch_mem = scratch_memory_data + scratch_mem_per_voice * task_index;
        // do the actual DSP work here...
    }
};

However, I can see a few reasons why this might not be an ideal solution... For example, if the host thread pool only has 8 threads, then reserving enough scratch memory for 32 voices to be processed in parallel is a bit wasteful. Maybe the best solution is to have two scratch memory mechanisms... one scratch buffer that is intended to be accessed during the process() callback, and another to be accessed during the thread_pool_exec() callback?

typedef struct clap_host_scratch_memory {
    bool(CLAP_ABI *pre_reserve_process_scratch)(const clap_host_t *host, size_t scratch_size_bytes);
    void*(CLAP_ABI *access_process_scratch)(const clap_host_t *host);

    bool(CLAP_ABI *pre_reserve_thread_pool_scratch)(const clap_host_t *host, size_t scratch_size_per_thread_bytes);
    void*(CLAP_ABI *access_thread_pool_scratch)(const clap_host_t *host, uint32_t task_index);
} clap_host_scratch_memory_t;

All that said, I don't have much experience with the thread-pool extension, so I'll defer to your more informed opinions :).

[abique]

Here is my proposal:

The scratch is a thread local pointer, so if you retrieve it from the process call, you'll get a pointer that you can share with all the jobs. If you retrieve it from the thread pool, you get a pointer that is only for the current job.

I think this is the correct direction because it corresponds to how the host will implement this feature: each audio thread will have a single scratch buffer (thread local) whose size is greater or equal to the max requested size of all plugin instances.

The total scratch memory is: num_threads * max_scratch_size.
You definitelly don't want max_scratch_size = max(plug_num_voices * plug_voice_scratch_size), but we want instead max_scratch_size = max(plug_voice_scratch_size).

[Trinitou]

Do we ever use size_t in any other extension? -> uint?_t ?

[abique]

Good point, uint32_t would do the job I think.

[baconpaul]

Why not use uint64_t which is size_t in most of our systems? The host has the option to return no for values out of bounds

[jatinchowdhury18]

I don't have a preference for uint32_t vs uint64_t... but Trinitou is right that CLAP doesn't use size_t anywhere else, so I don't think we should use it here.

[abique]

A scratch size bigger than 4 GB would be problematic I suppose, remember nthreads * max_scratch_size.
Anyway regardless of the type, many host will likely have their own threshold.
uint32_t seems sufficient to me, but I'm happy with uint64_t as well.

[jatinchowdhury18]

The scratch is a thread local pointer, so if you retrieve it from the process call, you'll get a pointer that you can share with all the jobs. If you retrieve it from the thread pool, you get a pointer that is only for the current job.

This makes sense! I've added some comments intended to clarify this point, but please let me know if there are ways I can improve my explanation :).

[Trinitou]

It looks pretty good to me now. Any further details can be discussed on next IMO.