[LV2] Worker extension

[Stefan Kersten]

On 3/23/12 4:13 PM, David Robillard wrote:
> On Fri, 2012-03-23 at 15:18 +0100, Stefan Kersten wrote:
>> On 3/23/12 2:42 PM, David Robillard wrote:
>>> On Fri, 2012-03-23 at 11:46 +0100, Stefan Kersten wrote:
> [...]
>>>> the only thing i am missing is priorities; e.g. when streaming from
>>>> disk, those messages could be made higher priority than other
>>>> non-realtime messages or be actually sent to a higher priority worker
>>>> thread. maybe the atom/message extension could be used to specify
>>>> worker/scheduler metadata (like priorities) in an extensible way?
>>>
>>> About 99999999 times more complicated than what I intend here, or what I
>>> think it's reasonable to demand hosts implement.  The idea here is to
>>> avoid a billion threads and ringbuffers everywhere, and it can be
>>> implemented with 2 ringbuffers, 1 buffer for the response (which could
>>> be eliminated at the cost of not giving the plugin a pointer to the
>>> response, but a read function instead), and a thread, all of which can
>>> be shared between plugins.
>>
>> sure, but there's no reason not to allow plugins provide information
>> that can *optionally* be interpreted by the host, is there?
>
> To be clear, I don't think priorities are good but unfortunately
> complicated, I actively think priorities are a bad idea for this.  Not
> only is it complicated, it's a long way from cost free, and introduces a
> whole new set of problems.  To me, it sounds like what you really want
> is multiple contexts to avoid one blocking the other.  This is
> reasonable.

that's probably what i meant, different execution contexts.

> I do think there is very good reason to not introduce troublesome ideas,
> especially ones that are not implemented.  We've learned the hard way in
> LV2 land that hypothetical functionality doesn't belong in extensions.

maybe if you backed your claims i could feel troubled, too ;)

>> call it contexts or priorities, but a way to transmit this information
>> to the host would be nice, e.g. as a parameter to `schedule_work'.
>
> These are dramatically different ideas.  A separate context would be a
> different instance of this entire mechanism, not a priority parameter.
> Separate callbacks, separate feature, separate everything.  Same API,
> different URI.

i don't see a real functional difference; depending on a context 
(priority) parameter the host can use a different queue + worker thread 
+ thread priority, just as with using a different uri to address another 
context. the only real difference is feature detectability as you note 
below.

> A real-time safe priority queue that avoids priority inversion is more
> like a research-level problem than something reasonable to expect from
> LV2 hosts.  Without a lot of restrictions it's not even theoretically
> possible.

the realtime thread is never waiting for the workers, so where's a 
chance for priority inversion? do you have some pointers? i'd be 
interested in reading more about this.

> Another context seems pretty easy, since the host can literally just
> provide a different instance of the exact same implementation if
> necessary.  You can use whatever URI for whatever context, and the
> existing feature/extension_data mechansim can be used to detect whether
> or not it is actually available.

yes, this makes sense.

> What is an example of a use case for this anyway?  I suspect there may
> be legitimate reason to add *one* more context, but only reluctantly.
> This is a very slippery slope.

e.g. in a plugin that streams samples from disk you obviously don't want 
to wait until a long-running non-realtime task has finished. this could 
be done separately in the plugin of course, but my plugins need to be 
realtime instantiatable, so i'll need the library descriptor extension 
from another thread on this list ...

with one more context i'd be happy, i guess ;)

<sk>