[LV2] should plugins somehow indicate whether they support MPE?

Stefan Westerfeld stefan at space.twc.de
Tue Jun 20 12:20:32 PDT 2017


On Tue, Jun 20, 2017 at 07:26:20PM +0200, Hanspeter Portner wrote:
> On 20.06.2017 18:11, Stefan Westerfeld wrote:
> > On Sat, Jun 17, 2017 at 05:39:08PM +0200, Hanspeter Portner wrote:
> >> On 17.06.2017 13:04, Stefan Westerfeld wrote:
> >>> I don't know if that affects other plugins (yet), but the next version of
> >>> SpectMorph will support MPE (Multidimensional Polyphonic Expression). That way
> >>> a host can change parameters individually of notes that are already playing.
> >>
> >> LV2 parameters/controls for hosts are strict singletons, there is no notion of
> >> neither polyphony nor per-note parameters/controls.
> >>
> >> But you can of course support that internally in your plugin.
> > 
> > Right. In a way, MPE is all a big workaround for the idea that the host and
> > plugin communicate notes via MIDI. It allows me to only provide my current
> > VST2.4 and LV2 plugin, and still get per-note parameters, such as pitch.
> > 
> > I wonder if I should look into supporting VST3, which seems to have this by
> > design.  But then, LV2, which doesn't have it by design, would also need to be
> > extended somehow.
> > 
> >>> What I implemented so far in SpectMorph is changing the pitch. Unlike
> >>> conventional pitch-bend messages, this allows users to bend each note
> >>> individually. So you could slide from a C major chord to D minor. Obviously in
> >>> this case, a per-note-pitch UI like Bitwig provides - which I used for
> >>> developing and testing - makes sense.
> >>
> >> How do you plan to implement the MPE controller messages (e.g. pressure/timbre
> >> in MPE terms). Would you take the current singleton lv2:Control/Parameter value
> >> as initial value for each new note and overwrite it accordingly to MPE
> >> controller messages? What if the initial parent lv2:Control/Parameter is changed
> >> by the host simultaneously? Just curious.
> > 
> > To be honest, I don't know yet how per-note timbre events should be handled. It
> > is probably not as simple as you say, because often we have internal LFOs in the
> > plugin which affect timbre. This is because (unlike Bitwig), most hosts - as far
> > as I know - do not allow you to say: this timbre parameter should by default
> > vary with a sine wave with 0.2 Hz.
> > 
> > So just taking the LV2 control parameter for timbre is often not good enough,
> > the user can already make this parameter vary according to an LFO.
> > 
> > It could work like: use either
> >  - internal LFO
> >  - LV2 control parameter
> > 
> > if no other information is there, and use MPE timbre directly otherwise. This
> > would mean that the MPE timbre overrides any other (default) specification
> > for timbre.
> >
> >>> In VST, my plugin reacts to a new canDo("MPE"), to indicate to hosts with MPE
> >>> support that MPE messages should be sent (like per-note pitch bend). Bitwig for
> >>> instance will not send any MPE messages to the plugin unless this canDo is
> >>> supported.
> >>>
> >>> Since MPE is midi-only, my LV2 plugin will automatically support MPE now.  I
> >>> wonder if it should somehow in the plugin description indicate that it does.
> >>
> >> Suport could be annotated on the port with an atom:suppports property.
> >>
> >> mybundle:myplug
> >>   a lv2:Plugin ;
> >>
> >>   lv2:port [
> >>     a lv2:InputPort, atom:AtomPort ;
> >>     lv2:index 0 ;
> >>     lv2:symbol "myPort" ;
> >>     atom:bufferType atom:Sequence ;
> >>     atom:supports midi:MidiEvent, midi:MpeMessage ;
> >>   ] .
> >>
> >> And midi.ttl be extended with a definition of midi:MpeMessage
> >>
> >> midi:MpeMessage
> >>   a rdfs:Class ;
> >>   rdfs:subClassOf midi:MidiEvent ;
> >>   rdfs:label "Multidimensional Polyphonic Expression" .
> > 
> > Sounds reasonable.
> > 
> >>> I don't know any LV2 host that supports MPE so far, so if the LV2 strategy for
> >>> MPE would be wait until we have at least one host which supports MPE, and then
> >>> discuss negotiation, that would be ok for me, too
> >> Or better: wait until the MPE spec actually is finalized. Currently the draft is
> >> under (closed) consideration by the MIDI association, iirc, and it may well
> >> change. Once something is in the LV2 spec, it cannot (easily) be changed...
> > 
> > Ok. So we can wait for that to happen before annotating it.
> > 
> >> I'm pretty interested in polyphonic expression, but the MPE draft is terribly
> >> broken, as it only allows 15!!! concurrent notes at max.
> > 
> > MPE is a workaround. The proper way to do it would be somehow like VST3, where
> > the host tags (as far as I understand it) each note on event with a unique id.
> Hm, this reminds me of something. I distantly remember to have proposed an
> addition to the spec to get unique IDs. Will bump it in the corresponding thread.

Right, but as far as I understand it, this only allows generating IDs, but to
make it useful, these also need to be communicated along with the note on

> > That way, subsequent note expression parameter changes go to a note id. Since this
> > is not limited to 15 ids, we can avoid the problem you mentioned.
> > 
> > But this would mean that LV2 somehow would need to have
> >  - a protocol (other than midi) to send note on events, so that a note id can be
> >    assigned by the host
> >  - a way to send per-note expression control changes
> >  - a way to define per-note controls
> > 
> > If LV2 had this, I could probably support it, and hosts that supported it would
> > simply work with LV2s per-note-expression protocol. Also standard parameters
> > like per-note-pitch should be defined, and would then only affect the note with
> > the right id.
> I'm experimenting/protyping with such event systems in LV2. This is definitely
> doable with LV2's extendable atom event system.

Yes, I believe so, too. LV2 is so flexible that adding note expression should
be possible without a complete redesign. A port type which supports
more-than-midi messages would be necessary.

> But I wonder if plugins should at all need to implement such complexity.
> Wouldn't it be more straight-forward to force such plugins to be monophonic in
> their very nature and just let the host spawn the needed amount of instances to
> achieve polyphony (I think Ingen can do that). The host would thus decode MIDI
> MPE (or something better), plugins wouldn't need to implement it, it would not
> interfere with LV2's single-value control/parameter scheme and state
> saving/restoration would also work as intended.

I am fairly certain that this is not good enough for all cases.

Consider a simple soundfont player. If you instantiate a monophonic plugin 64
times per potential voice, the naive implementation would load the soundfont 64
times.  You could propose workarounds, like making the plugins implicitely
share state, but then it is no longer as simple and as transparent as you say.

Also, a sound font player would probably want to apply effects such as reverb
to all notes being played. In your model, this would mean computing reverb 64
times, which is inefficient.

If I look at SpectMorph, there are LFOs which affect all voices. It is possible
to share the LFO phase between all voices. So if you play a chord, all voices
will have the same slowly changing timbre.  Sharing phase is no longer trivial
if the individual voices live different isolated plugins.

Another SpectMorph example is that the behaviour is not just determined by the
LV2 controls alone. The UI and plugin share a complex state "morph plan" that
determines the sound. Currently, we have a 1:1 mapping between the UI and dsp
code, so the UI sends state change events produced by the user to the single
dsp plugin. Now if we had a 1:64 mapping, each UI state change message would
have to be processed 64 times by all voices. This would be at least inefficent,
but might cause other problems.

Also, if we now added some visualization, with the current model, we can send
visualization data once from the dsp plugin to the UI. It is not clear to me
how this case would even work in your proposed monophonic model.

   Cu... Stefan
Stefan Westerfeld, http://space.twc.de/~stefan

More information about the Devel mailing list