[LV2] Combining MIDI + Atom Transport + Resize Port

Vladimir Sadovnikov sadko4u at gmail.com
Wed Apr 27 13:12:38 PDT 2016

Hello all!

I want to discuss LV2-related issue that got me stuck while preparing the new 
release of LSP Plugins.

I've built sampler plugin that has many parameters. If I implement all 
parameters as LV2 ports, I get about 1k ports.
Ardour just hung up while trying to show the generic UI with 1000 control ports.

So it's not a good practice to make all ports visible and I decided to 
'virtualize' the ports and implement LV2 transport.
So when the host displays UI, the UI queries plugin state from the host and 
initializes it's controls to proper values.
For the normal case the size of the plugin's state transferred between the UI 
and the plugin is about 60kbytes.
I specially split the transfer into different Atom objects of about 1kbyte size 
due to recent Carla's issues
(Carla couldn't handle atoms larger than 8kbyte).

Also the sampler plugin must receive and send (in reality - bypass) MIDI events.

The good practice is to have only one Atom port for the input and only one Atom 
port for the output.
We've discussed it with Robin Gareus in this Ardour's topic:

Also there is the Resize Port extension that allows to resize the buffer 
associated with the LV2 Atom port.

So my TTL generator analyzes the size of all 'virtual' ports of the plugin and 
generates the requested size for the worst case
(when string paths have PATH_MAX length etc...). For the 12-instrument sampler 
it's about 1M.

So the proper TTL for the Atom ports of the plugin should look like:

                 a lv2:InputPort, atom:AtomPort ;
                 atom:bufferType atom:Sequence ;
                 atom:supports patch:Message, midi:MidiEvent ;
                 lv2:designation lv2:control ;
                 lv2:index 70 ;
                 lv2:symbol "midi_in" ;
                 lv2:name "MIDI IN" ;
                 rdfs:comment "MIDI events input, UI <-> DSP communication" ;
                 rsz:minimumSize 619520 ;
         ] , [
                 a lv2:OutputPort, atom:AtomPort ;
                 atom:bufferType atom:Sequence ;
                 atom:supports patch:Message, midi:MidiEvent ;
                 lv2:designation lv2:control ;
                 lv2:index 71 ;
                 lv2:symbol "midi_out" ;
                 lv2:name "MIDI OUT" ;
                 rdfs:comment "MIDI events output, UI <-> DSP communication" ;
                 rsz:minimumSize 963584 ;

And what's in reality?
In reality I get Ardour's UI hanging up while trying to receive state (for low 
JACK buffer size), Carla hangs up for a while and does not show UI at all.
Both say that there is a Buffer Overflow in Atom transport.

So I disussed it with  Robin Gareus, provided source code of the plugins to him 
and asked to check the plugin 'from the Host-side'.

He accepted that when combining MidiEvents and Sequence in one Atom port, Ardour 
does not allocate buffer at all and
(If i right understand) uses JACK's buffer that is about 32k bytes. It's a nice 
surprise that Resize Port extension does not work in this case.
So Robin made additional commit that forces port to allocate the buffer (and 
this will be available for test after nightly build):

As an alternative solution Robin offered to separate MIDI transport and UI 
communication into different Atom ports.

OK, that's not a lot to do, so I've refactored the TTL generator and plugin 
wrapper. And what I got?

The MIDI ports:
                 a lv2:InputPort, atom:AtomPort ;
                 atom:bufferType atom:Sequence ;
                 atom:supports atom:Sequence, midi:MidiEvent ;
                 lv2:index 4 ;
                 lv2:symbol "in_midi" ;
                 lv2:name "Midi events input" ;
         ] , [
                 a lv2:OutputPort, atom:AtomPort ;
                 atom:bufferType atom:Sequence ;
                 atom:supports atom:Sequence, midi:MidiEvent ;
                 lv2:index 5 ;
                 lv2:symbol "out_midi" ;
                 lv2:name "Midi events output" ;

And the UI communication ports:

                 a lv2:InputPort, atom:AtomPort ;
                 atom:bufferType atom:Sequence ;
                 atom:supports atom:Sequence, patch:Message ;
                 lv2:designation lv2:control ;
                 lv2:index 72 ;
                 lv2:symbol "in_ui" ;
                 lv2:name "UI IN" ;
                 rdfs:comment "UI -> DSP communication" ;
                 rsz:minimumSize 615424 ;
         ] , [
                 a lv2:OutputPort, atom:AtomPort ;
                 atom:bufferType atom:Sequence ;
                 atom:supports atom:Sequence, patch:Message ;
                 lv2:designation lv2:control ;
                 lv2:index 73 ;
                 lv2:symbol "out_ui" ;
                 lv2:name "UI OUT" ;
                 rdfs:comment "DSP -> UI communication" ;
                 rsz:minimumSize 959488 ;

Now in Ardour I got SEGFAULT:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffd2c5b700 (LWP 3145)]
0x00007ffff723dd40 in lv2_evbuf_get_size () from 
Missing separate debuginfos, use: zypper install 
jamin-debuginfo-0.95.0-257.1.4.x86_64 ladspa-AMB-debuginfo-0.6.1-3.1.2.x86_64 
ladspa-MCP-debuginfo-0.4.0-3.1.2.x86_64 ladspa-REV-debuginfo-0.3.1-3.1.2.x86_64 
ladspa-VCO-debuginfo-0.3.0-3.1.2.x86_64 ladspa-WAH-debuginfo-0.0.2-3.1.x86_64 
ladspa-blop-debuginfo-0.2.8-3.1.2.x86_64 ladspa-bs2b-debuginfo-0.9.1-4.1.x86_64 
ladspa-caps-debuginfo-0.4.4-2.1.5.x86_64 ladspa-cmt-debuginfo-1.15-3.1.2.x86_64 
ladspa-debuginfo-1.13-26.1.2.x86_64 ladspa-foo-plugins-debuginfo-1.2-4.1.x86_64 
ladspa-matched-debuginfo-1-3.1.2.x86_64 ladspa-njl-debuginfo-0.2.1-2.1.x86_64 
ladspa-omins-debuginfo-0.2.1-4.1.x86_64 ladspa-preamp-debuginfo-2-2.1.2.x86_64 
ladspa-vcf-debuginfo-0.0.5-3.1.2.x86_64 ladspa-vlevel-debuginfo-0.5-3.1.2.x86_64 
ladspa-vocoder-debuginfo-0.3-3.1.2.x86_64 ladspa-wasp-debuginfo-0.1.4-3.1.x86_64 
libX11-6-debuginfo-1.6.2-5.3.1.x86_64 libX11-xcb1-debuginfo-1.6.2-5.3.1.x86_64 
libXau6-debuginfo-1.0.8-5.1.2.x86_64 libXcursor1-debuginfo-1.1.14-5.1.2.x86_64 
libXext6-debuginfo-1.3.3-2.1.2.x86_64 libXfixes3-debuginfo-5.0.1-4.1.2.x86_64 
libxcb-shm0-debuginfo-1.11-2.3.1.x86_64 libxcb1-debuginfo-1.11-2.3.1.x86_64 
(gdb) bt
#0  0x00007ffff723dd40 in lv2_evbuf_get_size () from 
#1  0x00007ffff723ddcd in lv2_evbuf_end () from 
#2  0x00007ffff723872f in ARDOUR::LV2Plugin::connect_and_run(ARDOUR::BufferSet&, 
ARDOUR::ChanMapping, ARDOUR::ChanMapping, unsigned int, long) () from 
#3  0x00007ffff709dd28 in 
ARDOUR::PluginInsert::connect_and_run(ARDOUR::BufferSet&, unsigned int, long, 
bool, long) () from /opt/Ardour-4.7.755/lib/libardour.so.3
#4  0x00007ffff709efaf in ARDOUR::PluginInsert::run(ARDOUR::BufferSet&, long, 
long, unsigned int, bool) () from /opt/Ardour-4.7.755/lib/libardour.so.3
#5  0x00007ffff71083ae in 
ARDOUR::Route::process_output_buffers(ARDOUR::BufferSet&, long, long, unsigned 
int, int, bool) () from /opt/Ardour-4.7.755/lib/libardour.so.3
#6  0x00007ffff7103c8e in ARDOUR::Route::passthru(ARDOUR::BufferSet&, long, 
long, unsigned int, int) () from /opt/Ardour-4.7.755/lib/libardour.so.3
#7  0x00007ffff7106b84 in ARDOUR::Route::no_roll(unsigned int, long, long, bool) 
() from /opt/Ardour-4.7.755/lib/libardour.so.3
#8  0x00007ffff6f22a10 in ARDOUR::Graph::process_one_route(ARDOUR::Route*) () 
from /opt/Ardour-4.7.755/lib/libardour.so.3
#9  0x00007ffff6f22d23 in ARDOUR::Graph::run_one() () from 
#10 0x00007ffff6f22da8 in ARDOUR::Graph::helper_thread() () from 
#11 0x00000000008b4383 in boost::function0<void>::operator()() const ()
#12 0x00007fffe841b8d9 in ARDOUR::JACKAudioBackend::_start_process_thread(void*) 
() from /opt/Ardour-4.7.755/lib/backends/libjack_audiobackend.so
#13 0x00000032d6e080a4 in start_thread () from /lib64/libpthread.so.0
#14 0x00000032d62e500d in clone () from /lib64/libc.so.6

In Carla I get strange behaviour: all atoms are serialized into MIDI ports, 
there is no transfer over 'in_ui' and 'out_ui' ports. So my plugin and UI does 
not see transferred Atom objects.

So we get that both variants do not work well and I can not release the LV2 
version of plugins (the LinuxVST version of plugin, indeed, works well because 
has no LV2 Atom Transport).

*BUT* the both cases are valid for the LV2 Specification. Also the official 
examples of the LV2 distribution (eg-sampler) do demonstrate the possibility of 
re-using Atom ports for MIDI event transfers.
Also we have deprecated LV2 Event extension that forces us to use Atom ports.

That's not the first issue with Atoms that delivered pain to me, so I want to 
discuss the proper behavior of
the plugin and the host in terms of combining LV2 Atom port, MIDI transport and 
Resize Port extensions:

- Is it a normal practice to separate MIDI ports into independent Atom ports? 
What's the proper solution?
- How the hosts should handle MIDI ports implemented as independent Atom ports?
- Is it a normal practice to combine MIDI event transport and Atom transport 
into one Atom port? What's the proper solution, again?
- How the hosts should handle combined MIDI and Atom transport with enabled 
Resize Port extension?
- How the hosts should handle multiple Atom ports with the same direction 
- Why official LV2 Specs do not say anything about how to use multiple Atom 
ports by the plugin and how to handle multiple Atom ports by the host?

Thanks a lot.
Maybe this discussion can improve the clarity of the LV2 standard and the 
stability of LV2 hosts and plugins.

More information about the Devel mailing list