[LV2] Some thoughts/RFCs on the host stack

[David Robillard]

Hi all,

I've been finding some time to put into my long-enqueued project of
cleaning up the LV2 host stack (probably towards putting everything
into a single distribution to be more palatable to people who don't
just rely on package managers, and generally being a lot simpler). 
Since the host mess has become a topic of conversation lately, here's a
related brain dump / progress report:

Currently I'm working on serd1 (really, serd2) which unifies serd and
sord, to be the one non-LV2-specific library things are based on. 
Concretely, the main thing I'm shooting towards here is collapsing the
billion (well, 3 or 4) different node types there currently are:

SerdNode (purely syntactic string stuff)
SordNode (SerdNode plus some store specific stuff)
LilvNode (SordNode plus some numeric stuff and an API wrapper)

The idea is to merge these all into one, and expose SerdNode directly
in the Lilv API, so there's just one type of data node everywhere. 
This will greatly simplify a lot of things, but leads me to what I'm
currently wondering: we also have these atom things.

I think one of the good things about LV2, static data and not needing
to run code to just discover plugins, is also a big weakness, because
we have to make hard decisions about what's static and what's dynamic,
and these worlds are quite different.  Certain dynamic functionality
(say, dynamic channel configuration) is still missing because of this. 
Things would be nicer if dealing with the static data (via lilv) and
interacting with the plugin (via atoms) were as similar and compatible
as possible.  In an ideal world, nearly indistinguishable.

... and there's the tie-in.  Currently my SerdNode is:

struct SerdNode {
	size_t        n_bytes;
	SerdNodeFlags flags;
	SerdType      type;
        // data immediately follows here
};

Look familiar?  The flags are an optimization which can be dispensed
with without too much trouble, leaving:

struct SerdNode {
	size_t   n_bytes;
	SerdType type;
        // data immediately follows here
};

Toss the ability to deal with very large (gigabyte scale) individual
values (which is... maybe fine?) and we're left with:

struct SerdNode {
	uint32_t n_bytes;
	SerdType type;
        // data immediately follows here
};

Which just so happens to be binary compatible with LV2_Atom.

So I'm wondering if this is worth pursuing.  To make this work, we
would have to deal with the dynamic atom type problem (previously
mentioned here, and would make a bunch of other things much simpler on
its own), carving out some space for static type IDs.

Then, lilv could talk in a unified way about/with static data and live
plugins. Conceptually this is quite nice, but I haven't thought much
about what this concretely brings.

Being able to ask, via lilv, if a plugin supports a particular message
(say, OSC), and have this transparently use the static data or talk to
an instance if it depends on configuration?  Doing the same for channel
configuration?  Loading messages from disk (plugin data, saved state,
host specific stuff, etc), and passing them directly to a running
plugin instance?

In terms of clarity, I think it would help a lot of LV2 had a simple
data model (basically the atom one, a relatively simple set of strings
and numbers and stuff), and 100% of host and plugin communication used
it.  Host authors wouldn't really need to care (much) that there's this
static RDF data, and also a plugin to talk to, because it all speaks
the same language and lilv hides any gory details in most places.  Kids
these days like JSON, right?  It's not far off, just a lot more
meaningful :)

It would be quite nice if, say, a { 4, LV2_INT, 12345 } struct could
represent the int 12345 absolutely everywhere in the LV2 ecosystem. 

... so, that's that thought.  Sorry for the wall of text, I can't
really distill this into something more concrete, I'm just trying to
find the common core of nicety here and strip away as much other crap
as possible with the benefit of hindsight (the LADSPA-plus-a-thing-or-
two days are long ago, now).  Thoughts of whatever variety welcome.

Cheers,

-- 
dr