MAIN GOAL: Figure out how the higher-level pieces of web synth are going to fit together wrt. the [midi-editor], [track-compositor], and [graph-editor]. Now that the new MIDI editor v2 has come to be, it's time to start thinking about how it's going to fit into the broader music production flow of the tool. We want to keep the MIDI editor as a first-class citizen of the [audio-graph] but still make it possible to embed in a more rich way into the track compositor so that we can do things like edit multiple tracks simultaneously.
There are in essence a few competing use-cases for the MIDI editor that it all needs to serve. Happily, the same MIDI editor works in all of them; it's very generic, and it doesn't really need to do anything different. Well that's mostly true; the live looping functionality will require a bit of extra features that aren't needed by the others, but perhaps not. Let's list these use-cases:
I... think that's it actually. Interesting, I can't really think of anything else I'd want it to do. Maybe there are things I'm forgetting or other things that come up, but I think that those two use-cases are a good starting point. I think that none of the work that has been done so makes either of these two things harder to achieve or blocks things in any way so that's good.
To start, I'll focus on the standalone use-case to match the existing functionality. We're happily already very close to achieving that; changing the data structure on the backend, de-coupling the rendering from the bookkeeping, and using a proper graphics library to render everything combined to allow the MIDI editor v2 to be built in like 10% of the time of the first one while also being WAY less buggy, look better, and have the same features.
We will need to add the loopping in, and we already have a rough idea of what that will look like. We have playback modes which determine what the MIDI Editor is doing at that moment, whether it's not playing back/edit mode, in loop mode, or playing back straight. We will need to add a cursor for all of these use-cases but that's easy enough. For playback, we have a couple of options. The easiest is to just use the [event-scheduler] to play all notes in all of the playback modes. The obvious drawback of this is that we gain latency and lose accuracy due to the fact that all of that is executed on the main/render thread which means it has to contest with all of the UI JS running for the application. However, the benefit is that it's deadly simple and is as generic as it gets since all of the scheduled events are just arbitrary closures.
The alternative to that idea would be to create a dedicated MIDI-processing AWP that converts MIDI events to param streams. It would allow us to get sample-accurate MIDI control for gating/ungating and stuff like that, but it would require uplifting the whole MIDI system to support that kind of thing, or at least creating a shim that can bridge between the existing JS-level MIDINodes and the audio thread level. It is clearly the better solution from a performance/features perspective, but I think it's too large of a change to commit to right now. I really believe that web synth can produce good stuff and be truly usable using the "hacky" function-call-based solution with event schedule for this kind of thing.
So in that case, I think we have a good idea of next steps. We will build the playback system for the MIDI editor, both the straight playback as well as looped mode. It will be implemented with event scheduler and synchronized with the global time/tempo counter. We should expand the time/tempo counter to support starting/stopping/resetting, and by doing that we can expand it for use across all modules and use it as the basis of a global synchronized beat/tempo counter to keep everything in sync. Really nice!
We won't have to implement a custom AWP for the MIDI editor at all; the [event-scheduler] will serve as the JS<->audio thread interface for all playback needs. We can use the SharedArrayBuffer-based communication solution for the cursor position which will be very nice, similarly to how we handle ADSR phase in the [fm-synth] and other places.
I'm excited again - this great. I know that I originally set off to solve some of the higher-level audio-graph-based problems for how the MIDI editor is going to connect to different "instruments", but that's a track compositor problem and I've realized we're not quite ready to deal with that yet. Happily, there is a clear (and from what I can tell mostly unblocked and achievable) path forward that I can pursue immediately.
Alright - enough time planning - let's build.
Actually I want to do a bit of planning for the actual scheduling implementation before I go ahead and do that.
Scheduling for the oneshot playthrough is pretty clear, I don't care too much about it.
For the looped one, it's a bit more complicated but I have an idea but I just want to get it down into words. We will schedule one loop, and then we will schedule an event to take place before the end of the current loop. It is at that point when we schedule the whole next loop and the re-scheduling event for the next loop, so it will be like callbacks in that regard.
I don't want to support starting scheduling from the middle of a loop; that sounds like needless complication and I don't think it adds anything. Whenever the loop is started, we jump the cursor to the start and init scheduling from there. We will snap the cursor to the selected beat snap interval just like all note operations.
One other thing I just thought of is adding a global start/stop callback registration system. Every VC can register callbacks to be triggered whenever the global start/stop state is toggled. That's quite nice. I think that the global start/stop toggle should have the effect of stoping the global beat counter when stopped and resetting the beat to zero when started. That can be the relied-upon behavior for all places that use it.
Once we support to the MIDI editor for this, we can add it to other modules as well like the sequencer. Cool - I actually think that's where we start with this. Build the global start/stop toggle, build the UI for it, implement its integration with the global beat counter/event scheduler, and we're off to a great start!