Creating audio plugin, part 7

All posts series:
Part 1. Introduction and setup
Part 2. Learning Code
Part 3. VST and AU
Part 4. Digital Distortion
Part 5. Presets and GUI
Part 6. Signal synthesis
Part 7. Receive MIDI Messages
Part 8. Virtual Keyboard
Part 9. Envelopes
Part 10. Refinement GUI
Part 11. Filter
Part 12. Low-frequency oscillator
Part 13. Redesign
Part 14. Polyphony 1
Part 15. Polyphony 2
Part 16. Antialiasing

---

So far, we have generated only a constant sound wave, which simply sounded at a given frequency. Let's take a look at how you can respond to MIDI messages, turn the wave generation on and off at the desired frequency, depending on the note being received.

Receive MIDI messages

MIDI Processing Basics

')
When a plugin is loaded into a host, it receives all MIDI messages from the track to which it is hooked. When a note starts and ends, the ProcessMidiMsg function is called in the plugin. In addition to the note in MIDI messages, information about portamento (Pitch Bend) and control commands ( Control Changes , abbreviated CC) can be transmitted, which can be used to automate plug-in parameters. The ProcessMidiMsg function is passed an IMidiMsg message that describes the MIDI event in its format-independent form. This description contains the parameters NoteNumber and Velocity , which contain information about the pitch of the sound of our oscillator.

Each time a MIDI message arrives, the system already plays the audio buffer that was previously filled. There is no way to cram new audio at the exact moment a MIDI message is received. These events must be memorized until the next call to the ProcessDoubleReplacing function. You also need to remember the time of receipt of the message, so we will leave this information intact for the next filling of the buffer.

IMidiQueue will serve as a tool for these tasks.

MIDI receiver

We will use our project Synthesis . If you are using a version control system, it's time to commit the project. Create a new MIDIReceiver class and check that .cpp compiles in each target. In MIDIReceiver.h, insert the interface between #define and #endif :

 #pragma clang diagnostic push #pragma clang diagnostic ignored "-Wextra-tokens" #include "IPlug_include_in_plug_hdr.h" #pragma clang diagnostic pop #include "IMidiQueue.h" class MIDIReceiver { private: IMidiQueue mMidiQueue; static const int keyCount = 128; int mNumKeys; // how many keys are being played at the moment (via midi) bool mKeyStatus[keyCount]; // array of on/off for each key (index is note number) int mLastNoteNumber; double mLastFrequency; int mLastVelocity; int mOffset; inline double noteNumberToFrequency(int noteNumber) { return 440.0 * pow(2.0, (noteNumber - 69.0) / 12.0); } public: MIDIReceiver() : mNumKeys(0), mLastNoteNumber(-1), mLastFrequency(-1.0), mLastVelocity(0), mOffset(0) { for (int i = 0; i < keyCount; i++) { mKeyStatus[i] = false; } }; // Returns true if the key with a given index is currently pressed inline bool getKeyStatus(int keyIndex) const { return mKeyStatus[keyIndex]; } // Returns the number of keys currently pressed inline int getNumKeys() const { return mNumKeys; } // Returns the last pressed note number inline int getLastNoteNumber() const { return mLastNoteNumber; } inline double getLastFrequency() const { return mLastFrequency; } inline int getLastVelocity() const { return mLastVelocity; } void advance(); void onMessageReceived(IMidiMsg* midiMessage); inline void Flush(int nFrames) { mMidiQueue.Flush(nFrames); mOffset = 0; } inline void Resize(int blockSize) { mMidiQueue.Resize(blockSize); } }; 

Here we need to enable IPlug_include_in_plug_hdr.h , because otherwise IMidiQueue.h will create errors.
As you can see, we have a private IMidiQueue object for storing a queue of MIDI messages. We also store information about what notes are being played and how many of them are being played. Three parameters mLast... are needed, mLast... our plugin will be monophonic: each next note will drown out the previous ones (the so-called priority of the last note ). The noteNumberToFrequency function converts a MIDI note number to a frequency in Hertz. We use it because the Oscillator class works with a frequency, not a note number.
The public section contains a number of inline getters and sends Flush and Resize to the mMidiQueue .
In the Flush body, we set mOffset to zero. Calling mMidiQueue.Flush(nFrames) means that we remove its part from the nFrames of the queue with the size of nFrames , since we have already processed the events of this part in the previous call to the advance function. Resetting mOffset ensures that the next time during the advance we will also process the beginning of the queue. The words const , after parentheses, mean that the function will not change the immutable members of its class .

Let's add the implementation of onMessageReceived in MIDIReceiver.cpp :

 void MIDIReceiver::onMessageReceived(IMidiMsg* midiMessage) { IMidiMsg::EStatusMsg status = midiMessage->StatusMsg(); // We're only interested in Note On/Off messages (not CC, pitch, etc.) if(status == IMidiMsg::kNoteOn || status == IMidiMsg::kNoteOff) { mMidiQueue.Add(midiMessage); } } 

This function is called each time a MIDI message is received. We are currently only interested in note on and note off messages (start / stop playing a note), and we add them to the mMidiQueue .
The next interesting feature is advance :

 void MIDIReceiver::advance() { while (!mMidiQueue.Empty()) { IMidiMsg* midiMessage = mMidiQueue.Peek(); if (midiMessage->mOffset > mOffset) break; IMidiMsg::EStatusMsg status = midiMessage->StatusMsg(); int noteNumber = midiMessage->NoteNumber(); int velocity = midiMessage->Velocity(); // There are only note on/off messages in the queue, see ::OnMessageReceived if (status == IMidiMsg::kNoteOn && velocity) { if(mKeyStatus[noteNumber] == false) { mKeyStatus[noteNumber] = true; mNumKeys += 1; } // A key pressed later overrides any previously pressed key: if (noteNumber != mLastNoteNumber) { mLastNoteNumber = noteNumber; mLastFrequency = noteNumberToFrequency(mLastNoteNumber); mLastVelocity = velocity; } } else { if(mKeyStatus[noteNumber] == true) { mKeyStatus[noteNumber] = false; mNumKeys -= 1; } // If the last note was released, nothing should play: if (noteNumber == mLastNoteNumber) { mLastNoteNumber = -1; mLastFrequency = -1; mLastVelocity = 0; } } mMidiQueue.Remove(); } mOffset++; } 

This function is called every sample until the audio buffer is filled. While there are messages in the queue, we process them and remove them from the beginning (using Peek and Remove ). But we do this only for those MIDI messages whose offset ( mOffset ) is no larger than the buffer offset. This means that we process each message in the corresponding sample, leaving relative time shifts intact.
After reading the noteNumber and Velocity values, the conditional if separates the note on and note off messages (the absence of a velocity value is interpreted as note off ). In both cases, we keep track of which notes are played and how many of them are at the moment. The values ​​for mLast... are also updated mLast... to fulfill the priority of the last note. Further, it is logical that the frequency of the sound should be updated here, which is what we are doing. At the very end, mOffset updated so that the recipient knows how far this message is in the buffer at the moment. This can be reported to the recipient in another way - by passing the offset as an argument.
So, we have a class that receives all incoming MIDI messages note on / off. It keeps track of which notes are being played, which is the last note and what is its frequency. Let's use it.

Using a MIDI receiver

First, make these changes to resource.h carefully:

 // #define PLUG_CHANNEL_IO "1-1 2-2" #if (defined(AAX_API) || defined(RTAS_API)) #define PLUG_CHANNEL_IO "1-1 2-2" #else // no audio input. mono or stereo output #define PLUG_CHANNEL_IO "0-1 0-2" #endif // ... #define PLUG_IS_INST 1 // ... #define EFFECT_TYPE_VST3 "Instrument|Synth" // ... #define PLUG_DOES_MIDI 1 

These lines inform the host that our plugin “can midi”. 0-1 and 0-2 indicate that the plug-in has no audio input and there is one output, i.e. mono ( 0-1 ), or it has no audio inputs and there is a stereo output ( 0-2 ).
Now add #include "MIDIReceiver.h" after Oscillator.h to Synthesis.h . In the same section, in the public section, add a member function declaration:

 // to receive MIDI messages: void ProcessMidiMsg(IMidiMsg* pMsg); 

Add a MIDIReceiver object in the private section:

 private: // ... MIDIReceiver mMIDIReceiver; 

In Synthesis.cpp, write this simple function:

 void Synthesis::ProcessMidiMsg(IMidiMsg* pMsg) { mMIDIReceiver.onMessageReceived(pMsg); } 

It is called each time a MIDI message is received, and we transmit the message to our receiver.
Let's put things in order now a little. Edit both enums at the top:

 enum EParams { kNumParams }; enum ELayout { kWidth = GUI_WIDTH, kHeight = GUI_HEIGHT }; 

And create only one default preset:

 void Synthesis::CreatePresets() { MakeDefaultPreset((char *) "-", kNumPrograms); } 

If you change the parameters of the plugin, do nothing

 void Synthesis::OnParamChange(int paramIdx) { IMutexLock lock(this); } 

The handle in the interface is no longer useful to us. Let's reduce the constructor to the minimum necessary:

 Synthesis::Synthesis(IPlugInstanceInfo instanceInfo) : IPLUG_CTOR(kNumParams, kNumPrograms, instanceInfo) { TRACE; IGraphics* pGraphics = MakeGraphics(this, kWidth, kHeight); pGraphics->AttachPanelBackground(&COLOR_RED); AttachGraphics(pGraphics); CreatePresets(); } 

When the sound settings in the system change, we need to tell the oscillator a new sampling frequency:

 void Synthesis::Reset() { TRACE; IMutexLock lock(this); mOscillator.setSampleRate(GetSampleRate()); } 

We still have the ProcessDoubleReplacing function. Consider: mMIDIReceiver.advance() each sample needs to be executed. After that, we will know the frequency and volume with getLastVelocity and getLastFrequency from the MIDI receiver. Then we call mOscillator.setFrequency() and mOscillator.generate() to fill the audio buffer with the sound of the desired frequency.
The generate function was created to handle the entire buffer; The MIDI receiver works at the level of an individual sample: messages can have any offset within the buffer, which means mLastFrequency can change on any sample. We'll have to refine the Oscillator class so that it also works at the sample level.

First, we twoPI out twoPI from generate and move the Oscillator.h section to private . While we are here, let's immediately add the linden bool variable to indicate whether the oscillator is muted (that is, not a single note is played):

 const double twoPI; bool isMuted; 

We initialize them by adding a constructor to the initialization list. So it looks like:

 Oscillator() : mOscillatorMode(OSCILLATOR_MODE_SINE), mPI(2*acos(0.0)), twoPI(2 * mPI), // This line is new isMuted(true), // And this line mFrequency(440.0), mPhase(0.0), mSampleRate(44100.0) { updateIncrement(); }; 

Add the inline setter to the public section:

 inline void setMuted(bool muted) { isMuted = muted; } 

And immediately below this paste the following line:

 double nextSample(); 

We will call this function each sample and receive audio data from the oscillator.
Add the following code to Oscilator.cpp :

 double Oscillator::nextSample() { double value = 0.0; if(isMuted) return value; switch (mOscillatorMode) { case OSCILLATOR_MODE_SINE: value = sin(mPhase); break; case OSCILLATOR_MODE_SAW: value = 1.0 - (2.0 * mPhase / twoPI); break; case OSCILLATOR_MODE_SQUARE: if (mPhase <= mPI) { value = 1.0; } else { value = -1.0; } break; case OSCILLATOR_MODE_TRIANGLE: value = -1.0 + (2.0 * mPhase / twoPI); value = 2.0 * (fabs(value) - 0.5); break; } mPhase += mPhaseIncrement; while (mPhase >= twoPI) { mPhase -= twoPI; } return value; } 

As you can see, twoPI used twoPI . It would be redundant to calculate this value for each sample, so we added two pi as a constant to the class.
When the oscillator does not generate anything, we return zero. The switch construction is already familiar to you, even though we don’t use a for loop. Here we simply generate one value for the buffer, instead of filling it in its entirety. Also, this structure allows us to move the phase increment to the end, avoiding repetition.
This was a good example of refactoring, caused by insufficient code flexibility. Of course, we could think for an hour or two before starting to write the generate function with a “buffer” approach. But this implementation took us less than an hour entirely. In simple applications (like this) it is sometimes more efficient to implement an approach and see how the code handles the task in practice. Most often, as we just saw, it turns out that the whole idea was correct (the principle of calculating different sound waves), but some aspect of the problem was missed. On the other hand, if you are developing a public API, then changing something later is, to put it mildly, inconvenient, so here you have to think about everything in advance. In general, it depends on the situation.

The setFrequency function setFrequency also be called every sample. So updateIncrement will also be called very often. But it is not yet optimized:

 void Oscillator::updateIncrement() { mPhaseIncrement = mFrequency * 2 * mPI / mSampleRate; } 

2 * mPI * mSampleRate changes only when the sample rate changes. So the result of this calculation is better to remember and recalculate it only inside Oscillator::setSampleRate . It is worth remembering that exorbitant optimization can make the code unreadable and even ugly. In the specific case, we will not have performance problems, since we are writing an elementary monophonic synth. When we get to polyphony, it will be another matter, and then we will definitely optimize.
Now we can rewrite ProcessDoubleReplacing in Synthesis.cpp :

 void Synthesis::ProcessDoubleReplacing( double** inputs, double** outputs, int nFrames) { // Mutex is already locked for us. double *leftOutput = outputs[0]; double *rightOutput = outputs[1]; for (int i = 0; i < nFrames; ++i) { mMIDIReceiver.advance(); int velocity = mMIDIReceiver.getLastVelocity(); if (velocity > 0) { mOscillator.setFrequency(mMIDIReceiver.getLastFrequency()); mOscillator.setMuted(false); } else { mOscillator.setMuted(true); } leftOutput[i] = rightOutput[i] = mOscillator.nextSample() * velocity / 127.0; } mMIDIReceiver.Flush(nFrames); } 

In the for loop, the MIDI receiver first updates the values ​​(called advance ). If a note sounds ( velocity > 0 ), we update the oscillator frequency and let it sound. Otherwise we drown it (then nextSample will return zeros).
Then it all comes down simply to calling nextSample to get the value, change the volume ( velocity is an integer between 0 and 127 ) and write the result to the output buffers. At the end, Flush is called to remove the start of the queue.

Tests

Run VST or AU. If the AU does not appear in the host, then you may have to change the PLUG_UNIQUE_ID in resource.h . If two plugins have the same ID, the host will ignore everything except one.
The plugin must submit some MIDI data to the input. The easiest way is to use the REAPER virtual keyboard ( View → Virtual MIDI Keyboard menu). On the track with the plugin on the left there is a round red button. Go to the MIDI configuration by right-clicking on it and select to receive messages from the virtual keyboard:



In the same menu, turn on the Monitor Input . Now that the focus is on the virtual keyboard window , you can play the synthesizer with a regular keyboard. Type your name or password from the password manager and listen to how it sounds.
If you have a MIDI keyboard, then by connecting it, you can also test an independent application. The main thing is to choose the right MIDI input. If no sound is heard, try removing ~ / Library / Application Support / Synthesis / settings.ini .

The whole project at this stage can be downloaded from here .

Next time we will add a nice keyboard to the interface :)

Original article:
martin-finke.de/blog/articles/audio-plugins-009-receiving-midi