Creating audio plug-ins, part 8

All posts series:
Part 1. Introduction and setup
Part 2. Learning Code
Part 3. VST and AU
Part 4. Digital Distortion
Part 5. Presets and GUI
Part 6. Signal synthesis
Part 7. Receive MIDI Messages
Part 8. Virtual Keyboard
Part 9. Envelopes
Part 10. Refinement GUI
Part 11. Filter
Part 12. Low-frequency oscillator
Part 13. Redesign
Part 14. Polyphony 1
Part 15. Polyphony 2
Part 16. Antialiasing

---

Setting up a virtual keyboard in REAPER is not so obvious, and besides, the user's host may not have this functionality at all. Let's add our little on-screen keyboard to the GUI.

GUI element

')
In WDL-OL, GUI elements are called controls . This library has an IkeyboardControl class that has all the necessary functionality for this task.
It uses one background image and two additional sprites: in one image the black key is pressed, in the other several whites are pressed. It is logical: all the black keys have the same shape, while the white ones are different. When you press the keys, these sprites will be displayed over the background image of the keyboard, which will always be visible.
If you want to draw your own great custom keys, go over this guide . Well, those that come with the library look like this:







Download these files and drop them into the project / resources / img / folder. If you use Xcode, drag them to the window to add to the project. As usual, working with graphics begins with adding file names to resource.h . At the same time, while you are there, remove the links to knob.png and background.png , and remove the files themselves from the project.

 // Unique IDs for each image resource. #define BG_ID 101 #define WHITE_KEY_ID 102 #define BLACK_KEY_ID 103 // Image resource locations for this plug. #define BG_FN "resources/img/bg.png" #define WHITE_KEY_FN "resources/img/whitekey.png" #define BLACK_KEY_FN "resources/img/blackkey.png" 

You will need a larger window size:

 // GUI default dimensions #define GUI_WIDTH 434 #define GUI_HEIGHT 66 

On Windows, to include .png files in an assembly, you also need to edit the Synthesis.rc header:

 #include "resource.h" BG_ID PNG BG_FN WHITE_KEY_ID PNG WHITE_KEY_FN BLACK_KEY_ID PNG BLACK_KEY_FN 

Now in the public section of the Synthesis.h file, add several members of the Synthesis class:

 public: // ... // Needed for the GUI keyboard: // Should return non-zero if one or more keys are playing. inline int GetNumKeys() const { return mMIDIReceiver.getNumKeys(); }; // Should return true if the specified key is playing. inline bool GetKeyStatus(int key) const { return mMIDIReceiver.getKeyStatus(key); }; static const int virtualKeyboardMinimumNoteNumber = 48; int lastVirtualKeyboardNoteNumber; 

In the initialization list in Synthesis.cpp you need to add lastVirtualKeyboardNoteNumber:

 Synthesis::Synthesis(IPlugInstanceInfo instanceInfo) : IPLUG_CTOR(kNumParams, kNumPrograms, instanceInfo), lastVirtualKeyboardNoteNumber(virtualKeyboardMinimumNoteNumber - 1) { // ... } 

When the source of the played notes is the host, they should appear as pressed on the keyboard of the plug-in. The keyboard will call getNumKeys and getKeyStatus to find out which keys are pressed. We have already implemented these functions in the MIDIReceiver last time, so we go further.

In the private section you also need to add a couple of lines:

 IControl* mVirtualKeyboard; void processVirtualKeyboard(); 

The IControl class is the base class for all GUI controls. We cannot declare an IkeyboardControl object IkeyboardControl , IkeyboardControl it is “unknown” to the .h files. Therefore, we will have to use pointers. IKeyboardControl.h has comments that say: “this header should be added (#include) after your plugin’s class is declared, so it’s best to add it to the main plug-in .cpp file”.
To clarify the situation, let's look at Synthesis.cpp . Add #include "IKeyboardControl.h" before #include resource.h .
Now change the constructor:

 Synthesis::Synthesis(IPlugInstanceInfo instanceInfo) : IPLUG_CTOR(kNumParams, kNumPrograms, instanceInfo), lastVirtualKeyboardNoteNumber(virtualKeyboardMinimumNoteNumber - 1) { TRACE; IGraphics* pGraphics = MakeGraphics(this, kWidth, kHeight); pGraphics->AttachBackground(BG_ID, BG_FN); IBitmap whiteKeyImage = pGraphics->LoadIBitmap(WHITE_KEY_ID, WHITE_KEY_FN, 6); IBitmap blackKeyImage = pGraphics->LoadIBitmap(BLACK_KEY_ID, BLACK_KEY_FN); // C# D# F# G# A# int keyCoordinates[12] = { 0, 7, 12, 20, 24, 36, 43, 48, 56, 60, 69, 72 }; mVirtualKeyboard = new IKeyboardControl(this, kKeybX, kKeybY, virtualKeyboardMinimumNoteNumber, /* octaves: */ 5, &whiteKeyImage, &blackKeyImage, keyCoordinates); pGraphics->AttachControl(mVirtualKeyboard); AttachGraphics(pGraphics); CreatePresets(); } 

Interesting things start when we attach a background image. First, we load the pressed black and white keys in the form of Ibitmap objects. The third argument of the LoadIBitmap ( 6 ) function tells the graphics system that there are six frames in whitekeys.png :

By default, pRegularKeys should contain 6 images (C / F, D, E / B, G, A, upper C), while pSharpKey contains only 1 image (for all flat / sharps).

- IKeyboardControl.h

The keyCoordinates array tells the system the offset of each key relative to the left border. This action needs to be done with only one octave, and IKeyboardControl calculate the offsets for all other octaves.
In the next line, we, roughly speaking, initialize the new IKeyboardControl new object and assign it the name mVirtualKeyboard . We pass a lot of information:

• Pointer to a plugin instance. This is an example of a delegation pattern : the virtual keyboard will call GetNumKeys and GetKeyStatus for this instance ( this );
• Keyboard coordinates in GUI;
• The lowest note number. When you click on the leftmost key, this particular note will be played;
• The number of octaves;
• Links to pictures of keystrokes;
• Relative X coordinate within one octave;

Interestingly, the virtual keyboard object does not even know about the existence of the bg.png file. He simply does not need him, everything will work this way. This is a plus, since the image of the keyboard can be part of the background image, and then you would have to cut this piece only to pass it on to the designer IkeyboardControl .

If you have programming experience in C ++, a conditioned reflex should arise: the constructor contains new , it means that you must write delete mVirtualKeyboard in the destructor. But if we do this, and then remove the plugin from the track, the runtime exception will pop up. The reason is that when a call is made

 pGraphics->AttachControl(mVirtualKeyboard); 

we transfer memory management to the graphics system, and managing this memory area is no longer our responsibility. Using delete we will try to detach an already free area of ​​memory.

Now remove the body of the CreatePresets function:

 void Synthesis::CreatePresets() { } 

And add kKeybX and kKeybY to ELayout:

 enum ELayout { kWidth = GUI_WIDTH, kHeight = GUI_HEIGHT, kKeybX = 1, kKeybY = 0 }; 

For performance reasons, IKeyboardControl does not redraw itself. This is a common practice in graphics programming: mark the GUI element as “dirty”, that is, its image will be updated only in the next cycle of redrawing. If you look at IKeyboardControl.h , in particular on OnMouseUp and OnMouseUp , you will see that mKey assigned a value and that the SetDirty function is SetDirty (as opposed to Draw ). SetDirty is a member function of the IControl class (the implementation of which can be found in IControl.cpp , respectively). It sets the value of the mDirty parameter to true . Each redraw cycle the graphics system redraws all GUI elements whose mDirty is true . I went into such details, as it is important to understand this aspect of the graphics system.

Reaction to external MIDI messages

So far, the keyboard becomes “dirty” when it is pressed. From mMIDIReceiver it receives data about mMIDIReceiver pressed, but it must also receive external MIDI data. mVirtualKeyboard and mMIDIReceiver do not know anything about each other, so let's edit ProcessMidiMsg in Synthesis.cpp :

 void Synthesis::ProcessMidiMsg(IMidiMsg* pMsg) { mMIDIReceiver.onMessageReceived(pMsg); mVirtualKeyboard->SetDirty(); } 

First, mMIDIReceiver updates the mLast... members mLast... according to the received MIDI data. Then mVirtualKeyboard is marked as dirty. Then in the next redraw cycle, Draw will be called for mVirtualKeyboard , which, in turn, will call GetNumKeys and GetKeyStatus . At first, this may seem crappy, but in reality it is a transparent, well-structured design that allows you to avoid redundancy and unnecessary movements.
Our virtual keyboard now responds to external MIDI messages and correctly draws keystrokes.

Reaction to pressing on the virtual keyboard

It remains to make the keyboard respond to clicking on the virtual keyboard built into the host, generate MIDI messages and send them to the recipient mMIDIReceiver .
Add this ProcessDoubleReplacing call immediately before the for loop:

 processVirtualKeyboard(); 

And write the appropriate function:

 void Synthesis::processVirtualKeyboard() { IKeyboardControl* virtualKeyboard = (IKeyboardControl*) mVirtualKeyboard; int virtualKeyboardNoteNumber = virtualKeyboard->GetKey() + virtualKeyboardMinimumNoteNumber; if(lastVirtualKeyboardNoteNumber >= virtualKeyboardMinimumNoteNumber && virtualKeyboardNoteNumber != lastVirtualKeyboardNoteNumber) { // The note number has changed from a valid key to something else (valid key or nothing). Release the valid key: IMidiMsg midiMessage; midiMessage.MakeNoteOffMsg(lastVirtualKeyboardNoteNumber, 0); mMIDIReceiver.onMessageReceived(&midiMessage); } if (virtualKeyboardNoteNumber >= virtualKeyboardMinimumNoteNumber && virtualKeyboardNoteNumber != lastVirtualKeyboardNoteNumber) { // A valid key is pressed that wasn't pressed the previous call. Send a "note on" message to the MIDI receiver: IMidiMsg midiMessage; midiMessage.MakeNoteOnMsg(virtualKeyboardNoteNumber, virtualKeyboard->GetVelocity(), 0); mMIDIReceiver.onMessageReceived(&midiMessage); } lastVirtualKeyboardNoteNumber = virtualKeyboardNoteNumber; } 

GetKey gives us the note number corresponding to the pressed key. IKeyboardControl does not support multitouch, so only one key can be pressed at a time. The first if releases the key, which is no longer pressed (if any). Since this function is called every mBlockSize samples, the second if ensures that only one note on message is generated for this click (and not every mBlockSize samples). We memorize the lastVirtualKeyboardNoteNumber value to avoid these “repeated clicks” each time the function is called.

Go!

We are ready to launch our sync again! If everything is done correctly, you can play on its keyboard. And the use of the virtual keyboard of the host or any other connected MIDI source should be displayed on the keyboard of the plug-in (in turn, showing the last key pressed). True, the sound will also correspond only to this one last key. We will deal with polyphony a little later.
You can still brag to your friends and play your beloved Beethoven on a classic synth sound. Only here is the sound of some kind of "wooden", and you hear clicks when you press and release the key. This is especially noticeable if a sine is generated. So you need to add envelopes. Let's do it in the next post.

Project files at this stage can be downloaded from here .

Original article:
martin-finke.de/blog/articles/audio-plugins-010-virtual-keyboard