New life old synthesizer. Part 2
Continuation of the story about the old burned synthesizer, into which I am trying to breathe new life by completely replacing the hardware responsible for sound generation with a software synthesizer built on the base of the EmbedSky E8 mini-computer with Linux on board. As is often the case, much more time passed between the publication of the first and second part of the article than planned, but, nevertheless, we will continue.
The previous part outlined the process of choosing a hardware platform for the new “brain” of the synthesizer with a description of the technical characteristics of the solution, briefly highlighted the process of assembling the necessary libraries and the problems encountered in the process. Now, with regard to hardware, we will see how the keyboard matrix of the synthesizer is arranged, and then there will be more details on the software part.
')
The keyboard matrix of the synthesizer is very similar to the usual keyboard matrix, which many fans of microcontrollers have probably already connected to their Arduino. For each synthesizer key, it provides from one (in the cheapest models) to two (in the bulk of the models) switches. Using two adjacent switches, one of which, when a key is pressed, closes a little before the other, the microcontroller can determine the conventional force, or rather the speed with which the key was pressed, so that the sound of the corresponding volume could be subsequently played. It looks like this:
On the reverse side of the board are placed diodes that prevent the “false” reading of keystrokes while simultaneously pressing several keys. Here is a fragment of the concept of the keyboard matrix, on which these two switches and the diodes connected to them are visible:
To scan the matrix, the microcontroller sequentially pulls the columns (pins labeled N) to power, and checks the level on the rows (pins labeled B). If the level of a row is high, the key corresponding to the currently active column-row combination is pressed. The diagram shows only a part of the keyboard - there are 76 keys on it (13 lines and 6 x 2 columns, which gives a total of 156 possible options when scanning the matrix and 25 used microcontroller pins). Scanning the entire keyboard is carried out several dozen times per second and unnoticed by the performer.
In my synthesizer, the microcontroller responsible for scanning the keyboard was originally an 8-bit, one-time programmable Hitachi HD63B05V0 microcontroller, operating at 8 MHz and having 4 KB of ROM and 192 bytes of RAM memory. Unfortunately, this controller turned out to be inoperable after the power incident described at the beginning of the first article. But, fortunately, it turned out to be almost pin-compatible with the ATmega162 controller I had, to which I replaced it, cutting and re-soldering only 2 tracks on the board, one of which was the RESET pin, which turned out to be in the wrong place like the HD63B05V0.
Since the inclusion of the controller did not allow me to use the built-in UART (since it was also on the other outputs), I used this one-way (write only) implementation of the serial port to display information about the keys pressed. Also, the TinySafeBoot bootloader, also using the serial port software implementation, was flooded into the microcontroller to enable future firmware updates. Since I chose Python + Qt5 as the language for the rapid development of all high-level synthesizer software, I also wrote a Python module for TinySafeBoot that allows you to read and write firmware to the AVR microcontroller. The AVR microcontroller itself is connected to the UART1 serial port on the EmbedSky E8 board and is powered by 3.3V to avoid the need for level conversion.
As a programmer for the AVR, I first used a Launchpad MSP430 programmer , of which I have several pieces, and then this self-made miracle (not bad working, by the way), gave way to the TL866CS MiniPro programmer from China. The sensations from the new programmer are extremely positive.
A very detailed description of the synthesizer's keyboard device and methods for scanning it, including one very original way of scanning via the AVR microcontroller interface for connecting an external RAM chip, is described on the OpenMusicLabs website
Partly to get more control over the scheduler and reduce latency when playing a sound, and partly from sports interest, I decided to use the PREEPMT RT kernel , one of the main features of which is that interruptions also become “processes” that can be preempted by the scheduler given priority. The original core, supplied by Samsung for the processor S5PV210, on the basis of which the system is built, is based on the kernel version 3.0.8, apparently from Android. None of the RT_PREEMPT patches available on the project site intended for this kernel version (3.0.8), did not want to overlap the source without conflicts, but in the end, by resolving all conflicts manually, we managed to patch the version 3.0.8-rt23.
Due to the fact that the basic structures such as spinlock and mutex were also modified in the kernel modified in this way, the proprietary drivers of some peripheral devices, such as a video camera, a capacitive touchscreen controller, and, most terrible, were ceased to be linked to it. audio codec. Let us return to them later, and now turn them off and try for the first time to start the board with a freshly assembled real-time kernel and ... get an instant kernel panic. It occurred even before running the kgdb debugger (which, as it turned out later, would not have worked anyway, even if it had started), so for debugging I had to insert printfs into the
As I mentioned above, one of the side effects was that the module responsible for audio I / O stopped linking to the new kernel. This was partly because the kernel with PREEMPT RT supports (in version 3.0.8) only the SLAB memory management mechanism, and the module was originally compiled with the SLUB mechanism turned on, which is not supported by the new kernel. However, I was lucky to work at Kaspersky Lab, and I persuaded a colleague to decompile the driver and codec files for me using the Hex-Rays decompiler for ARM, after which they managed to almost completely recreate their source code. Practically, because as a result, with the “new” driver, the audio interface was determined, however, due to some differences in the low-level register initialization procedure of the WM8960 chip, the sound was played with artifacts. For some time I tried to fix my driver, but then I chose an easier way - I sent a technical support to EmbedSky Tech, a Chinese company, where I bought a mini-computer, my PREEMPT_RT patch, and asked them to compile for me and send the audio driver files. The guys quickly responded and sent me files, with which the sound finally worked as expected.
By the way, while I was messing with my decompiled driver, I found that the kgdb debugger does not work with either my or the original kernel. As it turned out, his work requires the support of synchronous (polling) polling of the serial port, which was absent in the Samsung serial port driver (
We dig further. The second side effect of the new core was extremely low, with large “lags”, the speed of all four long-suffering serial ports of the system on the S5PV210 chip, as a result of which normal operation in the terminal through the serial port was impossible, and also did not work as it should be done by flashing the AVR controller, Keyboard Keyboard synthesizer. For a long time I tried to understand the reason, but noticed only that the input of each character in the terminal led to the generation of several million serial port interrupts - the kernel did not seem to be in a hurry to process them. In the end, I solved this problem by using the above-mentioned IRQF_NO_THREAD flag to make all interrupt on serial ports non-streaming. This decision was not very beautiful, because in addition to the Samsung driver had to make changes to the files
In the original core, which, as I said above, supports various peripheral devices, such as video cameras, hardware codecs, HDMI, etc., only about 390 MB of 512 MB of RAM were available and the rest was reserved for the above devices , and always (even if they were disabled during the kernel configuration process). It is very wasteful, especially considering that the extra 120 MB of RAM will not even interfere with the synthesizer for storing samples. Memory was reserved in the file
The last change that was made to the core concerned the minimum buffer size for audio data, which in the original was equal to one memory page, which at a sampling frequency of 44100 Hz, 2 channels of 16 bits each gave about 20 ms — a bit too much. This value was changed in the
Kernel source code with PREEMPT RT and all modifications on GitHub
The AVR is connected to the minicomputer's serial port, and spits out ready-made MIDI messages to its software UART. In order to rid himself of the need to write drivers, it was decided to use the JACK server as a transport for all audio and MIDI data. A small C application connects to the serial port, registers itself with JACK as MIDI-OUT and starts redirecting all received MIDI messages, and JACK already delivers them to LinuxSampler. Cheap and angry.
This solution also allows you to play MIDI files via JACK using
Thanks to the nefelim4ag comment to the previous post, I learned about the existence of libhybris, a library that allows using Android drivers in a regular Linux system. After some dances with tambourines, all the details of which I, unfortunately, no longer remember, I managed to get libhybris in my system and rebuild Qt 5 and PyQt5 with support for OpenGL ES 2.0, EGLFS and Qt Quick 2.0. Now my user interface is using Qt Quick andlooks in accordance with the latest fashion trends mowed under Android 4.0.
A small demo is only audio for now, since the synthesizer is now in a half-disassembled state. The video will be in the next post, which will most likely be born in August, after the board ordered in China arrives, bringing together all the parts of the synthesizer. In addition, the next post will most likely be devoted not to such a low-level manipulation of the kernel, but to the process of bringing to the mind the user part of the software on PyQt5 and QtQuick and, of course, demonstrating the resulting
If anyone is interested:
If you need to collect something from this list and have any problems, I will be happy to share my experience. In addition, much of what has been said here is true for another popular platform called FriendlyARM Tiny210, which is based on the same S5PV210 processor and maybe someone will need to use a real-time kernel with it.
The previous part outlined the process of choosing a hardware platform for the new “brain” of the synthesizer with a description of the technical characteristics of the solution, briefly highlighted the process of assembling the necessary libraries and the problems encountered in the process. Now, with regard to hardware, we will see how the keyboard matrix of the synthesizer is arranged, and then there will be more details on the software part.
')
Keyboard matrix
The keyboard matrix of the synthesizer is very similar to the usual keyboard matrix, which many fans of microcontrollers have probably already connected to their Arduino. For each synthesizer key, it provides from one (in the cheapest models) to two (in the bulk of the models) switches. Using two adjacent switches, one of which, when a key is pressed, closes a little before the other, the microcontroller can determine the conventional force, or rather the speed with which the key was pressed, so that the sound of the corresponding volume could be subsequently played. It looks like this:
On the reverse side of the board are placed diodes that prevent the “false” reading of keystrokes while simultaneously pressing several keys. Here is a fragment of the concept of the keyboard matrix, on which these two switches and the diodes connected to them are visible:
To scan the matrix, the microcontroller sequentially pulls the columns (pins labeled N) to power, and checks the level on the rows (pins labeled B). If the level of a row is high, the key corresponding to the currently active column-row combination is pressed. The diagram shows only a part of the keyboard - there are 76 keys on it (13 lines and 6 x 2 columns, which gives a total of 156 possible options when scanning the matrix and 25 used microcontroller pins). Scanning the entire keyboard is carried out several dozen times per second and unnoticed by the performer.
In my synthesizer, the microcontroller responsible for scanning the keyboard was originally an 8-bit, one-time programmable Hitachi HD63B05V0 microcontroller, operating at 8 MHz and having 4 KB of ROM and 192 bytes of RAM memory. Unfortunately, this controller turned out to be inoperable after the power incident described at the beginning of the first article. But, fortunately, it turned out to be almost pin-compatible with the ATmega162 controller I had, to which I replaced it, cutting and re-soldering only 2 tracks on the board, one of which was the RESET pin, which turned out to be in the wrong place like the HD63B05V0.
Since the inclusion of the controller did not allow me to use the built-in UART (since it was also on the other outputs), I used this one-way (write only) implementation of the serial port to display information about the keys pressed. Also, the TinySafeBoot bootloader, also using the serial port software implementation, was flooded into the microcontroller to enable future firmware updates. Since I chose Python + Qt5 as the language for the rapid development of all high-level synthesizer software, I also wrote a Python module for TinySafeBoot that allows you to read and write firmware to the AVR microcontroller. The AVR microcontroller itself is connected to the UART1 serial port on the EmbedSky E8 board and is powered by 3.3V to avoid the need for level conversion.
Source code for AVR firmware
#include <avr/io.h> #include <avr/interrupt.h> #include <util/delay.h> #include <string.h> #include "dbg_putchar.h" #define MIDI_BASE 18 #define ZERO_BASE 28 #define KEYS_COUNT 76 #define hiz(port, dir) do { \ (dir) = 0; \ (port) = 0; \ } while(0) #define alow(port, dir) do { \ (dir) = 0xff; \ (port) = 0; \ } while(0) uint8_t keys[KEYS_COUNT]; /* Get state of a row by its index * starting from 1 to 13 */ uint8_t getRow(uint8_t idx) { if (idx <= 8) { return (PINC & (1 << (8 - idx))); } else if (idx >= 9 && idx <= 11) { return (PINE & (1 << (11 - idx))); } else if (idx == 12) { return (PINA & (1 << PIN6)); } else if (idx == 13) { return (PINA & (1 << PIN4)); } return 0; } inline void activateColumn1(uint8_t idx) { PORTD = 0x00 | (1 << (8 - idx)); PORTB = 0x00; } void activateColumn2(uint8_t idx) { if (idx <= 3) { PORTB = 0x00 | (1 << (idx + 4)); PORTD = 0x00; } else if (idx == 4) { PORTB = 0x00 | (1 << PIN4); PORTD = 0x00; } else if (idx == 5 || idx == 6) { PORTD = 0x00 | (1 << (idx - 5)); PORTB = 0x00; } } inline void deactivateColumns(void) { PORTD = 0x00; PORTB = 0x00; } inline void initPorts(void) { hiz(PORTA, DDRA); hiz(PORTC, DDRC); hiz(PORTE, DDRE); PORTB = 0x00; DDRB = 0xfe; DDRD = 0xff; } void resetRows(void) { /* output low */ alow(PORTC, DDRC); alow(PORTE, DDRE); /* don't touch PA7 & PA5 */ DDRA |= 0x5f; PORTA &= ~0x5f; _delay_us(10); /* back to floating input */ hiz(PORTC, DDRC); hiz(PORTE, DDRE); DDRA &= ~0x5f; } /* base MIDI note number is 25: C#0 */ int main(void) { uint8_t row, col, layer; uint8_t note, offset; initPorts(); memset(keys, 0, sizeof(keys)); dbg_tx_init(); dbg_putchar('O'); dbg_putchar('K'); while(1) { for (layer = 0; layer < 2; layer++) { for (col = 1; col <= 6; col++) { if (!layer) activateColumn1(col); else activateColumn2(col); for (row = 1; row <= 13; row++) { note = 6 * row + col + MIDI_BASE; offset = note - ZERO_BASE; if (getRow(row)) { if (!layer) { /* increase velocity counter */ if (keys[offset] < 254 && !(keys[offset] & 0x80)) keys[offset]++; } else { if (!(keys[offset] & 0x80)) { /* generate note-on event */ dbg_putchar(0x90); dbg_putchar(note); /*dbg_putchar(keys[offset]);*/ dbg_putchar(0x7f); /* stop counting */ keys[offset] |= 0x80; } } } else { if (layer) continue; if (keys[offset] & 0x80) { /* generate note off event */ dbg_putchar(0x90); dbg_putchar(note); dbg_putchar(0x00); /* reset key state */ keys[offset] = 0x00; } } } deactivateColumns(); resetRows(); } } } return 0; } Python module for TinySafeBoot
import serial import binascii import struct import intelhex import sys class TSB(object): CONFIRM = '!' REQUEST = '?' def __init__(self, port): self.port = serial.Serial(port, baudrate=9600, timeout=1) self.flashsz = 0 def check(self): if not self.flashsz: raise Exception("Not activated") def activate(self): self.port.write("@@@") (self.tsb, self.version, self.status, self.sign, self.pagesz, self.flashsz, self.eepsz) = \ struct.unpack("<3sHB3sBHH", self.port.read(14)) self.port.read(2) self.pagesz *= 2 self.flashsz *= 2 self.eepsz += 1 assert(self.port.read() == self.CONFIRM) def rflash(self, progress=None, size=0): self.check() self.port.write("f") self.addr = 0 self.flash = "" size = self.flashsz if not size else size while self.addr < size: if progress is not None: progress("read", self.addr, size) self.port.write(self.CONFIRM) page = self.port.read(self.pagesz) if len(page) != self.pagesz: raise Exception("Received page too short: %d" % len(page)) self.addr += len(page) self.flash += page return self.flash.rstrip('\xff') def wflash(self, data, progress=None): if len(data) % self.pagesz != 0: data = data + "\xff" * (self.pagesz - (len(data) % self.pagesz)) assert(len(data) % self.pagesz == 0) self.check() self.port.write("F") self.addr = 0 assert(self.port.read() == self.REQUEST) while self.addr < len(data): if progress is not None: progress("write", self.addr, len(data)) self.port.write(self.CONFIRM) self.port.write(data[self.addr:self.addr + self.pagesz]) self.addr += self.pagesz assert(self.port.read() == self.REQUEST) self.port.write(self.REQUEST) return self.port.read() == self.CONFIRM def vflash(self, data, progress=None): fw = self.rflash(progress, len(data)) return fw == data def info(self): print "Tiny Safe Bootloader: %s" % self.tsb print "Page size: %d" % self.pagesz print "Flash size: %d" % self.flashsz print "EEPROM size: %d" % self.eepsz if __name__ == "__main__": import argparse def progress(op, addr, total): sys.stdout.write("\r%s address: $%0.4x/$%0.4x" % (op, addr, total)) sys.stdout.flush() parser = argparse.ArgumentParser() parser.add_argument("filename", help="firmware file in Intel HEX format") parser.add_argument("--device", help="Serial port to use for programming", default="/dev/ttyUSB0") args = parser.parse_args() tsb = TSB(args.device) tsb.activate() tsb.info() fw = intelhex.IntelHex(args.filename) assert(tsb.wflash(fw.tobinstr(), progress)) assert(tsb.vflash(fw.tobinstr(), progress)) print "\nOK\n" As a programmer for the AVR, I first used a Launchpad MSP430 programmer , of which I have several pieces, and then this self-made miracle (not bad working, by the way), gave way to the TL866CS MiniPro programmer from China. The sensations from the new programmer are extremely positive.
A very detailed description of the synthesizer's keyboard device and methods for scanning it, including one very original way of scanning via the AVR microcontroller interface for connecting an external RAM chip, is described on the OpenMusicLabs website
Realtime Preemption Core Cooking
Partly to get more control over the scheduler and reduce latency when playing a sound, and partly from sports interest, I decided to use the PREEPMT RT kernel , one of the main features of which is that interruptions also become “processes” that can be preempted by the scheduler given priority. The original core, supplied by Samsung for the processor S5PV210, on the basis of which the system is built, is based on the kernel version 3.0.8, apparently from Android. None of the RT_PREEMPT patches available on the project site intended for this kernel version (3.0.8), did not want to overlap the source without conflicts, but in the end, by resolving all conflicts manually, we managed to patch the version 3.0.8-rt23.
Due to the fact that the basic structures such as spinlock and mutex were also modified in the kernel modified in this way, the proprietary drivers of some peripheral devices, such as a video camera, a capacitive touchscreen controller, and, most terrible, were ceased to be linked to it. audio codec. Let us return to them later, and now turn them off and try for the first time to start the board with a freshly assembled real-time kernel and ... get an instant kernel panic. It occurred even before running the kgdb debugger (which, as it turned out later, would not have worked anyway, even if it had started), so for debugging I had to insert printfs into the
init/main.c file, the start_kernel function, to determine the location, in which everything collapses. Thus, it turned out that the last thing the kernel hrtimers_init() do was call the hrtimers_init() function, which initializes the high-resolution timers and interrupts them. This code depends on the specific platform, and in our case is in arch/arm/plat-s5p/hr-time-rtc.c . As I said, one of the main features of the kernel with the PREEMPT RT patch is that interrupts become threads. This is also possible in a regular kernel, but the kernel with PREEMPT RT by default tries to do almost all interrupts. Further analysis of the code showed that the kthreadd_task task is used for these threads, which is initialized at the very end of the start_kernel function — much later than the initialization of the timers occurs. The fall occurred due to the fact that the kernel interrupted the timer interrupt and made it threaded, while kthreadd_task was still NULL. This is solved by setting for individual interrupts that should not be streamed under any circumstances, the IRQF_NO_THREAD flag which was added to the timer interrupt flags in hr-time-rtc.c . Hooray! The kernel has booted, but this is just the beginning ...As I mentioned above, one of the side effects was that the module responsible for audio I / O stopped linking to the new kernel. This was partly because the kernel with PREEMPT RT supports (in version 3.0.8) only the SLAB memory management mechanism, and the module was originally compiled with the SLUB mechanism turned on, which is not supported by the new kernel. However, I was lucky to work at Kaspersky Lab, and I persuaded a colleague to decompile the driver and codec files for me using the Hex-Rays decompiler for ARM, after which they managed to almost completely recreate their source code. Practically, because as a result, with the “new” driver, the audio interface was determined, however, due to some differences in the low-level register initialization procedure of the WM8960 chip, the sound was played with artifacts. For some time I tried to fix my driver, but then I chose an easier way - I sent a technical support to EmbedSky Tech, a Chinese company, where I bought a mini-computer, my PREEMPT_RT patch, and asked them to compile for me and send the audio driver files. The guys quickly responded and sent me files, with which the sound finally worked as expected.
By the way, while I was messing with my decompiled driver, I found that the kgdb debugger does not work with either my or the original kernel. As it turned out, his work requires the support of synchronous (polling) polling of the serial port, which was absent in the Samsung serial port driver (
drivers/tty/serial/samsung.c ). I added the required support to the driver based on this patch, after which the debugger started working.We dig further. The second side effect of the new core was extremely low, with large “lags”, the speed of all four long-suffering serial ports of the system on the S5PV210 chip, as a result of which normal operation in the terminal through the serial port was impossible, and also did not work as it should be done by flashing the AVR controller, Keyboard Keyboard synthesizer. For a long time I tried to understand the reason, but noticed only that the input of each character in the terminal led to the generation of several million serial port interrupts - the kernel did not seem to be in a hurry to process them. In the end, I solved this problem by using the above-mentioned IRQF_NO_THREAD flag to make all interrupt on serial ports non-streaming. This decision was not very beautiful, because in addition to the Samsung driver had to make changes to the files
serial_core.c and serial_core.h , affecting in general all serial ports. Because in the PREEMPT RT kernel, you cannot use spin_lock_t in drivers that are NO_THREAD, but you must use raw_spinlock_t.In the original core, which, as I said above, supports various peripheral devices, such as video cameras, hardware codecs, HDMI, etc., only about 390 MB of 512 MB of RAM were available and the rest was reserved for the above devices , and always (even if they were disabled during the kernel configuration process). It is very wasteful, especially considering that the extra 120 MB of RAM will not even interfere with the synthesizer for storing samples. Memory was reserved in the file
arch/arm/mach-s5pv210/mach-tq210.c , which is the main collection point for all the information about the configuration and devices of a particular machine (in our case, the boards). We comment on the allocation of memory - a call to the function s5p_reserve_bootmem , and we get 120 MB of additional memory for the synthesizer to work.The last change that was made to the core concerned the minimum buffer size for audio data, which in the original was equal to one memory page, which at a sampling frequency of 44100 Hz, 2 channels of 16 bits each gave about 20 ms — a bit too much. This value was changed in the
sound/soc/samsung/dma.c to 128 bytes, after which the minimum buffer size was reduced to a few milliseconds without compromising stability and performance.Kernel source code with PREEMPT RT and all modifications on GitHub
How does the AVR microcontroller communicate with LinuxSampler
The AVR is connected to the minicomputer's serial port, and spits out ready-made MIDI messages to its software UART. In order to rid himself of the need to write drivers, it was decided to use the JACK server as a transport for all audio and MIDI data. A small C application connects to the serial port, registers itself with JACK as MIDI-OUT and starts redirecting all received MIDI messages, and JACK already delivers them to LinuxSampler. Cheap and angry.
Source code of the bridge application between the serial port and JACK
#include <stdio.h> #include <stdlib.h> #include <sys/types.h> #include <sys/time.h> #include <unistd.h> #include <assert.h> #include <string.h> #include <sysexits.h> #include <errno.h> #include <signal.h> #include <fcntl.h> #include <termios.h> #include <jack/jack.h> #include <jack/midiport.h> #define UART_SPEED B9600 jack_port_t *output_port; jack_client_t *jack_client = NULL; int input_fd; void init_serial(int fd) { struct termios termios; int res; res = tcgetattr (fd, &termios); if (res < 0) { fprintf (stderr, "Termios get error: %s\n", strerror(errno)); exit (EXIT_FAILURE); } cfsetispeed (&termios, UART_SPEED); cfsetospeed (&termios, UART_SPEED); termios.c_iflag &= ~(IGNPAR | IXON | IXOFF); termios.c_iflag |= IGNPAR; termios.c_cflag &= ~(CSIZE | PARENB | CSTOPB | CREAD | CLOCAL); termios.c_cflag |= CS8; termios.c_cflag |= CREAD; termios.c_cflag |= CLOCAL; termios.c_lflag &= ~(ICANON | ECHO); termios.c_cc[VMIN] = 3; termios.c_cc[VTIME] = 0; res = tcsetattr (fd, TCSANOW, &termios); if (res < 0) { fprintf (stderr, "Termios set error: %s\n", strerror(errno)); exit (EXIT_FAILURE); } } double get_time(void) { double seconds; int ret; struct timeval tv; ret = gettimeofday(&tv, NULL); if (ret) { perror("gettimeofday"); exit(EX_OSERR); } seconds = tv.tv_sec + tv.tv_usec / 1000000.0; return seconds; } double get_delta_time(void) { static double previously = -1.0; double now; double delta; now = get_time(); if (previously == -1.0) { previously = now; return 0; } delta = now - previously; previously = now; assert(delta >= 0.0); return delta; } static double nframes_to_ms(jack_nframes_t nframes) { jack_nframes_t sr; sr = jack_get_sample_rate(jack_client); assert(sr > 0); return (nframes * 1000.0) / (double)sr; } static double nframes_to_seconds(jack_nframes_t nframes) { return nframes_to_ms(nframes) / 1000.0; } static jack_nframes_t ms_to_nframes(double ms) { jack_nframes_t sr; sr = jack_get_sample_rate(jack_client); assert(sr > 0); return ((double)sr * ms) / 1000.0; } static jack_nframes_t seconds_to_nframes(double seconds) { return ms_to_nframes(seconds * 1000.0); } static void process_midi_output(jack_nframes_t nframes) { int t, res; void *port_buffer; char midi_buffer[3]; jack_nframes_t last_frame_time; port_buffer = jack_port_get_buffer(output_port, nframes); if (port_buffer == NULL) { printf("jack_port_get_buffer failed, cannot send anything.\n"); return; } jack_midi_clear_buffer(port_buffer); last_frame_time = jack_last_frame_time(jack_client); t = seconds_to_nframes(get_delta_time()); res = read(input_fd, midi_buffer, sizeof(midi_buffer)); if (res < 0 && errno == EAGAIN) return; res = jack_midi_event_write(port_buffer, t, midi_buffer, 3); if (res != 0) { printf("jack_midi_event_write failed, NOTE LOST."); } } static int process_callback(jack_nframes_t nframes, void *notused) { if (nframes <= 0) { printf("Process callback called with nframes = 0; bug in JACK?"); return 0; } process_midi_output(nframes); return 0; } int connect_to_input_port(const char *port) { int ret; ret = jack_port_disconnect(jack_client, output_port); if (ret) { printf("Cannot disconnect MIDI port."); return -3; } ret = jack_connect(jack_client, jack_port_name(output_port), port); if (ret) { printf("Cannot connect to %s.", port); return -4; } printf("Connected to %s.", port); return 0; } static void init_jack(void) { int i, err; jack_client = jack_client_open("midibridge", JackNullOption, NULL); if (jack_client == NULL) { printf("Could not connect to the JACK server; run jackd first?"); exit(EXIT_FAILURE); } err = jack_set_process_callback(jack_client, process_callback, 0); if (err) { printf("Could not register JACK process callback."); exit(EXIT_FAILURE); } char port_name[32]; snprintf(port_name, sizeof(port_name), "midi_out"); output_port = jack_port_register(jack_client, port_name, JACK_DEFAULT_MIDI_TYPE, JackPortIsOutput, 0); if (output_port == NULL) { printf("Could not register JACK output port '%s'.", port_name); exit(EXIT_FAILURE); } if (jack_activate(jack_client)) { printf("Cannot activate JACK client."); exit(EXIT_FAILURE); } } static void usage(void) { fprintf(stderr, "usage: midibridge -a <input port>\n"); exit(EXIT_FAILURE); } int main(int argc, char *argv[]) { int ch; char *autoconnect_port_name = NULL; while ((ch = getopt(argc, argv, "a:")) != -1) { switch (ch) { case 'a': autoconnect_port_name = strdup(optarg); break; default: usage(); } } input_fd = open("/dev/ttySAC1", O_RDWR | O_NOCTTY | O_NDELAY | O_NONBLOCK); if (input_fd < 0) { fprintf(stderr, "Cannot open serial port %s\n", strerror(errno)); return EXIT_FAILURE; } init_serial (input_fd); init_jack(); if (autoconnect_port_name) { if (connect_to_input_port(autoconnect_port_name)) { printf("Couldn't connect to '%s', exiting.", autoconnect_port_name); exit(EXIT_FAILURE); } } getc(stdin); return 0; } This solution also allows you to play MIDI files via JACK using
jack-smf-player , which I compiled for ARM and WAV / MP3 via mplayer with support for audio output in JACK.Bonus
Thanks to the nefelim4ag comment to the previous post, I learned about the existence of libhybris, a library that allows using Android drivers in a regular Linux system. After some dances with tambourines, all the details of which I, unfortunately, no longer remember, I managed to get libhybris in my system and rebuild Qt 5 and PyQt5 with support for OpenGL ES 2.0, EGLFS and Qt Quick 2.0. Now my user interface is using Qt Quick and
At last
A small demo is only audio for now, since the synthesizer is now in a half-disassembled state. The video will be in the next post, which will most likely be born in August, after the board ordered in China arrives, bringing together all the parts of the synthesizer. In addition, the next post will most likely be devoted not to such a low-level manipulation of the kernel, but to the process of bringing to the mind the user part of the software on PyQt5 and QtQuick and, of course, demonstrating the resulting
If anyone is interested:
List of all software that was cross-compiled for ARM
- alsa-lib-1.0.27.2
- alsa-utils-1.0.27.2
- libaudiofile-0.3.6
- dbus-1.8.0
- dropbear-2014.63
- fftw-3.3.3
- fluidsynth-1.1.6
- fontconfig-2.11.0
- freetype-2.5.3
- glib-2.34.3
- libicu-52.1
- jack-audio-connection-kit-0.121.3
- jack-smf-utils-1.0
- libffi-3.0.13
- libgig-3.3.0
- libgig-svn
- libhybris
- libsamplerate-0.1.8
- libsndfile-1.0.25
- linuxsampler-1.0.0
- linuxsampler-svn
- mplayer SVN-r36900-4.4.6
- openssl-1.0.0l
- psutil-1.2.1
- pyjack-0.5.2
- PyQt-gpl-5.2
- pyserial-2.7
- Python-2.7.6
- strace-4.8
- tslib-1.4.1
If you need to collect something from this list and have any problems, I will be happy to share my experience. In addition, much of what has been said here is true for another popular platform called FriendlyARM Tiny210, which is based on the same S5PV210 processor and maybe someone will need to use a real-time kernel with it.
Source: https://habr.com/ru/post/224987/
All Articles