It's hard to underestimate the role of GPIO , especially in the world of embedded ARM systems. In addition to being an extremely popular material for all beginner manuals, GPIO provides a way to manage many peripherals, act as a source of valuable interruptions, or may even be the only available way to communicate with the world for SOC.

Based on my own humble experience, I can say that interruption is not the most consecrated topic in the Linux community. Because of its features, as well as a strong binding to the hardware, all training materials on interrupts are devoid of a real and easily reproducible example. This fact prevents understanding that very often interrupts and GPIO are inseparable, especially in the field of embedded Linux. Many people start to believe that GPIO is a very simple and boring thing (which, by the way, has become such thanks to the sysfs subsystem).

')
Even in the example given in LDD3 (snull driver), interrupts are emitted by explicitly calling the pair device function. There are also examples in USFCA courses ( http://cs.usfca.edu/~cruse/cs686s08/ ), but they use someone else's interrupt, are closely related to x86 architecture and are very outdated.

The proposed solution can solve these problems. From the point of view of user space and, in many respects, in the internal implementation, the driver is indistinguishable from most of the “real” ones that provide interrupts for general purpose I / O ports. At the moment, the driver supports interrupts on the leading or trailing edge and can be used as a source of interruptions for other devices.

ivshmem - Inter-VM shared memory

Designed for sharing shared memory (allocated on the host platform through the POSIX shared memory API mechanism) by multiple QEMU processes with different guest platforms. In order for all guest platforms to have access to the shared memory area, ivshmem models a PCI device by providing memory access as a PCI BAR.



From the point of view of the virtual machine, the ivshmem PCI device contains three basic address registers (BAR).

• BAR0 is a MMIO domain that supports registers and interrupts in case MSI is not used, one kilobyte in size.
• BAR1 is used for MSI-X if MSI support is enabled.
• BAR2 to access the shared memory object.

This mechanism was introduced by Cam Macdonnel in the original report “Nahanni - a shared memory interface for KVM” (later became known as ivshmem), in which he put forward the following theses:

• zero-copy data access
• interrupt mechanism
• guest / guest and host / guest interaction

and analyzed the overall performance.

At the moment, officially, nobody escorts ivshmem, however, Red Hat employees are making a big contribution to the development of ivshmem.

purpose

ivshmem can serve as the basis for simulating and debugging many classes of devices.
In this article, we look at the general-purpose input / output (GPIO) virtual pci, which is also a source of interrupts, and the corresponding driver with access and control through the sysfs mechanism.

Prerequisites:

• Qemu 2.5.1.1 source code (not recommended for a lower version)
• Linux-kernel 4.1 source code

For development and testing, the virtual board qemu versatilepb (system ARM) was used.

Optional:

• arm-cross-toolchain
• nairobi-embedded - Guest-side ivshmem PCI device test sources

Legend:

g >> - commands or output executed on the guest system.
h >> on the main one.

Sample and original code

To begin with, we will demonstrate the original code based on the original code ( https://github.com/henning-schild/ivshmem-guest-code ), and modified, subsequently, Siro Mugabi.

h>> qemu: += -device ivshmem,shm=ivshmem,size=1 g>> # insmod ne_ivshmem_ldd_basic.ko ivshmem 0000:00:0d.0: data_mmio iomap base = 0xc8c00000 ivshmem 0000:00:0d.0: data_mmio_start = 0x60000000 data_mmio_len = 1048576 ivshmem 0000:00:0d.0: regs iomap base = 0xc88ee400, irq = 27 ivshmem 0000:00:0d.0: regs_addr_start = 0x50002400 regs_len = 256 g>> # ./ne_ivshmem_shm_guest_usr -w "TEST STRING" h>> $ xxd -l 16 /dev/shm/ivshmem 0000000: 5445535420535452 494e 4700 0000 0000 TEST STRING..... 

In principle, this is quite enough to emulate GPIO already in this form. And in many cases they did this when a simple input or output state is enough, using sysfs and interrupts implies a small add-on to I / O mem.

Implementation

Note that / dev / ivshmem0 and ne_ivshmem_shm_guest_usr.c are no longer needed, all work from the guest machine from the user space (user-space) will be handled using the sysfs interface.

Before marking our device in memory, I would like to note that we simply duplicate the scheme used in most gpio drivers.

Firstly, all gpio inputs / outputs are divided into ports, usually 8, 16, 32 inputs. Each port has at least an input status register ( GPIO_DATA ), a direction register if in / out switching is supported ( GPIO_OUTPUT ). Further (if there is support in the device itself), the interrupt status register, the interrupt registers on the rising edge (rising) and the falling edge and on the level (high and low). The hardware interrupt supplied by the main interrupt controller is usually one for the entire port and is shared between all the inputs of the port.

Examples of existing implementations with comments

Sitara am335x

better known in the beaglebone board

Developer: Texas Instruments
Documentation: AM335x Sitara Processors Technical Reference Manual (page 4865)
Its corresponding gpio driver is: linux / drivers / gpio / gpio-omap.c
Corresponding title: linux / include / linux / platform_data / gpio-omap.h
Number of inputs / outputs: 128 (4 gpio ports - 32 contacts each)



Note: GPIO_IRQSTATUS_N is also used for IRQ ACK. Managing bounce, as well as nutrition is beyond the scope of this article.

The presence of the GPIO_CLEARDATAOUT and GPIO_SETDATAOUT registers in addition to the GPIO_DATAOUT register, as well as GPIO_IRQSTATUS_SET_N and GPIO_IRQSTATUS_CLR_N in addition to GPIO_IRQSTATUS_N, is explained by two ways of recording the output status:

• Standard: Read register entry completely at main address
• Task and cleaning (recommended by the manufacturer): Two corresponding registers are used to set and clear the corresponding contact as an output, the same applies to interrupt control.
  
  

ep9301

Developer: Cirrus Logic
Documentation: EP9301 User's Guide (page 523)
Its corresponding gpio driver is: linux / drivers / gpio / gpio-ep93xx.c
Corresponding header: linux / arch / arm / mach-ep93xx / include / mach / gpio-ep93xx.h
Number of inputs / outputs: 56 (7 gpio ports - 8 contacts each)



Note:
Of these, 7 ports are available for 8, 8, 1, 2, 3, 2, 4 inputs / outputs, with only the first, second and fifth ports having interrupt registers.
The table only considers port A.
One of the features of ep9301, is that the type of interrupts both at the hardware level is not supported; the driver switches to the moment the interrupt is triggered. Another interesting feature is that on port F each contact has its own interruption.

Bt848

Last example: pci board Bt848, with gpio.

Developer: Intel
Documentation: Bt848 / 848A / 849A (page 68)
The corresponding gpio driver is: linux / drivers / gpio / gpio-bt8xx.c
Corresponding header: linux / drivers / media / pci / bt8xx / bt848.h
Number of inputs / outputs: 24

Bt848 is a video capture card.



There is no support for interrupts. Only two registers - the state and setting in / out.

We mark our device in memory

To begin, select a place for data and state management.

Let the device have 8 general purpose inputs / outputs, then:

Register Name	Bias	Driver Name	Description
DATA	0x00	VIRTUAL_GPIO_DATA	The status of the inputs / outputs is readable and writeable.
OUTPUTEN	0x01	VIRTUAL_GPIO_OUT_EN	Monitors the input / output status (in / out)

Gpio interface quick reference

 struct gpio_chip { /*   gpio */ const char *label; /*     */ int (*direction_input)(struct gpio_chip *chip, unsigned offset); /*   */ int (*get)(struct gpio_chip *chip, unsigned offset); /*     */ int (*direction_output)(struct gpio_chip *chip, unsigned offset, int value); /*   */ void (*set)(struct gpio_chip *chip, unsigned offset, int value); /*      ,       -1 */ int base; /*   */ u16 ngpio; }; 

Documentation:
https://www.kernel.org/doc/Documentation/gpio/sysfs.txt

Link to source code:
linux-kernel 4.1

Output state when switching

It is necessary to note the int value parameter in the direction_output function, which serves the file / sys / class / gpio / gpioN / direction, which takes a value not only “in” / ”out”, but also “high” / “low”, the values ​​of which are transmitted as the value parameter ( this simple fact, for some reason, is rarely mentioned in the beginner's guides ).

 g>> /sys/class/gpio # echo low > gpio0/direction g>> /sys/class/gpio # cat gpio0/value 0 g>> /sys/class/gpio # echo high > gpio0/direction g>> /sys/class/gpio # cat gpio0/value 1 

Dynamic assignment int base and legacy ARCH_NR_GPIOS

Historically, the number of GPIOs in the kernel was limited to the ARCH_NR_GPIOS parameter, defaulting to 256 and subsequently increased to 512 ( version 3.18 ).

Its meaning is quite simple, in the kernel there can be no more GPIO than the parameter value, if the planned quantity was greater than the default value, it was redefined in the corresponding platform header file.

The reason for this behavior was the definition of the GPIO description table as static and the maximum offset for each port was limited:

 static struct gpio_desc gpio_desc[ARCH_NR_GPIOS]; 

GPIO ports and their offsets were rigidly defined in files describing the hardware of a particular SOC, for example:


Starting from version 3.19, the static array was replaced with a dynamic array for each GPIO port allocated in the gpiochip_add () function .

However, ARCH_NR_GPIOS is still here (at the time of version 4.7) and is used to search for an offset when dynamically assigning a base.

 /* dynamic allocation of GPIOs, eg on a hotplugged device */ static int gpiochip_find_base(int ngpio); 

The base parameter of the gpio_chip structure can be defined as -1, then the offset will be defined as the first free range starting from the end, that is, if the port has the number of contacts equal to 8, the offset will be 248 with the ARCH_NR_GPIOS parameter equal to 256 ( ARCH_NR_GPIOS - ngpio) if the port is logged in first.

Define the following features of our driver

Set the appropriate contact as input:


Reading current contact status:


Set corresponding contact as output:


Set exit status:


The function of registering our driver as a gpio_chip device:


vgread and vgwrite are just wrappers for the iowrite8 and ioread8 functions:

 #define vgwrite(dat, adr) iowrite8((dat), vg->data_base_addr+(adr)) #define vgread(adr) ioread8(vg->data_base_addr+(adr)) 

Passing the gpiobase value as a parameter when dynamically loading a module

Note: Starting from version 4.2, this is the recommended way to register a GPIO port.

 static int modparam_gpiobase = -1; /* dynamic */ module_param_named(gpiobase, modparam_gpiobase, int, 0444); MODULE_PARM_DESC(gpiobase, "The GPIO base number. -1 means dynamic, which is the default."); 

Loading and testing module

 h>> $ rm /dev/shm/ivshmem h>> Adding parameters to qemu launch command line += -device ivshmem,shm=ivshmem,size=1 g>> # ls /sys/class/gpio/ export unexport g>> # insmod virtual_gpio_basic.ko PCI: enabling device 0000:00:0d.0 (0100 -> 0102) ivshmem_gpio 0000:00:0d.0: data_mmio iomap base = 0xc8a00000 ivshmem_gpio 0000:00:0d.0: data_mmio_start = 0x60000000 data_mmio_len = 1048576 ivshmem_gpio 0000:00:0d.0: regs iomap base = 0xc88e6400, irq = 27 ivshmem_gpio 0000:00:0d.0: regs_addr_start = 0x50002400 regs_len = 256 g>> # ls /sys/class/gpio/ export gpiochip248 unexport g>> # cat /sys/class/gpio/gpiochip248/label 0000:00:0d.0 g>> # cat /sys/class/gpio/gpiochip248/base 248 g>> # cat /sys/class/gpio/gpiochip248/ngpio 8 g>> # rmmod virtual_gpio_basic Unregister virtual_gpio device. g>> # insmod virtual_gpio_basic.ko gpiobase=0 g>> # ls /sys/class/gpio/ export gpiochip0 unexport g>> # echo 0 > /sys/class/gpio/export g>> # echo high > /sys/class/gpio/gpio0/direction 

Simple check:

 h>> $ xxd -b -l 2 -c 2 /dev/shm/ivshmem 0000000: 00000001 00000001 .. 

DATA is set, OUTPUTEN is set.

Add interrupts

Tagging interrupt registers and basic interrupt handling

Note: The virtual driver only considers EDGEDETECT_RISE and EDGEDETECT_FALL.

Note: Please use only qemu versions older than 2.5.0 or qemu-linaro. Idshmem interrupt support is broken in 2.5.0 or simply does not work in some versions earlier than 2.5.0. If using 2.5.0 you must use a patch for 2.5.0 ( http://lists.gnu.org/archive/html/qemu-stable/2015-12/msg00034.html ).

Add the following registers:

Register Name	Bias	Driver Name	Description
INTERRUPT_EN	0x01	VIRTUAL_GPIO_INT_EN	Enable interrupts on a given input.
INTERRUPT_ST	0x02	VIRTUAL_GPIO_INT_ST	Interrupt Status Register
INTERRUPT_EOI	0x03	VIRTUAL_GPIO_INT_EOI	Register for notification of a processed interrupt
EDGEDETECT_RISE	0x04	VIRTUAL_GPIO_RISING	Enable / Disable Interrupts for Front-End Entry
EDGEDETECT_FALL	0x05	VIRTUAL_GPIO_FALLING	Enable / disable interrupts for trailing edge input
LEVELDETECT_HIGH		NC	NOT CONNECTED
LEVELDETECT_LOW		NC	NOT CONNECTED

The following function is responsible for handling the pci bus interrupt, at the moment its role is only in the notification of the processed interrupt:


This stage will require an external daemon, which is included in the standard qemu-ivshmem-server delivery. The -chardev path to the UNIX socket is added to the qemu startup line, messages exchanged between running qemu, ivshmem-server and ivshmem-client exchanges are implemented using the eventfd mechanism.

 h>> $ ivshmem-server -v -F -p ivshmem.pid -l 1M #  qemu    h>> $ += -chardev socket,path=/tmp/ivshmem_socket,id=ivshmemid -device ivshmem,chardev=ivshmemid,size=1,msi=off g>> # echo 8 > /proc/sys/kernel/printk g>> # insmod virtual_gpio_basic.ko h>> $ ivshmem-client #   qemu ivshmem    ivshmem-server     id cmd> int 0 0 # :       cmd> help #   : g>> VGPIO: interrupt (status = 0x0001) 

irq_chip and chained_interrupt concept

We will not go into details, this topic is well covered in the first patch presented by irq_chip , the kernel documentation and the book “Professional Linux Kernel Architecture” (currently it is outdated, but irq_chip is also not a new thing).

At the moment, the main thing for us is the fact that GPIO ports that provide interrupts are a common practice in the days of Linux that interrupt cascading from the parent interrupt controller.

This is why part of the GPIO driver responsible for interrupts uses irq_chip . In other words, such a driver uses two subsystems simultaneously: gpio_chip and irq_chip .

A quick look at the irq subsystem gives us the following picture:

High-Level Interrupt Service Routines (ISRs) - Performs all necessary interrupt service work on the device driver. For example, if an interrupt is used to indicate readable new data, the work of the ISR will be to copy the data to the appropriate place.

Interrupt Flow Handling - This subsystem is responsible for features in the implementation of interrupt processing, such as triggering on a signal level (level) or on a front (edge).

Edge triggering occurs when determining that a potential change has occurred on the line. Triggering by level (Level-triggering), is defined as a specific value of the potential, and the change in potential does not matter.

From the point of view of the kernel, the level triggering is a more complicated case, since, at the beginning of each interrupt, it must be masked.

Chip-Level Hardware Encapsulation - Used to encapsulate the features of the implementation of working with the hardware. This subsystem can be considered as a kind of “device driver” for interrupt controllers.

As we can see, the kernel takes control of the processing of the interrupt chain and the difference in the implementation of types (by front and level), if we provide the appropriate infrastructure.

IRQ Domains

The IRQ Domain subsystem, which appeared in the irq: add irq_domain translation infrastructure patch, allowed separating the interrupt controller numbers from the interrupt numbers in the kernel, providing a common array of interrupt numbers. Quoting official documentation: “Today the IRQ number is just a number . ”

Prior to this update, hardware numbers were mapped to kernel numbers as 1: 1, and cascading was not supported. By hardware numbers, we mean local interrupt numbers for the controller, which in our case coincide with local GPIO numbers.

The following types of mapping exist in the IRQ Domain:

• Linear
• Tree view
• And type "No map" (No display)

Since our interrupt vector is rather small, and we definitely have no interest in the "No map" map, our map is linear, in fact the numbers are matched 1: 1 with the offset, the difference with the old approach is that the assignment of irq numbers and the calculation of the offset core, while ensuring continuity of the allocated range.

In each function of the irq_chip interface, a pointer is passed to the struct irq_data structure, where irq_data-> irq is the interrupt number in the linux kernel, and irq_data-> hwirq is our local interrupt number within the driver. Also, a pointer to our struct virtual_gpio structure is passed to struct irq_data , which is not surprising.

Binding irq_chip and gpio_chip

If we were guided by younger versions of the kernel, we would have to use the irq_domain_add_simple function to display our number, but from version 3.15 in the gpio patch : add IRQ chip helpers in gpiolib patch there is no need to directly use the IRQ Domain interface.

Therefore, instead of using the IRQ Domain interface directly and providing the infrastructure for mapping local numbers to global ( .map () ops), we will use the functions gpiochip_irqchip_add and gpiochip_set_chained_irqchip (depend on the GPIOLIB_IRQCHIP parameter Kconfig).

A great example of use and ease of use is the gpio-pl061 driver .

Bind our irq_chip to an existing gpio_chip :

 gpiochip_irqchip_add(&vg->chip, &virtual_gpio_irq_chip, 0, handle_edge_irq, IRQ_TYPE_NONE); 

handle_edge_irq is one of the built-in stream handlers that takes control of the chain of interrupts on fronts.

Note: Frontal interrupts are the most common. The main difference from level interrupts lies in the management of the chain, the level interruption is masked in the kernel immediately upon receipt.

 gpiochip_set_chained_irqchip(&vg->chip, &virtual_gpio_irq_chip, pdev->irq, NULL); 

By calling the gpiochip_set_chained_irqchip function , we tell the kernel that our irq_chip uses a PCI bus interrupt and our interrupts are cascaded from pdev-> irq .

We ’ll modify our handler to generate interrupts depending on the state of VIRTUAL_GPIO_INT_ST :

 pending = vgread(VIRTUAL_GPIO_INT_ST); /* check if irq is really raised */ if(pending) { for_each_set_bit(i, &pending, VIRTUAL_GPIO_NR_GPIOS) generic_handle_irq(irq_find_mapping(vg->chip.irqdomain, i)); } 

irq_find_mapping is an auxiliary function for translating the local input number into the global interrupt number.

Putting it all together

First of all, we note that the irq_chip interface of our driver looks like this:

 static struct irq_chip virtual_gpio_irq_chip = { .name = "GPIO", .irq_ack = virtual_gpio_irq_ack, .irq_mask = virtual_gpio_irq_mask, .irq_unmask = virtual_gpio_irq_unmask, .irq_set_type = virtual_gpio_irq_type, }; 

The ack () function is always closely related to the hardware specificity of the controller. Some devices, for example, require confirmation of the processing of an interruption request before subsequent requests can be served.


In our case, the program vg_get_set - rather rough emulation of the eoi register is used. After setting the interrupt status flag, the eoi register is constantly polled in the loop. When the interrupt notification input bit is set by the driver, the eoi register is cleared and the interrupt status bit is cleared at the input.

Masking and unmasking is done by writing the corresponding value to the INTERRUPT_EN register .

Interrupt masking:


:


irq_type — :
IRQ_TYPE_NONE —
IRQ_TYPE_EDGE_RISING —
IRQ_TYPE_EDGE_FALLING —
IRQ_TYPE_EDGE_BOTH —
IRQ_TYPE_LEVEL_HIGH —
IRQ_TYPE_LEVEL_LOW —

user space, vg_guest_client. gpio_sysfs, “ select , () exceptfds” .

:

  FD_ZERO(&efds); maxfd = 0; for(i = 0; i < gpio_size; i++) { FD_SET(gpios[i].fd, &efds); maxfd = (maxfd < gpios[i].fd) ? gpios[i].fd : maxfd; } ready = pselect(maxfd + 1, NULL, NULL, &efds, NULL, NULL); if(ready > 0) for(i = 0; i < gpio_size; i++) if(FD_ISSET(gpios[i].fd, &efds)) { read(gpios[i].fd, &value, 1); /*    lseek  http://lxr.free-electrons.com/source/fs/kernfs/file.c?v=4.1#L769 */ if(lseek(gpios[i].fd, 0, SEEK_SET) == -1) perror("lseek"); printf("gpio number=%d interrupt caught\n", gpios[i].number); } 

sysfs :

 g>> # echo 504 > /sys/class/gpio/export g>> # echo 505 > /sys/class/gpio/export g>> # echo 506 > /sys/class/gpio/export g>> # echo rising > /sys/class/gpio/gpio504/edge g>> # echo rising > /sys/class/gpio/gpio505/edge g>> # echo rising > /sys/class/gpio/gpio506/edge 

: gpio .

 #      gpiochip   g>> # ./vg_guest_client 504 gpio_chip: base: 504 ngpio: 8 Added gpio 504 to watchlist. Added gpio 505 to watchlist. Added gpio 506 to watchlist. Entering loop with 3 gpios. h>> $ ./vg_get_set -p 1 -i 0 g>> gpio number=504 interrupt caught 

pselect:

 static irqreturn_t virtual_gpio_interrupt (int irq, void *data) int generic_handle_irq(unsigned int irq); ... static irqreturn_t gpio_sysfs_irq(int irq, void *priv); static inline void sysfs_notify_dirent(struct kernfs_node *kn); void kernfs_notify(struct kernfs_node *kn); static void kernfs_notify_workfn(struct work_struct *work); 

Conclusion

, , , , - . Qemu ivshmem . .

gpio sysfs sysfs, GPIO , . .

, , . generic-gpio , , , mmio gpio , , , . , .

, , :

• Device Tree
• generic-gpio mmio gpio
• , GPIO
• gpio — , ,

It is also impossible to lose sight of the latest changes in gpiolib - sysfs gpio is now outdated. A new ioctl- based interface for gpiolib on the way to becoming a new standard for communicating with GPIO. But the younger versions will be used for a long time, besides, no one is going to remove the old interface from the kernel at the moment. For example, I still have devices that successfully work on the 2.6.34 kernel version.

List of materials:

1. http://nairobi-embedded.org/category/device-drivers.html [Siro Mugabi]
2. http://lxr.free-electrons.com/source
3. Professional Linux Kernel Architecture [Wolfgang Mauerer]
4. LDD3 [Jonathan Corbet, Alessandro Rubini, and Greg Kroah-Hartman]

:

1. http://derekmolloy.ie/writing-a-linux-kernel-module-part-1-introduction/ ( )
2. https://developer.ridgerun.com/wiki/index.php?title=Gpio-int-test.c
3. http://www.assert.cc/2015/01/03/selects-exceptional-conditions.html

, Makefile README:
https://github.com/maquefel/virtual_gpio_basic