Experience starting AHCI in VxWorks653

Introduction

I develop applications and drivers for various aviation applications. Aviation uses operating systems with stricter reliability requirements (ARINC 653), such as VxWorks653, PikeOS, or LynkOS. While developing avionics applications, there was a problem of slow access to data on solid-state drives connected via the ATA interface. This was due to the use of the slow ATA software interface. I solved this problem by implementing the AHCI driver.

In this article I want to briefly describe the work of AHCI.


Device architecture with AHCI controller.
')

AHCI Description

Advanced Host Controller Interface (AHCI) is a mechanism used to connect data storage devices using the Serial ATA protocol.

To work with the device in AHCI mode, you must perform the following steps:

• Configure AHCI Controller
• Fill management structure
• Set the start flag in the controller
• Wait until the controller reads the structures with the list of commands from the memory through DMA and then returns the data to the buffers specified in the structures. After performing all the actions, the transaction completion flag is set.
• Profit!

All interaction with AHCI registers occurs through the PCIe BAR5 window. It is called ABAR (AHCI Base Address Region).


The distribution of the address space ABAR.

AHCI Controller Configuration

The protocol specification is publicly available at: www.intel.com/content/www/us/en/io/serial-ata/ahci.html
Current latest version of AHCI is 1.3.1

First we look for all the mass storage controllers on the PCIe bus. We are interested in devices with class id 0x01 (mass storage device) and subclass id 0x06 (serial ATA).

Controller configuration is as follows:

• Reset controller;
• Enable AHCI
• Write the physical addresses of the command lists and FIS to ABAR
• Set required flags
• Configure interrupts

Through registers AHCI - ABAR, you can make a limited number of actions - set up AHCI, read its version, etc. Access to the configuration of the data carrier and to the data itself through these registers is not possible.

To gain access, special command lists are used. Lists of commands are stored in the RAM of the calculator. The physical address of these lists is written to the AHCI controller. The AHCI controller itself calls on the DMA to the RAM of the calculator, reads the lists of commands, executes them, and reads and writes the memory blocks from the RAM. The memory allocation architecture is shown in the figure below. The OS initialization should allocate the necessary arrays in memory, calculate their physical address and, at the initialization stage, write their addresses to the corresponding ABAR registers.


Memory allocation in ABAR - HBA

Filling management structures

For each port in ABAR, 2 structures must be specified - a list of commands and FIS.

The command list consists of 2 parts - one command list header.
The header indicates the size of the command list, the size of the FIS, and a link to the physical address of the command list itself.
The list of commands contains the addresses of the buffers that will be needed when executing the command specified in the FIS and from the sizes.


AHCI Port Structures

Command header generation:
opts = (20 >> 2) | (sg_count << 16);
pp->cmd_slot->opts = cpu_to_le32(opts);
pp->cmd_slot->status = 0;
pp->cmd_slot->tbl_addr = cpu_to_le32(pp->cmd_tbl & 0xffffffff);
pp->cmd_slot->tbl_addr_hi = 0;

Here, the CFL field is filled with the FIS size value (I have a constant of 20) in double words. The PRDTL field is filled with the number of 4Mb buffers used.


Command list header format

Formation of the command table:

  #define MAX_DATA_BYTE_COUNT (4*1024*1024) sg_count = ((buf_len - 1) / MAX_DATA_BYTE_COUNT) + 1; for (i = 0; i < sg_count; i++) { ahci_sg->addr = cpu_to_le32((uint32_t) buf + i * MAX_DATA_BYTE_COUNT); ahci_sg->addr_hi = 0; ahci_sg->flags_size = cpu_to_le32(0x3fffff & (buf_len < MAX_DATA_BYTE_COUNT ? (buf_len - 1) : (MAX_DATA_BYTE_COUNT - 1))); ahci_sg++; buf_len -= MAX_DATA_BYTE_COUNT; } return sg_count; 

The function fills the structure in which it indicates which buffers will participate in data transfer. Since the size of the buffer in the record can not exceed 4Mb, the function divides the buffer into several, if its size exceeds 4Mb.


Command Table Format

Description FIS in the documentation for AHCI no. You should look for it in the description on SATA. The FIS structure describes the operation code (Read identification, read, write, etc.) and parameters (offset, size, etc.).


FIS structure

When reading the identification information about the drive, the following FIS is generated:

  memset(fis, 0, 20); fis[0] = 0x27; fis[1] = 1 << 7; fis[2] = ATA_CMD_IDENT; 

When a data block record is requested, the FIS is generated:

  memset(fis, 0, 20); /* Construct the FIS */ fis[0] = 0x27; /* Host to device FIS. */ fis[1] = 1 << 7; /* Command FIS. */ fis[2] = ATA_CMD_WR_DMA; /* Command byte. */ /* LBA address, only support LBA28 in this driver */ fis[4] = ((unsigned char) (start))&0xff; fis[5] = ((unsigned char) (start>>8))&0xff; fis[6] = ((unsigned char) (start>>16))&0xff; fis[7] = (((unsigned char) (start>>24)) & 0x0f) | 0xe0; /* Sector Count */ fis[12] = (unsigned char) blocks & 0xff; fis[13] = ((unsigned char) (blocks>>8))&0xff; 

When a read request for a data block is generated FIS:

  memset(fis, 0, 20); /* Construct the FIS */ fis[0] = 0x27; /* Host to device FIS. */ fis[1] = 1 << 7; /* Command FIS. */ fis[2] = ATA_CMD_RD_DMA; /* Command byte. */ /* LBA address, only support LBA28 in this driver */ fis[4] = ((unsigned char) (start))&0xff; fis[5] = ((unsigned char) (start>>8))&0xff; fis[6] = ((unsigned char) (start>>16))&0xff; fis[7] = (((unsigned char) (start>>24)) & 0x0f) | 0xe0; /* Sector Count */ fis[12] = (unsigned char) count & 0xff; fis[13] = ((unsigned char) (count >> 8))&0xff; 

Run for execution

After the controller has been configured via ABAR, all the lists in the memory are prepared; it is enough to write 0x1 to PORT_CMD_ISSUE and wait until the flag is cleared. You can wait in a loop (I did just that) or by interrupt.

Immediately after writing to the PORT_CMD_ISSUE, the controller through DMA will access the processor's RAM and perform the actions that were expected of it.

So as a result of the read operation, we will get the data from the media in the buffers specified in the command list.

Conclusion

Before using AHCI in the project, the read / write speed was: ~ 100kb / s
After using AHCI in the project, the read / write speed was: 30/10 Mb / s