⬆️ ⬇️

How we built the A320 Aviation Simulator: History in Pictures (Part 1)

Hello to all!



In the last post, I wrote about the invaluable experience gained in the construction of the Boeing B737NBG aircraft simulator.



Many different events happened from that post, in particular, our Boeing experienced a major overhaul in time-pressure mode, and finally I got around to writing a post about Airbas A320.

')

I will break the post into two - one will be mainly about IT, and the second - mainly about the stages of construction, with slides.



Go.





Introductory


In August 2012, I joined the newly started project of building a new flight simulator. At that time, things were as follows:





First of all, oddly enough, I undertook to paint the IT infrastructure.



IT infrastructure


In my head everything was simple: as a simulator, we have X-Plane 10 with its capabilities to bend the image under a cylindrical projection.



Scaling is necessary - it means there will be five servers: one calculates the dynamics, three - form an image for three projectors, the fourth - generates a picture for the video recording system for clients.



Project Magenta components do almost everything else.



ProjectMagenta


  1. Drawing pictures on the cab screen - Airbus Glass Cockpit, ABGC

  2. Calculation of aircraft systems - pmSystems

  3. Implementation of the work Flight management and guidance system - MCDU

  4. Autopilot. Implemented in the module Flight control unit, FCU

  5. Electro-remote control system of the aircraft, in our opinion, Fly-By-Wire. It also provides an interface with controls - sidedics, pedals, engine control knobs, etc. In addition, it is in this module that the “laws” are in place that prevent too big rolls, dangerous pitches and so on - Flight Control Laws - pmFBW

  6. Sounds - so far without them. pmSounds




It seems nothing is forgotten.



Checks and Balances


Infrastructure design was obtained from a combination of limitations and experience of Boeing.



One of the problems with the Boeing experience is that the servers in the rack get terribly dusty, then start to overheat. So the most loaded computers - and this is X-Plane - should stand in the server.



Pull the USB out of the cab - dismiss, then there must be on-board computers.



If they are on board a mobile platform, then they are either with SSD or diskless. The SSD in my memory has overheated two times, so we will know the diskless boot. All the "onboard" software is under Windows, so it will be iSCSI.



Download 6 computers over the network - bandwidth is needed, which means we must have switches with trunk support. The choice fell on the D-Link DGS-1210-16.



There will be several X-Plane, and the scripts on them should be the same, plus iSCSI, backups, etc., so there must be a file server, or rather a NAS. In terms of price-performance ratio, Synology RS-812 + came up with a memory upgrade of up to 3GB. At the same time it supports trunks.



The server should be clean, that is, the door to it should be closed, then remote access is realized as much as possible, it means that IP KVM should be (in reality, it was useful once, we mostly use VNC).



In addition, it is not smiling at all to get into the cabin space where the onboard computers are located, so you should at least be able to turn them on and off. So you need a reinforced concrete device that allows you to press the "Power" button. It was ordered to a familiar electronics engineer. After a series of iterations, a device with Eth was obtained, which can “push” the power button.

It looks like this:





The top board is “master”, the “slaves” are connected to it via RS-485. Each “slave” has 8 channels, in each channel there are two discrete inputs (for connecting the power and HDD LEDs) and one output for the Power button.



We have expensive and rare projectors, you cannot turn them off without pre-cooling, but if the input voltage is gone, you need to turn off the projectors and the on-board computers automatically, without relying on people. So we need UPSs with a network interface, the projectors and so it is, you just need to use it.



In general, as a result of a couple of weeks of design, we got this scheme:





We have on-board computers on the Celeron G540 2.5 GHz with 4 GB of memory and a Low profile video card on a Radeon HD 6450 chip, with a gigabyte of memory.



Graphic servers for X-Plane - Core i5-3470 with 8GB of RAM and two 500GB disks in the mirror. For drawing pictures are GeForce GTX 670 c 2048MB. As an OS we have Ubuntu 12.04 LTS 86_64, now X-Plane is also 64-bit.



The server that calculates the dynamics - under Win7_32 on Core i7-3770, the integrated video card is used. X-Plane is 32-bit here, and the server is under Windows because the plug-in for interface with the mobile platform is only for 32-bits and for Windows.



Cables and wires


An over-goal was to build a simulator so that only twisted pairs and power cables entered the cabin. So it happened in the end.

A 220V power cable, two twisted pairs for connecting switches (trunk), three twisted pairs for HDMI extenders and one more twisted pair for the control panel of the ladder and platform go to the cab from the outside.



The extenders I used Hama HDMI, they cope with their task.



Twisted pair everywhere - CAT6 FTP.



Separately, it would be worth writing how we decided to contrive and stretch our twisted pairs in the existing pipe between the cabinet and the mobile platform. It was a stupid idea, even with the use of cable lubricant, and then I still stretched two more pipes.



Electricity


In Boeing, we ran into the problem of an incorrect electrical network, so this time I sat down to draw a single-line diagram (I can not find it).

The idea was that we protect all lines with automatic devices, plus we protect people from electric shock.



Calculations of the load showed that the entire cabin (projectors, computers, input-output controllers, onboard ventilation, switch) is enough 3000VA.

On 5 servers, NAS, switch and KVM also need about the same.



External cab fans, an instructor station and a trap drive need less than 1500 VA, so the old 1500 SmartUPS were useful.

Total UPS costs four - two Ippon Smart Winner 3000 VA with an additional battery pack, and two 1500VA SmartUPS, left over from one of the iterations of improving Boeing.

On one triple-kilovatnik the server hangs, on the other - the whole cabin. On one 1500, the cabin ventilation and instructor station hangs, and another one — in reserve.



Given the availability of autonomous energy sources (UPSs), the RCD to protect people stand after the UPS, i.e. bespereboynik us "tied" to the shield.







Ventilation and air conditioning


Our server room is quite small, about 5 square meters, and 2kW of heat is emitted from the rack there, and probably 1kW from the platform control cabinet. So we will hang a condo.



Conde was chosen Samsung AQ12TSBN, he has a performance in the cold about 3.5kW. I made the installation myself, the benefit of the tools and equipment was purchased before and was used for mounting the kondey in the country.

Summer showed the correctness of choice, the temperature in the server fluctuated around the established 19 degrees.



It was a stupid mistake to hang the indoor unit over an electrical shield, so I had to pay particular attention to drainage. Pah-pah, there were no excesses, but by next summer I will install a drainage pump, for sure.



In a cabin with ventilation, the situation is not so simple.

We have two volumes - the cockpit itself, in which people are sitting, and the cabin space, in which there are irons-projectors and computers.

From the very beginning, we decided that we would not do air conditioning - too many problems with condensate.



In the behind-the-cabin space, inflow and exhaust are made by low-noise fans with a capacity of 250 cubic meters per hour. Inflow is made to the place of installation of computers





Hood - near the projectors, at the top point (sorry for the quality of the pictures)





In fact, it would not hurt to plug the fan into the hole in the "roof", but temperature measurements do not show such a need.



In order for the air to circulate in the cabin, there is a large 2000-cc pressure fan outside, well, and the exhaust is done by two consecutive silent silencers:





Initially, according to calculations, I was mistaken with the choice of fans, the 2000-cc supercharger appeared later, at first there were 250-cc Silent.

Even with a fan in the cabin 2000 is hot, so we are thinking about how to improve the system.



Crutches, crutches and crutches


From the very beginning it was clear that even “shelf” products would need to be wrapped with numerous crutches.



There were several problems for peeling:

  1. Sharing General X-Plane Data
  2. Automation of on-off projectors
  3. Restart failed Project Magenta components




X-Plane: sync and launch


In order to synchronize the X-Plane data, rsync was finally selected from NAS.

I tried to run directly from the NFS-balls, but considering that we have both Winda and Linux, and the X-Plane directory structure, I think, was invented in a drunken delirium, I decided that everyone has their own configuration and scripts and common data synchronized over the network every time X-Plane starts or restarts.



By the way, as the idea was worked out with the launch of X-Plane right from the balls, I got to the point where I edited the binary so that all the files opened for recording were in the same directory. Here is a picture from this stage:



Everything was successfully completed, and in two days an update was released. After that, I decided to do rsync.



The startup script turned out like this:

#!/bin/bash LOGFILE="/var/log/xplane-starter.log" log_msg () { DATE=`/bin/date` /bin/echo "$DATE: $@" >> $LOGFILE } log_msg "Starting xplane-startup.sh" while [ 1 ]; do log_msg "Rsyncing" /root/bin/rsync_64.get >> $LOGFILE /bin/sed -i".bak" '/UNSAFE/d' "/xplane.64/Output/preferences/X-Plane Screen Res.prf" /usr/bin/pactl set-card-profile 'alsa_card.pci-0000_01_00.1' 'output:hdmi-stereo-extra1' /usr/bin/xinit /root/bin/xinitrc.64 >> $LOGFILE 2>&1 & XINIT_PID=$! log_msg "xinit pid = $XINIT_PID" while [ 1 ]; do sleep 1 # Check that the process is still alive if [ -e /proc/${XINIT_PID} -a /proc/${XINIT_PID}/exe ]; then sleep 1 else log_msg "Process dead, restarting" break fi done done 




Since X-Plane doesn’t need a window manager at all, the actual launch is done via xinit:

 /usr/bin/xsetroot -cursor_name top_left_arrow DISPLAY=:0.0 /usr/bin/X11/xset s 0 DISPLAY=:0.0 /usr/bin/X11/xset s noblank /root/bin/shm_wipe.sh /usr/bin/x11vnc -display :0 -ncache 10 -many & umask 0000 /xplane.64/X-Plane-x86_64 --no_crash_reporter /usr/bin/killall -9 x11vnc 




shm_wipe.sh - my laziness. Periodically, x11vnc ceases to run due to the exhaustion of shared memory, so the search recipe and “unbind” unused segments are found on the Internet.



Additionally, X-Plane is restarted through cron in the early morning, because there are memory leaks in it. In addition, at night, just in case, a command is given to turn off the projectors.



On Windows, not everything is so kosher, I do rsync synchronization periodically with my hands ...



Onboard computer monitoring


To deal with the hanging components of ProjectMagenta, a whole PowerShell script was written on the on-board computer, as well as with .NET elements.



The script began with detecting the failed USB devices, and then it was overgrown with everything else.



Before the heap through this script, the components are launched, and it is checked whether anyone has fallen out during the play.



A separate line will tell about EHID.

The EHID is the interface between the cab hardware (I / O controllers) and software components. It is distributed under the NDA, I signed it and now I am the happy owner of the specification. For the time being, I applied it for sticking a bug with a tiller in X-Plane, which I will discuss below.

The bottom line is that there is a component A320_EHID.exe (and, respectively, B737_EHID.exe for Boeing), which holds in itself the "tree" of all all the components of the cabin - and analog axes, and LEDs, and toggle switches and everything.

Each item has its own name and 32-bit ID.



Applications are not addressed directly to the hardware, but to the EHID. Communication - over TCP, Event-driven. Polling is all implemented in EHID, so this concept makes life much easier.



The specification involves two levels - between iron and EHID, and between EHID and applications.

The lower level is represented, including the description of the protocol on top of the USB HID, and is implemented including in the controller firmware.



Update: drew a picture of the interaction between the components of the simulator





The trouble is that the “last mistake” was found at the junction of Vind drivers and EHID implementations in the firmware: when data from a large number of devices are received at the same time, something magical happens inside the EHID, which leads to DoS. Found, thank God, easy: EHID stops responding to application requests. Monitoring is based on this.



Govnokod - that one, do not kick.

 Import-Module c:\scripts\Release\DeviceManagement.psd1 [int] $Port = 21843 $IP = "192.168.XX" $Address = [system.net.IPAddress]::Parse($IP) $LogDirs = @("C:\EHID","C:\MCDU","C:\RMCDU","C:\FBW","C:\pmSystems","C:\FCU","C:\xpuipc") Write-Host "Starting the supervisor script..." # Check if we just started $p = Get-Process -Name xpwideclient -ErrorAction silentlycontinue if (!($p)) { Write-Host "Just started" # Find and remove logfiles foreach($logdir in $LogDirs) { del $logdir\*.log -ErrorAction SilentlyContinue } del R:\*.tmp -ErrorAction SilentlyContinue # Set the routing for the goddamn windows shitty default gw to iSCSI target route delete 0.0.0.0 route add 0.0.0.0 mask 0.0.0.0 192.168.XX # Start XPUIPC Start-Process C:\xpuipc\xpwideclient.exe -WorkingDirectory "C:\xpuipc" -WindowStyle Minimized Start-Process C:\EHID\A320_EHID.exe -ArgumentList "-run" -WorkingDirectory "C:\EHID" -WindowStyle Minimized Start-Sleep -Seconds 3 } else { Write-Host "Already working" $startFlag=0 } # Cycle forever while ( 2 -ge 1) { # Clear the flag $failFlag = 0 # Get the list of the failed devices $FailedDevices = Get-WmiObject Win32_USBControllerDevice |%{[wmi]($_.Dependent)} | Where-Object {$_.Status -ne "OK"} foreach($fDevice in $FailedDevices) { # Set the flag to restart the processes $failFlag = 1 # Disable and then enable device $dDev = Get-Device | Where-Object {$_.InstanceID -eq $fDevice.DeviceID} Write-Warning "Failed device found:" $fDevice.DeviceID Disable-Device -TargetDevice $dDev -Verbose Enable-Device -TargetDevice $dDev -Verbose } # Detect hang of A320_EHID # Create IP Endpoint $End = New-Object System.Net.IPEndPoint $address, $port # Create Socket $Saddrf = [System.Net.Sockets.AddressFamily]::InterNetwork $Stype = [System.Net.Sockets.SocketType]::Stream $Ptype = [System.Net.Sockets.ProtocolType]::TCP # Create byte array [Byte[]] $Message = 0x02,0x03,0x00,0x04 $buffer = new-object System.Byte[] 8192 Try { $Sock = New-Object System.Net.Sockets.Socket $saddrf, $stype, $ptype $Sock.TTL = 26 $Sock.ReceiveBufferSize = 8192; $Sock.ReceiveTimeout = 500; $Sock.Blocking = 1; # Connect to socket $Sock.Connect($end) $Sent = $Sock.Send($Message) Start-Sleep -m 30 $Received = $Sock.Receive($buffer) $Sock.Close() } Catch [Exception] { Write-Host "Error checking EHID" $_.Exception.Message; $failFlag=1; } # Detect restart file-flag $FileExists = (Test-Path "R:\restart.txt" -PathType Leaf) if ($FileExists) { Write-Host "Restart file exists!" del R:\restart.txt $failFlag = 1 } # Restart the processes if ($failFlag -eq 1) { Write-Warning "Restarting processes" Write-Warning "Freezing..." Start-Process C:\scripts\sim_pause.exe -WorkingDirectory "C:\scripts" -WindowStyle Minimized Stop-Process -Name A320_EHID -force -ErrorAction silentlycontinue Stop-Process -Name pmFBW -force -ErrorAction SilentlyContinue Stop-Process -Name pmSystems -force -ErrorAction SilentlyContinue Stop-Process -Name MCDU -force -ErrorAction SilentlyContinue Stop-Process -Name RMCDU -force -ErrorAction SilentlyContinue Stop-Process -Name FCU -force -ErrorAction SilentlyContinue # Set start flag $startFlag=1 } if ($startFlag -eq 1) { Start-Process C:\EHID\A320_EHID.exe -ArgumentList "-run" -WorkingDirectory "C:\EHID" -WindowStyle Minimized Start-Sleep -Seconds 3 } # FIXME Workaround for the falling PM processes $p = Get-Process -Name RMCDU -ErrorAction silentlycontinue if (!($p)) { Write-Host "RMCDU dead, restarting." Start-Process C:\RMCDU\RMCDU.exe -WorkingDirectory "C:\RMCDU" -WindowStyle Minimized } $p = Get-Process -Name FCU -ErrorAction silentlycontinue if (!($p)) { Write-Host "FCU dead, restarting." Start-Process C:\FCU\FCU.exe -WorkingDirectory "C:\FCU" } $p = Get-Process -Name MCDU -ErrorAction silentlycontinue if (!($p)) { Write-Host "MCDU dead, restarting." Start-Process C:\MCDU\MCDU.exe -WorkingDirectory "C:\MCDU" } $p = Get-Process -Name pmFBW -ErrorAction silentlycontinue if (!($p)) { Write-Host "pmFBW dead, restarting." Start-Process C:\FBW\pmFBW.exe -WorkingDirectory "C:\FBW" } $p = Get-Process -Name pmSystems -ErrorAction silentlycontinue if (!($p)) { Write-Host "pmSystems dead, restarting." Start-Process C:\pmSystems\pmsystems.exe -WorkingDirectory "C:\pmSystems" } if ($startFlag -eq 1) { # Clear the flag and start the processes $startFlag = 0 Start-Sleep -Seconds 45 Write-Warning "Un-Freezing..." Start-Process C:\scripts\sim_unpause.exe -WorkingDirectory "C:\scripts" -WindowStyle Minimized } Start-Sleep -Seconds 5 } 




X-Plane, Tiller and Crutch


The tiller is the front strut control knob. Need to taxi on the ground.

You can steer in two ways - tiller and pedals. To achieve a certain speed, the thiller stops acting, only the pedals remain.



The problem is that in X-Plane (as acknowledged by its author) the dataref “lost”, i.e. internal variable where it would be possible to record the positions of the thiller.

In addition, if the joystick is not connected to the X-Plane, then he believes that we are poor simmer who fly on the keyboard, and therefore it is necessary to make a U-turn on commands from the steering-sided control.



I had a very tumultuous correspondence on this topic with X-Plane author Austin Meyer (during which he showed himself to be a hysterical goat, sorry Mua), who promised to correct this joint in one of the releases. We wait.



While we are waiting, I wrote a crutch, which on one side clings to the virtual joystick (VJoy), and on the other, via the network and the EHID takes the position value of the tiller's handle.

In X-Plane, I assigned a single axis - this same tiller, and, in the eyes of X-Plane, we ceased to be rogue, and therefore the wheel reversal function with the side stick turned off.

There is nothing complicated or unusual in this crutch, so I see no reason to give the source code. Written an hour on .NET in my opinion.



Conclusion


At this point I’ll probably stop, and so the post turned out to be very thick.

In the next part there will be slides and the history of the construction of Airbasa.

For the seed - a photo from this stage:





Thank you for reading!

Source: https://habr.com/ru/post/200046/



All Articles