It’s been a while since my last entry and although I’ve had plenty to write about, I’ve been enjoying the summer here in Minnesota that will inevitably end all too soon.
But I did want to write an update on the status of the FAA’s En Route Automation Modernization (ERAM) since I’ve written about it so much previously.
ERAM is the enroute air traffic control system computer replacement that’s supposed to be a cornerstone of the FAA’s much touted NextGen air traffic control system, although due to numerous bugs and problems with the ERAM program its testing on live traffic was suspended this spring.
Since my facility (Minneapolis ARTCC or ZMP) is no longer a key site for ERAM we’ve been spared runs on live air traffic for the time being. But that also takes us out of the loop a bit when it comes to the status of ERAM development, even though we have been running live traffic on our backup computer system (EBUS/DARC) in the middle of the night (on the mid-shifts) in order to perform testing on ERAM.
The current two key sites for ERAM are Salt Lake Center (ZLC) and Seattle Center (ZSE) and they’re the only facilities authorized to run ERAM on live traffic. Once ZLC and ZSE testing determine that ERAM is fit for use on live traffic again they will have an In Service Decision (ISD) after which non key sites can start running ERAM on live traffic as well.
ZMP was brought on board as a key site in part to be able to test the ERAM functionality with Non-Hosted Terminal (approach control) facilities, a configuration that apparently both ZLC and ZSE lacked.
Non-hosted Terminals are non-ERAM facilities adjacent to an ERAM facility that need to be able to receive and send flight plans back and forth to ERAM. Since the two key sites didn’t have the adjacent facilities to be able to test this functionality, ZMP offered to do the job.
However, with all the problems they were having with ERAM it was eventually decided that the testing of the Non-hosted Terminal functionality could wait, and ZMP was no longer a key site.
This in not only indicative of how haphazard the ERAM development cycle has been, but of how far ERAM has still to go before it’s ready for deployment across the country.
Right now the FAA and the ERAM contractor, Lockheed Martin, are still working on ERAM to get it to a state where it can reliably perform basic functionality and not worrying about the extra functionality ERAM will need to work across the country.
That’s what they should have been doing initially, instead of rushing to use ERAM on live traffic simply to meet an unrealistic deployment schedule well before ERAM was able to perform basic functionality reliably.
The FAA is now tentatively planning on starting to deploy ERAM at other enroute facilities early next year although the “waterfall” (deployment schedule) is far from set in stone and thereby isn’t being published anywhere.
So much for the project being on time and on budget…
ZLC started testing ERAM on live traffic in August again and has completed some multi-day operational runs. Since they still had some critical problems, they’re waiting for a new version of the software before they’ll make an attempt at other extended live runs and eventually a continuous 24/7 operational run again. (The last time they tried this was back in March of this year.)
The ERAM software sounds like it’s improving, but they’re still apparently experiencing critical bugs with basic functionality such as missing aircraft computer identification numbers in data blocks and some tracking issues, as well as continued memory leaks that make the system sluggish and/or unstable after running for only a few hours.
The fact that they’ve been working continuously on ERAM bugs since this spring, when they stopped testing it on live traffic, and still have enough critical bugs to preclude a continuous operational run show that there was no way ERAM was fit and/or ready for use on live traffic last spring even though they were using it for that purpose.
It’s just too bad the FAA couldn’t come to that conclusion on their own, instead finally bowing to pressure from various sources to stop testing the faulty software on live traffic.
In the meantime they also finally developed a method to more easily transition back and forth from the current HOST computer system to ERAM they’re calling “shadow mode”. Shadow mode has the ERAM system shadowing the HOST computer so that it can get flight plan data prior to switching over, making the entire transition a lot less work intensive for the air traffic controllers and a lot less messy.
This functionality should have been part of ERAM initially, but apparently the FAA and Lockheed Martin were delusional enough to believe that they were just going to turn ERAM on and everything would work great.
When that didn’t happen and they realized they would need to perform the transition back and forth from ERAM to HOST repeatedly they finally decided to develop a method to more easily do so.
That missing functionality is another indication of how screwed up both the FAA and Lockheed Martin’s approach to the ERAM program has been from the very beginning.
There’s functionality that should have been included from the very beginning that they’re just getting around to adding now. There’s other functionality they didn’t seem concerned with and/or didn’t even know they needed (like the Non-hosted Terminal issue) because it wasn’t part of the original testing regimen.
Thus there’s sure to be more problems encountered as they find new issues they hadn’t thought about when they start deploying ERAM across the country, not the least of which is running ERAM with lots of air traffic.
The FAA also had intentions of decommissioning both its HOST computer system, and its backup computer system (EBUS/DARC) after ERAM was up and running.
However, since ERAM is the backup for ERAM (running the same software on multiple channels) if there is a critical bug in the software that causes one ERAM channel to crash, it would be fair to assume that the bug would occur on the other channels as well causing all the ERAM channels to crash in turn.
That means that until ERAM attains a much higher level of stability and reliability the FAA should maintain a legacy backup system (such as EBUS/DARC), but that’s not to say they will.