It’s official: both the Federal Aviation Administration (FAA) and the air traffic controllers’ union (NATCA) have decided that ERAM (En Route Automation Modernization) is close enough for government work.
My facility, Minneapolis Center (ZMP) will start running the latest version of ERAM this week, planning to run it continuously. Denver Center (ZDV) started running it full time over a week ago.
We recently ran the same version of the software for a week towards the end of May. Apparently the primary reason ZMP didn’t start running ERAM full time then was because we don’t have enough personnel trained (mostly support staff) with ERAM.
Does that mean that the latest version has all the major bugs fixed? Not by a long shot. In fact the latest build even introduced a significant new bug that broke the “trial plan” functionality in the EDST (Enhanced Decision Support Tool – currently the User Request Evaluation Tool or URET) when using altitudes.
The EDST contains the flight plan information for each aircraft and allows controllers to test plan altitude and route changes to see if those changes would place aircraft too close together.
The fix for that bug – don’t use the trial plan function with altitudes. For all intents and purposes the EDST trial planning is unreliable.
Then during our latest ERAM run one of the sector EDST computers crashed (a red “X” filling its screen); the first time I’ve actually seen part of ERAM completely fail. I would have expected however, that ERAM was becoming more reliable as time went by and not less…
During our last ERAM run I also noticed what they used to call “vector wobble”. Controllers can run out a vector line from each aircraft. The vector line is calculated based on the speed and heading of the aircraft and indicates where the aircraft will be after 1 and up to 8 minutes of flying time.
The vector line is a critical tool for keeping airplanes separated.
Since controllers are required to monitor to ensure that aircraft don’t inadvertently fly off course, they also use the vector line for that purpose. They can display a route line that shows the filed route of the aircraft and then turn up the vector line. If the vector line is close to matching the route line, then the aircraft is on course.
But there are minute errors in data and the way ERAM process it causes the calculations of where the aircraft will be in the future to vary more than with HOST. So the vector line moves around a bit.
I noticed that I was distracted several times believing an aircraft was flying a bit off course based on its vector line, only to see it correct itself. It’s not something that happens much in HOST.
Things like that aren’t really bugs (some might go so far as to suggest they’re “features”) and controllers will get used to them over time, but they’re not improvements by any means (certainly not what one would expect from a next generation air traffic system).
Before we ran the last version of ERAM over a month ago, controllers were advised of the most serious (known) bugs in the ERAM software and mitigations for those bugs. Before the latest run there were no briefings on bugs at all, other than a computer briefing item on the aforementioned trial plan bug.
But the “bug sheet” containing the major bugs for ERAM and their mitigations looked very similar to the last ERAM bug sheet. This time it was simply placed at each sector for controllers to find and read on their own while working.
It’s clear that the myriad bugs in ERAM have simply become background noise. There have been so many bugs in ERAM for so long, as long as ERAM isn’t outright crashing few are really noticing.
Notably, years later due in large part to bugs, ERAM still doesn’t have some of the basic functionality that the current HOST computer has.
The current HOST notifies controllers of important computer information with a “message waiting”. When a new message arrives there is both a visual and an aural alert (a beep). While controllers are able to set the volume of the aural alert in HOST, it can’t be turned off – that’s how important they thought the message waiting aural alert was.
Well, in ERAM the aural alert doesn’t work. It never has, and at this point it looks like they’ve given up on trying to fix it. Apparently they’re still too busy trying to fix all the other major bugs in ERAM, and they think this particular one isn’t that important.
Will they ever fix this and other bugs like this? At this point I’m beginning to doubt it.
The end result is habituation. I wrote about it over a year ago. Controllers and everyone else become habituated to the bugs in their equipment and do their best to simply work around them.
They’re distractions and shortcomings nonetheless, but the longer they’re around, the more everyone becomes habituated to them. And the FAA can deny that there is a real problem – a bug isn’t a bug apparently unless it’s directly causing aircraft to get too close to each other.
The FAA knows habituation will happen, and they’re apparently OK with it. We still have bugs in URET that have been there for years that haven’t been fixed.
Any bug has the potential to distract controllers from keeping the airplanes safely apart. In the past software bugs have distracted controllers and resulted in operational errors.
I recently read in the news where two small aircraft collided in Virginia. The notable thing about this crash was that the aircraft contained pilots from both the FAA and the National Transportation Safety Board (NTSB).
To preserve impartiality, the FAA and the NTSB decided that an outside agency should conduct the accident investigation.
I bring it up because as I wrote last time, currently both the FAA and NATCA have too much political capital at stake with ERAM to be impartial with respect to the project.
There was an Independent Operational Assessment (IOA) of ERAM in early 2011 that determined that ERAM wasn’t ready for deployment at other facilities then. There has been no follow-up outside assessment of the project since then.
And to date no one has done any real risk assessment for ERAM either.
Who’s looking out for the best interest and safety of the flying public here?
Maybe ERAM is close enough for government work. But those with the authority to deploy it shouldn’t be the only ones making that decision regardless.
It’s a disappointment that the “Next Generation” enroute computer system (ERAM) is no better than the one we’ve got (and is in many ways worse), but working for the FAA for over 20 years has been both frustrating and disappointing, so it’s nothing new to me really.
As an organization, the FAA has never truly believed in the expression, “If a job is worth doing, it’s worth doing well.” Instead they definitely tend to use the “close enough for government work” ideology.
ZMP went ORD in record time! The ATM must have been under enormous pressure to declare ORD in such a short time. I guess the FAA feels the rigorous standards of yesteryear no longer apply. We worry that something bad will happen with “the good enough for Government work” system. If that happens Congress will take scalps and the FAA or at least the air traffic function will be privatized.
Next time you write a blog like this, email me. I’ll spread it around. Keep fighting the good fight. You’re doing the right thing and everybody knows it — even the FAA.