ERAM: Is it Good Enough or Not?

The Federal Aviation Administration’s (FAA) En Route Automation Modernization (ERAM) development continues, slowly but surely.

The entire ERAM program was a disaster that the FAA had been in denial about for years.  In 2009 they were proclaiming the program on time and on budget.

When they started trying to use ERAM at Salt Lake Center (ZLC) they found it was unstable and bug ridden.  But they had no way to test it other than on live air traffic.  So they did.

Because timelines were clearly more important to the FAA than safety, in spite of all the problems they went ahead with their plans to deploy ERAM at other facilities (including mine – Minneapolis Center or ZMP).

However, in the spring of 2010, it finally came to light how bad ERAM really was, and for a few months the FAA took a break from live testing of the system to work on fixing its many problems.  Then a few months later they started running it again on live traffic at its two keysites, Salt Lake Center (ZLC) and Seattle Center (ZSE).

But after continued testing and development, by spring of 2011, according to an Independent Operational Assessment (IOA), the system still wasn’t ready for further deployment.  The controllers’ union (NATCA) agreed with that assessment.

What was the FAA’s response to that critical assessment? – declare an In Service Decision (ISD) which meant it could be deployed at other facilities.

Some time ago I believed NATCA was setting itself up to be a fall guy/patsy for the failures of ERAM.  But when the FAA finally asked NATCA for real collaboration with the ERAM program the final nail was put in the coffin.

The instant the FAA asked NATCA to collaborate on ERAM, NATCA was placed in a lose/lose situation – they couldn’t claim to be a critical part of the National Airspace System (NAS) and yet not get involved with ERAM when the FAA conceded they needed help from its controller workforce.

But apparently NATCA also saw the program as an opportunity to be used for political gain.  NATCA believed they could step in and save ERAM, and thereby show how incompetent the FAA was and how smart they were.  Unfortunately they didn’t consider how messed up the ERAM program was.

When NATCA got the opportunity to become partners with the lemons that were ERAM, they decided to make lemonade.

However, that immediately turned NATCA’s involvement with ERAM into a conflict of interest.  They suddenly needed ERAM to succeed as much as the FAA did.  It was no longer about safety, but politics.

Today neither the FAA nor NATCA is willing to openly admit how much development ERAM still needs, as both have underlying political motives at heart; neither considers the controllers who have to work with it nor the overall safety of the air traffic system.

Case in point, for some time both Salt Lake Center (ZLC) and Seattle Center (ZSE) have been running versions of ERAM neither the FAA nor NATCA is will to run full time at other facilities because of all the known bugs (the known bug count is somewhere around 1000!).

But nonetheless, last month my facility ran the same version of ERAM on live air traffic two separate times for a total of 3 days.  But it was only on the weekend, and apparently people who fly on airplanes through ZMP’s airspace on the weekend don’t really count…

Don’t get me wrong – ERAM has made a lot of progress since the FAA last tried running it on live air traffic back in 2009.  But any air traffic computer system that has over 1000 known bugs isn’t fit for use on live air traffic.  If there are that many known bugs, one can only speculate as to the number of latent (unknown) bugs still in the software.

I’ve been accused of being an ERAM “hater” because of my views on it from the management side of the FAA, and more recently from the air traffic controllers’ union (NATCA) side.

As is typical of people wanting to avoid addressing facts, that’s a gross oversimplification that ignores my complaints and concerns about ERAM.

The truth is that I’m not opposed to ERAM.  What I object to, and have always objected to, is the way the ERAM program has progressed from its very inception.  It’s another one of the FAA’s many repeated examples of how not to upgrade a system that’s critical to how air traffic controllers do their jobs.

ERAM is an overly ambitious program, the likes of which the FAA has historically proved unable to successfully manage.  As proof consider the disastrous Advanced Automation System (AAS) including the Initial Sector Suite System (ISSS) in the 1980-90s and the problems with the Standard Terminal Automation Replacement System (STARS) program of late 1990’s/early 2000s.

Obviously the FAA thinks more of its ability as an organization to successfully manage these major programs than they’ve repeatedly demonstrated the ability (or lack thereof) in the past.

As I accept that as a given, what I object to most is alpha and beta testing software on controllers and the flying public.

I take my job, and its inherent safety critical aspects very seriously.

When the FAA first started running ERAM on live traffic by any standard it was alpha software.  It was unsafe and shouldn’t have been used on live air traffic, a fact that was eventually conceded, but not before it was used for many hours on the unsuspecting flying public.

Since then it has been improved, but the last time my facility ran ERAM (last month) the version of the software we ran had over 1000 known bugs.  By any reasonable standard it still can only be considered beta software.

ERAM was put into use well before it was ready, forcing air traffic controllers into the unfortunate position of working with and around its many shortcomings and problems.  Today that situation still exists.

Since air traffic uses many analogies with pilots, because of its current state controllers have been forced to act as “test pilots” with ERAM.  Test pilots volunteer for that duty, but controllers haven’t been given a choice to “test fly” ERAM.

Most pilots wouldn’t be comfortable flying an aircraft they didn’t think was airworthy.  But pilots are given the final authority as to whether or not to fly their aircraft.  Controllers on the other hand don’t have that choice, and many have very little confidence in the “airworthiness” of ERAM.

At the very least, no test pilot would put passengers in aircraft they were testing, but that’s exactly what the FAA has been doing with ERAM.

The software wasn’t tested adequately before acceptance and there was no way to fully test the software other than on live air traffic.  The fact that testing could only be performed on live air traffic was itself a fundamental design flaw of ERAM.

Additionally the design of ERAM clearly didn’t consider the things controllers need to do their jobs more safely and efficiently.  As I don’t know what tools the technicians use to keep that system running, I can only assume the same is true for the tools ERAM gives them.

What I do know is that whatever analysis (if any) was put into the examination of how controllers work and what tools would help them do their jobs better is absent from ERAM’s design.

One of the long-running FAA managers’ catchphrases for those criticising their system “upgrades” is that controllers are resistant to change.

While that’s true (controllers don’t want to re-learn how to do their jobs), it’s also true that I would love (like most other controllers) to have tools to enable me to do my job better.

Instead most often what we get are tools that make parts of our jobs more difficult.  Parts of ERAM actually make routine parts of controllers’ jobs more tedious.

Most notably, with ERAM, dropping a data tag on an aircraft that a controller is no longer required to monitor is a two step process instead of the current single step process.

Because attention to detail is important in air traffic control, distractions of any form are undesirable for controllers working air traffic.  However, ERAM has many ways that it creates distractions for controllers.

Bugs in ERAM are a distraction.  Wondering if and when ERAM will crash is a distraction.

But a new “feature” in ERAM also creates distractions for controllers.

Air traffic control is highly compartmentalized.  Controllers are delegated a specific section of airspace they are responsible for keeping airplanes moving safely and efficiently in.

There are numerous rules to make sure that the responsibility for keeping aircraft separated in a section of airspace falls on a single controller to avoid confusion.  Controllers cannot let aircraft enter another’s airspace without permission, and can only give control instructions to aircraft in their airspace.

However, for contingent safety reasons should they notice, controllers are also responsible for pointing out problems they see in adjacent sections of airspace.  But under the current HOST system though there are few ways for a controller to see or know for sure what is occurring in adjacent airspace.

Thus, in general controllers work their own airspace, separate their own traffic and assume the controllers working adjacent airspace are doing the same; a prudent and reasonable expectation.

But instead of adding tools to ERAM to help controllers better work their own airspace, ERAM added the “feature” of AOI (Area of Interest), expanding the areas in which controllers would be alerted to problems outside their airspace, adding more responsibility for controllers to monitor their “neighbor’s” airspace as well as their own.  The claim is that the AOI will help controllers work more efficiently near their sector boundaries, but because of the way the air traffic rules are written it also expands their responsibility outside their sector.

Although to a casual observer it might make sense to have more eyes watching traffic, it starts to blur the lines of compartmentalized responsibility that is one of the cornerstones of air traffic control.  The end effect is that the ERAM AOI has the potential to distract controllers from monitoring traffic within their own sector.

That’s not to mention the various known bugs, little glitches and quirks of ERAM that will be around for some time to come.  The expectation clearly is that controllers will get used to them and work around them.  They’re distractions nonetheless, and those distractions are detrimental to safety.

It was clear that the FAA has always driven more by timelines and budget than safety when it came to ERAM deployment, but now the controllers’ union has political motives as well with regards to ERAM, so they’re both willing to roll the dice and run ERAM on live traffic even though they know it’s not ready.

If ERAM is ready for use on live traffic, then why doesn’t the FAA and NATCA just use it at all the facilities 24/7?

The answer is obvious: because they know it’s not ready.

But they’re running it at more and more facilities nonetheless.

You can’t have your cake and eat it too.

If both the FAA and NATCA are willing to run ERAM full time at some facilities and part-time at others, how can they argue it’s not safe to use?  Air traffic and the people on the aircraft flying at facilities with less traffic, and at busier facilities on the overnight shifts and on the weekends are no less important than those flying elsewhere and at other times.

If it’s “safe enough” (i.e. “close enough for government work”) then I say just turn it on and let the chips fall where they may.  Otherwise we still shouldn’t be running it on live traffic.

6 comments

  1. Wow…can’t say I disagree with all of your assessment of the program. But what I do know as someone who is involved with NextGen, as a human factors professional who sees it as their goal to understand the controller’s needs and safety concerns, I am having a HELL of a time getting access to the people!!!! ERAM exists…FAA is clearly pursuing it, someone is funding it…if you, as a controller, as a union member, (as a tax payer!!!) don’t want these systems to be deployed without being optimized for the end user (that’s the controllers) then you need to INSIST, PUSH, and DEMAND that there be a human factors/cognitive engineering component in the process. AND, PLEASE, please, please…let us talk to controllers, ACTIVE controllers. Let us help them make systems that makes your job easier!!! If I dont’ get access to you, as the controller, then its the system developer that’s making decision about how you do your job. How do we, as HF professionals, get unions to understand that we want to represent their peoples need in the development process? I got into this field (HF/cognitive engineering) because I want to be part of designing systems that support how the operator does their job, not how the stakeholder or management sees the job. But that hinges on gaining access to the actual operators. And from my experience with NextGen, it seems that the union is stopping that from happening (please correct me if that is wrong! Is it the FAA that is stopping me?)
    How do we solve this?

  2. Thanks for the comment!

    Too often it’s been my experience that by the time controllers in the field see new technology, it’s too far developed to make significant changes to it. And it usually falls far short of what it could have been.

    I’m well aware that those involved in the development of the new technology work hard to create good products that they think will help controllers do their jobs better. But as you noted, they’re often not given access to the end users of those systems (including both controllers and technicians).

    In the end controllers get technology that doesn’t suit their needs, frustrating both them and those who believed they had been developing useful tools for them.

    I would love to be involved in helping develop new technology, but in the course of my career those opportunities rarely if ever have arisen.

    URET, ERIDS and ERAM all have significant shortcomings that I’m forced to work around.

    It would be ideal if controllers from every facility got a chance to be involved early in the development of new technology. That’s because different facilities use the equipment in different ways. What works well at one facility might not work well at another.

    For example, URET (our computerized flight data system) doesn’t work with non-radar, and there are plenty of facilities (including mine) that have to separate non-radar traffic. URET is useless for that. I can only assume that the first facilities to use URET didn’t experience those shortcomings because they didn’t need to use it in a non-radar environment.

    Years ago MITRE was involved in researching ways to modify URET to work in a non-radar environment. They visited my facility and met with controllers to that end. I don’t know what happened to that project.

    I suspect that the FAA (mistakenly) believed that new technology would eliminate non-radar completely and thereby didn’t feel the need to modify URET. Now ERAM includes URET (renamed EDST), with the same shortcomings.

    ERIDS (En Route Information Display System) is a computerized information system that could have been a fantastic tool for en route controllers. Instead it too has significant shortcomings including a poor interface and a display that has resolution that is inadequate for the job.

    To top things off, last fall we got new ERIDS touchscreen displays that frequently go out of calibration making them unusable. It’s assumed that the new displays weren’t tested adequately before deployment into the field.

    A large part of the problem would be solved if the FAA were willing to commit the finances to develop those systems properly. Involving many end users of the systems (including those at different facilities) early in development would go far in producing much better systems. But that would come at a significant cost, something the FAA has largely proved unwilling to invest.

    I don’t know why NATCA would object to controllers being involved in that process, but unfortunately it’s clear that there are plenty of politics involved…

  3. This is the finest piece I have read on ERAM. Both responders above make excellent points. The FAA brought NATCA in to the process at such a late date because of the two reasons you mentioned. One to squelch controller descent and two, NATCA now has “skin in the game” therefore they share in the blame as well as the benefits. The new contract only cemented the deal by putting the controllers in golden handcuffs.

    I think you are dead on when you point out if the system is good enough for ZSE, ZLC why not the entire country? Just turn it on and do away with Host if the FAA has such confidence in it. Or if not, go to Congress and tell them we screwed up, trash ERAM, re-host Host and start on a real replacement system with the controllers there at the very beginning. That would take about seven years. No contractors, just controllers and technical FAA personnel who understand air traffic control software/hardware and have the controllers tell us what the next system should look like. Yea I know, I can hear the laughing from Washington now. What a radical idea having the end users of the system there at the very beginning of its design.

    ERAM was never seriously tested anywhere even though the tools where there to convert current scenario into ERAM scenario. Instead it’s been tested on the flying public a point you make perfectly and indisputably. The Host contract ends in 2013 and the FAA has to make a decision as to whether to extent Host for hundreds of millions of dollars or simply turn ERAM on nation wide. The next couple of months will be interesting with ZMP being the next victim.

    As more sites come online with ERAM 24×7 will there be an increased risk of a serious incident? On the automation side just reading the AIMS tickets opened and unresolved on track jumps and problems with the tracker will the controllers be able to mitigate all these issues?

    Again, a brilliant piece of writing!

  4. Article in AVSTOP:http://avstop.com/June_2012/faa_progress_and_challenges_to_consolidate_air_traffic_facilities.htm

    ERAM still has go through a Tech Refresh before it will be fully functional. But the FAA has no money, so they will privatize the EnRoute system which frees them from having to purchase the new equipment. Also as the article points out the FAA can no longer afford to maintain 20 separate centers plant and equipment.

    The ultimate goal of ERAM has always been center consolidation everything else has been smoke and mirrors. Hope you guys make out better the the FSS people did. NATCA has been played by a group of pro’s.

Leave a Reply to NMU Cancel reply

Your email address will not be published. Required fields are marked *