Tonight marks the Initial Operational Capability (IOC) of ERAM at my facility, Minneapolis ARTCC (ZMP). It’s the first test of ERAM on live traffic at ZMP, starting at around 11 PM local time.
I wish my air traffic control coworkers the best of luck during the test!
But not to worry, because I’m sure there will be lots of FAA managers earning overtime pay to “supervise” the affair. We had extra managers on duty when Salt Lake City (ZLC) went IOC a while ago.
Most importantly, as the last line of this memorandum from the ZMP ERAM Lead says:
“Cake will be served!”
The FAA is famous for having cakes as a way of self-congratulations for a job well done.
But that would mean they were celebrating before the test is even done, or for that matter, even started. I wonder if that means they’re planning on the test being a success…
Either that or it it’s intended as a bonus for the air traffic controllers who will be acting as FAA software beta testers tonight.
Please let us know how it goes, we haven’t heard anything from ZLC. Thanks.
Which office weenie wrote that crap? They are going from “first time use” to IOC after 5 hours of use from 2300-0400? So if it works for 5 hours sign off on it and bonuses and cake all around! I hope they have a big airplane on the cake…it would look so neat!
Just one issue for me. My pref-setting for all my high altitude sectors was labeled HI. When I invoked it all of the pref-setting in my list switched over to Chris Korkowski’s whose operating initials are HI.
I didn’t hear of any sectors freezing up but then we were only operating on it for maybe 2.5 hours. I’ve been told that most of the freeze-up issues apparently happen after a much longer period.
The cake wasn’t served until after reverting back to HOST. Strawberry filling….mmmmmm. I had 2 pieces!
atc freq, and all! especially pnshd!
i am the office weenie that wrote “that crap”, and have also wrote what i prefer to think of as interesting editorial bits (but to each their own) for the weekly newsletter here at zmp. i am constantly attempting to throw in a little bit of humor, regardless of how weak my attempts might be, and thus it is with the cake. i applaud atc for his efforts at providing information! and while it appears that his view of eram is more of the glass half empty, while mine is more that the glass is half full (or maybe even a little more at this point in time) i think it is important to have good discussion about the issues.
in addition, i was the “weenie” that thought of the idea and bought the cake with my own funds (glad that kevin g enjoyed it, he wasn’t the only one) although a few other weenies pitched in and helped; and my reasoning was centered more on being a bit tongue in cheek while at the same time providing a little extra something to those ZMP atc folks who happened to be there that night. eram is a new adventure, and has its share of new pratfalls as well, and what better way to celebrate growing pains than with a birthday cake?
however i am curious to know if atc or any contributors were around in the late ’80s when we brought Host in to replace the old IBM 9020. it had more than it’s share of growing pains after deployment, and most of the folks who know how to write code for it are well on their way to their second retirements. and just last night we installed another new Host system with yet more patches to fix problems that are being uncovered and have to be dealt with by controllers until those patches have been made. so while eram isn’t perfect, it isn’t a decrepit burned out hunk of junk either.
Cake Eater
PS – isn’t the first time, won’t be the last that pizza or cake or root beer floats or some other personal gesture of appreciation has been supplied by this “weenie” and it won’t be the last! we have a great group of atc folks here at zmp and i am proud to work with them.
PS #2 – most of our current transition team member are rather new to the experience, and i would be the last to call them “mind numbed”! they are a very conscientious and hard working crew, and they are doing a great job of keeping the interests of the atc folk here at zmp in the forefront of their efforts!
PS #3 – ok, i promise i’ll quit after this one. maybe i need to start my own blog? atc geek??? :c) have a good weekend all!
Thanks for the comment, Cake Eater!
But you’re not getting off the hook that easily. 🙂
I don’t buy the cake was “tongue in cheek” at all. Sheet cakes are the FAA’s way of celebrating. That’s why there were so many managers there for IOC. They showed up for the party (as usual).
It was a significant event to both the FAA and especially to the facility managers; because dates and deadlines are more important to them than performance (when it comes to pretty much anything other than air traffic controllers of course).
But the cake was a celebration of growing pains at a child’s birthday party. How about letting ERAM mature before setting it loose in the world?
Because ERAM is behind schedule the FAA is sending a child to do an adult’s work.
Ever heard the expression, “Close enough for government work”? Too many within the FAA work to that standard.
You say looking at ERAM is an issue of “glass half empty” or “glass half full”. I say it’s an issue of safety and performance.
ERAM isn’t up to the task yet, but it’s being ramrodded through nonetheless, apparently because there are too many that think it’s “close enough for government work.”
Putting a happy face on the ERAM project is simply denial and doesn’t make testing the software in its current state on live traffic any less reckless, irresponsible or unsafe.
And just because the FAA may have rolled the dice back when the HOST was installed doesn’t make it OK now with ERAM. (I wasn’t there for that, and I guarantee I would have said the same things I’m saying about ERAM if they handled it the same way.)
That’s the biggest difference between you and me: you see the ERAM project as business as usual and accept that.
I, on the other hand, while acknowledging it’s most certainly business as usual, know that it doesn’t have to be and that we could do better. I refuse to simply accept more FAA mediocrity, while realizing that the FAA revels in mediocrity, rewarding (and often promoting) employees that do the same.
(I also realize that I’m just swimming upstream though…)
Since the FAA loves to use race driver analogies for controllers I’ll use one now.
If I was a race car driver and you were my crew chief, you’d be knowingly and willingly sending me out in a car that you were aware had known technical problems, hoping that as the driver I could deal with whatever malfunctions the car might experience while on the track and somehow avoid killing myself or someone else.
But I’m assuming you wouldn’t expect me to win the race either, because you couldn’t possibly believe that sending me out in a car that you know doesn’t work right would put me in a position to win (unless all the other drivers were also driving broken cars). This is the mediocrity factor.
Sure you might feel badly if the car crashed because of some problem you knew about that manifested itself during the race, but as the driver I’d actually have to deal with it. At that point you’d just be a spectator.
(Wouldn’t a cake serve to ease your guilty conscience?…)
Would Boeing or Airbus deliver a fly-by-wire passenger jet to the airlines to load with passengers and fly with known software deficiencies?
But isn’t that precisely what the FAA is doing right now with ERAM?
Of course doing a job right means extra effort, extra time, and extra cost; none of which the FAA is interested in investing.
There are just too many people in the FAA, including controllers, that are just getting by, making the most of broken or inferior equipment or procedures, addressing or working around problems on the fly but never really fixing them properly once and for all. They accept mediocrity.
But don’t we have an obligation to the flying public to do the best job we can?
As an organization the FAA rarely does things the way they should, mostly because doing that would be more difficult. The FAA has a lazy, uncaring culture, an attitude that comes from the upper levels of the organization. The managers say one thing but then do something else (or more often than not, nothing at all).
The FAA talks “safety culture” and “safety is our passion”, but safety has always ultimately been the air traffic controllers’ problem; the rest of the FAA thinks safety just happens.
In the case of ERAM if it breaks while in use controllers will have to deal with it. None of the managers or support people are going to do anything to keep the airplanes apart.
So when it’s all said and done someone is going to owe controllers more than just some cake (nothing in my job description says anything about beta software tester) but I’m not holding my breath.
Undoubtedly any software in development is going to have growing pains. But rationalizing the development and testing of the ERAM (or any other safety system) software on live traffic and the flying public is really just “close enough for government work” thinking…
And I refuse to accept that as the way we should be doing things.
atc,
I like the Race Car analogy, but there are some pieces missing. First,our car is radically different than any of the existing race cars; but we need to be able to race with those old cars and some other new cars from other teams. So we, and the other two teams using the new car, have had the car on the test track and put countless miles on it. There have been probolems, and we have sent the car back in and had the problems worked many times saying: “We won’t let our full-time drivers sit in the driver seat, the car isn’t ready. In fact, if we had behaved as you suggest, we would have had you driving the car a year ago, when we were first scheduled to turn ERAM on. We have taken the car through its paces; I, and other members of the pit crew (including many controllers)have driven the car on the test track, first by itself, and then with some of the old cars, and also with one of the new cars. The whole car program has gone through a safety risk management process (that continues to this day)and had the race circuit officials declare that yes, this car is ready for actual races. There are still issues with the vehicle, but the crew (including the controllers on the crew)
Oops, hit the wrong key. Anyways, the whole crew is assured the car is safe enough to race, and some of the crew (CPC’s) have actully driven during some of the previous races; and are ready to jump in as an alternate driver in any current race if need be.
We continue to monitor both the vehicle’s performance as the race goes on, and also keep an eye on our drivers to make sure that we are not exceeding their capabilities with the car. We have only entered smaller races with fewer cars in them up to his point. Here at ZMP we already know that we aren’t taking this car down to Daytona yet; but we can’t just keep testing it without some real track experience to know the full capabilities of the vehicle, we need to at least be at Elko or Shakopee.
The race circuit officials also know that they can’t just allow everyone to run out and start using the new car in races yet either; but they also know that they can’t simulate everything that the car needs to be able to handle without actually getting it into a race. The car needs and uses too much functionality with both the old cars and other new cars, and needs to be allowed to utilize all of it’s own parameters in a fully functional state as well.
And since last June, when ZLC first started driving the new series of cars, the car itself has only had to be pulled off the track three times (and that is with undreds of hours of race time) before the end of a race. And only one of those three was so immediate that it caused any siginifcant disruption to the flow of all of the other cars in the race. And as of this writing neither ZSE or ZMP has had to pull their cars off the track for the races they have been in. And most importantly, there has been no crashes (loss of separation) in the 3 new cars or any of the many old cars that have been racing with us that are attributed to the new car.
As mentioned I do have to admit that we do keep finding new issues with the cars, but we are focused on letting ZLC move into the bigger races first, and working that discovery. But since our drivers and our tracks are slightly different than theirs, we need to continue racing.
We have had URET for many years, but just over a year ago found out that there was a significant problem with routing to DFW that could easily lead to an ops error in ZKC by entries made here at ZMP. At that point, it is too late to even pull the car off the track, we had to let the drivers know that for DFW they had to be very careful how they handled the URET car; it has been that way all along. We do not have one automated system that I am aware of that is 100% perfect, and I don’t expect we will.
We do depend on the drivers to keep the car running during the race; and the drivers here at ZMP have proved to be exceptional among drivers. There are lots of reasons for that, and one of the reasons is that the crews that get their car ready and keep it running during pit stops are also pretty exceptional, and they have the utmost concern for providing a safe car for their drivers, even when ZMP has been chosen to help lead the race car industry to a new level. We aren’t trying to put a new car on the track that is just good enough, we want a car that will enable our drivers to win the race, day in and day out. And the sooner we can get our drivers safely using the car, the sooner we can start providing input so as the car is further developed the safety and functional features that we deem most important here at ZMP get the best chance of being adapted as the standard. The crew here has already developed a lot of ways to test the car or ensure it is running optimally that have been adopted not only by those who are still just testing their cars, but in some cases by those who were testing and driving before us.
So I continue to appreciate your skepticism, but you do need to be aware that you have a lot of good folks across many disciplines (Lockheed Martin, Tech Ops, FAST, AT Support Offices, ERAM Program Office – most of whom aren’t “FAA Managers”) in your pit crew that despite rumors to the contrary, are providing you with the best new car available, and making sure it is ready for any of the races we plan on entering it in.
PS – And believe it or not, the cake started out as a joke for the very reason that you point out.
Cake Eater,
I’m not surprised that you choose to rationalize the way the ERAM project is being handled. After all, it’s your job to be involved in projects like this, and convincing yourself that it’s all fine would be central to you being able to sleep at night (like any of the FAA employees who do this sort of thing routinely).
I’m just not buying what you’re selling.
“Safety was never compromised.” We’ve heard it time and time again from the FAA.
The ERAM program may have gone through an FAA safety risk management process. But the FAA’s entire attitude and approach to safety management is broken.
I know that there are plenty of those within the FAA and Lockheed Martin that deliberately try to downgrade the severity of bugs within the ERAM software to “expedite” its deployment.
In the FAA, safety is only compromised if someone has a deal (or is killed). You echo that attitude by commenting about the fact that there has been no loss of separation due to ERAM.
If I drive intoxicated and manage to make it home without killing myself and/or someone else, was safety compromised during my drunken drive?
And if you believe driving intoxicated without incident isn’t compromising safety, then you will likely continue to drive intoxicated in the future.
As an organization the FAA is a repeat drunk driving offender (that only gets “caught” sometimes).
To continue with the ERAM race car analogy:
Lockheed Martin sold the FAA on the idea that they could deliver to them a winning race car (on time and on budget).
They’ve yet to do that.
Their “new model car” isn’t fit for racing, because it still can’t perform the basic driving functions it needs to stay moving in control and in the right direction on the road.
Meanwhile you’re assuring the drivers that their new car is safe to drive in spite of the fact that you know it has deficiencies. That’s a blatant contradiction.
I have no problem with running this new car on the test tracks.
But the operational ERAM runs on real air traffic aren’t tests. They’re races.
Last fall the wheels were falling off the ERAM car on the test track.
Now during ZLC’s first race the wheels fell off again.
I’m not skeptical of the ERAM project. ERAM doesn’t work properly; that’s a fact that you admit. You just figure we can work around its deficiencies.
If the wheels fall off the ERAM car and the driver can’t keep the car off the wall (like you assume they will be able to) there’s a possibility of injuries or death.
It’s pretty simple to me: until the wheels can stay on the new ERAM car it shouldn’t be tested on the flying public in “races”. I have no desire to be driving a significantly defective vehicle.
It’s clear that the ERAM project is being rushed into service before it’s ready, and that’s detrimental to the safety of the air traffic system.
But we’ve seen this before with the FAA (and by your own admission), so this really is just another example of the FAA’s true “roll the dice with safety” culture.
At the end of the day if no one is killed, the FAA will consider this another successful project.
So good luck to all of us.