Or if you’d prefer a different title:
How a Week on the Inside Convinced Me of the Limitations of Government
Technically, I’ve worked inside the U.S. government before – I interned for the Army Corps of Engineers and was regailed for months by my mentor, a former Lieutenant of the Army and Air Force (both times doing weather recon and forecasting for military operations), about the spectacular inefficiency and politically-motivated arm-twisting that went on during his 22 year military weather career. As a result of his experiences, he became convinced that, while many individual forecasters within the NWS and other weather-related NOAA offices intended to do good and save lives, the top-down philosophical mandate of NOAA (as enforced by DOC officials and other political leadership) literally prevented most of them from truly forecasting. I’ve commented here in the past regarding the strange-seeming decisions made by the National Hurricane Center or the Climate Prediction Center that were, at best, self-serving verification hedges (if they didn’t have a watch area out, they never named the storm, even if it had all of the appropriate characteristics, e.g.)…and at worst, motivated by a desire to keep up funding. I was just guessing, however. I had no idea how close to right I was – in fact, there were days when the more sensible part of my brain told me to give those guys a break and the benefit of the doubt alongside.
What now follows are a series of anecdotes I experienced while shadowing a forecast office (who shall remained unnamed, since I actually highly respect this particular office for staying largely away from politicking and forecast gamesmanship) – some stories from other offices, some personally experienced moments of shear terror. These stories will serve to shed light on precisely what’s wrong with NOAA – on why we are no longer the premiere location for scientific advances in weather forecasting and why innovations within the NOAA branches seem counterproductive these days. And I hope they will also make a point regarding the general nature of governmental inefficiency and ineffectiveness.
(Cut for length.)
“WEATHER-READY NATION”
The Obama administratio’s top officials within the Commerce Department have recently been driving a top-down program called “weather-ready nation.” The stated goal of this program is to improve the dissemination of accurate and timely forecasts, warnings and emergency support services to the public through collaboration between many NOAA offices and public relations personnel. Sounds like a good idea, right? To hear about it from the folks at the particular field office, however, “Weather-Ready Nation” was a laughingstock. Why were the operational meteorologists and research scientists working here so skeptical?
They recalled a similar pronouncement by Clinton’s DOC made on January 14th, 2000 – this pilot program was called the “No-Surprise Weather Service.” That buzzphrase represented government funds meant to go toward improving communications between the weather service and their public/media customers regarding the uncertainty and range of possible outcomes for a given high impact weather event, with a special emphasis on improving our ability to forecast threats in the longer ranges (3+ days). People who live on the East Coast may remember that on January 24th, 2000…just ten days after the big roll-out of the “No-Surprises Weather Service” campaign, our models completely whiffed on forecasting a major Nor’easter which dumped 10-20 inches of snow on heavily populated regions from South Carolina to Central New York, including the DC area. The storm, in memory of the hilarious irony, was dubbed the “Surprise Snowstorm of 2000.”
The mets there believe, you see, that every penny spent flogging campaigns designed to send a politically motivated core message and/or push decision support services beyond their real limits of reliability (meaning asking the mets to do impossible things that their customers aren’t even requesting) is a penny not being spent on weather research and (especially) infrastructure improvements. It turns out that a huge chunk of the budget being spent on “Weather-Ready Nation” – which is more of a catchy slogan than a feasible and productive mandate – came out of the budget for a planned major upgrade to NCEP’s supercomputer mainframe. That would be the big room full of servers that cranks out our weather forecast model output four times a day.
For a little further background, the European Center for Medium-range Weather Forecasting currently has approximately 80 times our computing power. They produce a global forecast on a 12-km grid, while our best global forecast is on a 36 km grid. They run a 50-member 40 km global ensemble (a series of model runs driven by the same basic physics but with slightly different initial conditions in each run)…we run a 20-member 80 km global ensemble. Every week, they re-run their model with its’ most modern physics for the entire period from 1990 to present so that they have an up-to-date model climatology to compare with observations and future forecasts…at 40 km resolution. We have a 2.5×2.5 degree (~250 km!) global reanalysis that does not include any actual reforecasts…only our best guess at a verification. They initialize their model with state-of-the-art dynamic 4-dimensional analysis of the ingest data (satellite, surface and upper air observations, etc)…we are stil doing static variance analysis in 3-dimensions. And the leaders of DOC sacrificed the supercomputer budget…to…hold a series of town hall meetings about how best to communicate model ucnertainty and disseminate warnings to Homeland Security??
One of those town hall meetings was held the third day of my trip and after it was over (I was not invited since I’m not a NOAA employee), every met in the office returned infuriated by the needless politicking. The seasoned officers reported blantant collusion on the part of the NOAA public relations committee to plant favorable questions into the discussion and silence critics of NOAA’s ne3w policies (we’ll talk about one such policy shortly). Your tax dollars hard at work, folks.
“NO CREATIVITY ALLOWED”
Forecasters mostly have a strong background in the science of meteorology and many of them have some computer science background as well. So when they are routinely faced with problems that cannot be solved by the one-size-fits-all AWIPS terminal (a product of government standardization in the name of attempting to improve forecaster efficiency by focusing their weather and model viewing through one display system), they write computer applications to display the things they need. About two weeks ago, so say the staff at the office I visited, a directive came down from NOAA brass forbidding forecasters from making their applications public, even to other forecast offices (!)…our best guess is that they made this decision because they didn’t want forecasting methods for which there had been no oversight to become common, because if someone using an unapproved technique busted, they might be blamed for non-standard practices leading to a loss of money.
This was one of the topics at the aforementioned town hall meeting and, because the NOAA PR folks knew their decision was wildly unpopular, they planted a question to start the discussion and quickly changed topics when the room didn’t buy their reasoning. Bottom line…unless whatever trick you’re using to get a better forecast has been proven in accepted scientific literature and the admin types have come to understand what you’re doing and approve of it…you aren’t allowed to do it. And you certainly arent’ allowed to use government resources and time to write the code for it.
Incidentally…having seen AWIPS in use, I can tell you that it is clunky, frequently tempramental, monumentally resource inefficient and, IMHO, limited in utility compared to the panoply of web sites with more useful diagnostics that can be viewed to make a forecast. Standardizing isn’t always a good thing…each office has unique challenges and those challenges could be addressed by each office as best as they see fit.
“THAT’S MY TURF!”
If I haven’t relayed the frustrations of private sector meteorologists regarding NOAA’s position on competing forecasts, let me take a moment and do it now. I’ve watched several private meteorologists associated with successful private weather firms grumble about things like the National Hurricane Center’s decree that they should be the only ones giving forecast information regarding hurricanes, because competing (and differing) forecasts could lead to “public confusion.” Never mind that NOAA is the only firm issuing warnings and advisories in the ciritical times when customers must make decisions regarding preparedness…never mind that NOAA’s forecast is the only one backed by government authority and therefore the only one that can order an evacuation or activate FEMA…any competing forecasts are bad. The government wants a monopoly on weather and climate information (the climate aspect of this has been covered extensively at RightFans). But the monopolistic imperial mindset is not limited to public vs. private.
The various offices have their own budgets and eac budget is doled out based on the admin’s perceptions about the relevance of that office’s work. What therefore happens is that collaboration is actively discouraged despite top-down mandates that it should occur. The Hurricane Center is uninterested in working with the Ocean Prediction Center on a surge and wave forecast for a hurricane, or in coordinating with NWS offices affected by a landfalling storm on producing a single impact forecast including landfall, rain, wind, etc. The Climate Prediction Center and the Hydrometeorological Prediction Center both work on medium range weather forecasting – especially threat assessment…and there is austensibly a collaboration regarding threat maps and forecast discussions…but their joint forecast discussions are frequently (so say observers) pissing contests between rival factions touting their brand of forecasting tools. There is a huge war going on between the NCDC and NASA GISS over who should take possession of the US climate records. The various NWS branches must prove that they are needed and can’t be consolidated into fewer regions.
And it gets even sillier. The satellite/remote sensing comunity within NOAA and the modeling group at NCEP are actively working to wrestle funds away from operations and convince Congress that the human forecaster is less relevant now that our models are state of the art while the human forecasters and their union reps are lobbying congress to downsize the satellite division. And even within offices, I’ve heard stories of researchers actively undermining their colleagues’ potentially important work because it’s eating up a lot of the budget. Not to mention insane things like employees getting yelled at for helping their colleagues meet forecasting deadlines or clipping the hedges in their down time for fun rather than letting the union landscapers do it. One forecaster I spoke with at length said that he left another office after one year because it was “a living hell of territorial disputes and rock/hard place dilemmas.”
“COMMITTEE FINDINGS: A BLUE RIBBON PANEL IS NEEDED”
Say you have a new application or scientific paper or clever forecasting idea that you want to push forward to help your colleagues. Well, in the private sector, you pitch your idea at a board meeting, they hold an up or down vote or some sort of committee reviews your findings…and a decision is made. That’s rather painful, but in the government, you file a report with your superior officer, they file a report with their branch chief, he files a report to the congressional budget office or to NOAA’s bean counters. They convene a committee to decide WHETHER THE IDEA SHOULD BE CONSIDERED…not whether it should be implemented…and then if they agree…they form a commission/panel to look at it and run years of experiments to prove its efficacy.
A discussion broke out about the problems the coders had been having getting snow/liquid water ratios to be accurate in the models while I was eating lunch one day. I rattled off, based on my knowledge of microphysics from my fellow group-member who’s taking observations of snow crystals and properties in winter storms, a list of five different ways they could improve their modeled ratios. The branch chief, who was sitting in with us since our experimental group was a high priority big budget event for his office, liked every single one of those ideas from a scientific perspective, but suggested that each one would have to be tested independently and vetted by his bosses and their committees…and that to avoid pain and bad forecasts in the short run, he was going to recommend they use site specific climatology (the average ratio of snow to liquid water for each location on record) to populate the model.
“JUST THE FACTS (THAT FIT), MA’AM”
I sat in on a seminar that was about the impact of extra flight-level observations on the forecasted track and intensity of hurricanes. The lady giving the presentation had done a ton of hard work that involved taking the extra observations out of the model initial analyses and then rerunning it a bunch of times to try to prove that extra observations improved the forecast. She found that they had no such impact for her three case study events. Now she was careful to point out possible reasons for the lack of obvious track adjustments, and some of those caveats were indeed valid. But you know what the main concern was at this office? I’ll quote the first and fourth people to speak after she was done talking and asked for questions:
“I think you really need to be careful with the wording of your conclusions there. You know the climate in Congress right now…if they think our forecasts will be just as effective without additional observations, you can kiss that money goodbye and we’ll be observing hurricanes with satellite only.”
(that wasn’t even a question…it was a comment…and a politically charged one at that)…and…
“I think your recommendation should be to increase observation intervals within the storm and upstream of it…we’re obviously not getting enoguh extra data if what little we collected isn’t having an effect.”
Awesome. Forget the scientific method when money is on the line, baby.
“NOT MY PROBLEM”
And every day, I heard a new story about a really poor meteorologist or technician who couldn’t be fired because the union prevented it and the government didn’t care as long as it didn’t influence the final result. The typical solution to a bad employee has been to turf them off to another office or put them in charge of less important tasks. Like public school teachers and postal workers, bad employees frequently get paid to do nothing.
“CONCLUDING REMARKS”
Now you might all be tempted to think of spectacularly inefficient, politically charged private sector corporations that failed just as well as the NWS and NOAA are failing. Or you might be tempted to point to all fo the positives that have come out of government-run weather forecasting in the US. You would be right to do so in both cases…however:
Originally successful companies that fail always fail from the same types of in-fighting, political spin-doctoring, inefficiency and malaise that dominates every sector of American governance today (unless they fail due to incompetence…which is also a big problem in government). And frequently those sorts of inefficiences and in-fights come about because the government literally gets into the board rooms and stock holders meetings by investing.
None of what I’ve said in this piece should be read as a criticism of the bulk of the employees at the NWS or any of the NOAA branches. Most of them are good people and good scientists doing the best they can…and most of them are painfully aware of how badly mismanaged their organization actually is. The NWS has a long history of leading the field in innovation, in forecasting skill, and in public correspondence and transparency. That reputation si now eroding, but its’ accomplishments are nearly legendary. The problem is…they happened, by in large, when the NWS was a minor agency within another somewhat minor agency within an often-overlooked cabinet-level department (Commerce). Politics didn’t start screwing with the Weather Service as we know it until fairly recently (30-40 years ago)…and the more the NWS tries to innovate under the weight of the political machine that created it…the more it stagnates under institutional (one size fits all) thinking, territorialism, and waste. In many ways, NOAA (and NASA, for that matter) are microcosms of the entire American experience. Government science can do wonderful GLORIOUS things…when the entire government machine is bent on doing those things (and usually at a cost higher than necessary). We went to the moon using a calculator to plan our course. We standardized the format of upper air observing and surface station management. We learned to forecast the weather using sophisticated computer models. These are great things spearheaded by the United States. But the government model has an upper limit on productivity…private sector growth is literally unbounded if the ideas are good and the business plan is strong.
The decay of NASA and NOAA should be mourned…but it certainly shouldn’t be a surprise. It’s time to consider alternative strategies…it’s time to encourage innovation, cost prudence, customer satisfaction and infrastructure. Until NOAA is liberated from government overregulation and restraint…we will never catch up with the Europeans – whose office, incidentally, is run like a business and turns a profit each year.
Food for thought.
everymatt 