ERCOT blackout: Changes already made to emergency procedures

A map showing the location where power plants went out unexpectedly on the morning of Fed. 2, 2011. (Image: ERCOT)

Texas’ main electric grid operator has already begun to make changes in how it will handle situations like the Feb. 2 power emergency that led to rolling blackouts through much of the state, according to the organization’s CEO.

An e-mail alert system designed to warn of escalating problems didn’t reach all the parties that it should have in time, partly because the emergency came to a head overnight, said Electric Reliability Council of Texas CEO Trip Doggett in a presentation to the ERCOT board of directors today. You can look at the presentation here.

The warning system now includes more automation, with messages being sent directly from ERCOT’s control room instead of through third parties.

An improved communication plan is also being developed that will use social media to spread information, set up a phone bank so more ERCOT staff can field media calls, include the potential for real-time radio and TV interviews during emergencies, and add real-time grid data in an easy-to-find location on ERCOT’s web site.

The e-mails will also include phone calls to cell phones and home phone numbers that won’t stop until the receiver answers and acknowledges the message.

Marcie Zlotnik, an ERCOT board member who is also COO of Houston-based electric retailer StarTex Power, said she felt comfortable with how ERCOT’s staff handled the emergency and averted a statewide blackout that could have taken days to recover from.

But Zlotnik said she’s comfortable only because of the level of information she has access to as a board member.

“When people aren’t informed, as was the case for most of the people out there, perception is all that matters, and the perception is that there was a ton of confusion and ERCOT didn’t know what it was doing,” Zlotnik said.

The Feb. 2 emergency was sparked by the unprecedented failure of dozens of power plants as subfreezing weather hit most of the state, Doggett said. Most of the failures – some 50 plants with 8,000 megawatts of generating capacity – happened in the early morning hours.

A map showing the location where power plants went out unexpectedly on the morning of Fed. 2, 2011. (Image: ERCOT)

Between 5:08 a.m. and 5:43 a.m. emergency status of the grid jumped three levels, leading to the call for rolling blackouts. That rapid escalation made it hard for officials to warn the media or other public officials of the rolling blackouts in advance.

ERCOT has asked the operators of the power plants that went down unexpectedly over the course of the power emergency to allow it to publicly release the names of the power plants. On Monday afternoon ERCOT was still sorting the responses from the operators, but Doggett shared a few more details.

While 8,000 megawatts of power was the peak outage, over the course of the day 82 power plants with a combined generating capacity of 11,000 megawatts went offline.

About 40 percent of the downed plants were coal-fired power plants, 59 percent gas units and 1 percent wind power.

There was no clear pattern in the age of the power plants being a factor, Doggett said, or in location as plants failed throughout the state. A map showing the rough location of power plants that went down shows a cluster of plants around Houston, a line of them through Central Texas and another line across North Texas.

While some natural gas fired power plants had problems with fuel supplies due to the cold, Doggett said the emergency wasn’t caused by gas curtailment issues. He said wind power units and nuclear power plants produced at levels they were expected to.

Fuel-type for the power plants that were offline unexpectedly at 6:30 a.m. on Feb. 2, 2011.

Many of the plant shutdowns were due to the failure of equipment that monitors different aspects of the power plants, such as temperatures, pressures and water levels, as well as the failure of some control systems.

A number of operators reported pneumatic lines with some water in them froze, disrupting operations.

Another common problem was the failure of a device called a drum level transmitter, which reads the levels of steam and water in power plant boilers. The piping between drum and a switch the device controls typically is wrapped with heated tape and insulated, but the high winds that accompanied the cold may have been too much for even those precautions.

A number of ERCOT board members seemed surprised to learn that information about how power plants prepare for cold weather is considered confidential data that is not readily shared.

“I cannot imagine any reason why that would be confidential,” said board member Alton Patton. “There’s no real-time information involved there.”

ERCOT vice chair Michehl Gent echoed that concern.

“If you think you need to have data, then ask for it,” Gent said to ERCOT staffers. “We’ll back you up on it.”