Middlesbrough manager Tony Mowbray rued his side's inability to cope with pressure for a full 90 minutes after crashing out of the FA Cup.
Shaun Harrad struck in injury time as Burton caused a FA Cup upset at the Pirelli Stadium and piled more misery on Boro.
It looked like the Teessiders had done enough when Gary O'Neil put them ahead after 58 minutes.
However, Mowbray's side were stunned by two late goals from Shaun Harrad which saw them eliminated from the competition.
Mowbray said: "Last minute goals are killing us.
"It happened at Elland Road last weekend and it's happened again today.
"If we could have taken a better share of our chances it wouldn't really have mattered. They wouldn't have gambled like they did because they would have been two or three goals down and out of it.
"But when a team is only one goal down, they'll always put you under pressure in the last few minutes.
"It's another example of one goal never being enough.
"I don't even think it was a case of them putting us under any sort of pressure. It was just two isolated incidents.
"Even for the seven or eight minutes between their two goals there was only one team trying to score a winner – and that was us.
"Then they get a free kick in the 93rd minute and they scored from it.
"It was a case of two shots and two goals for Burton."
Boro made five changes from the draw against Norwich. Jason Steele, Barry Robson and Marvin Emnes missed out through injury, Tarmo Kink dropped down to the bench while speculation continued to surround the future of David Wheater.
Danny Coyne came in for his first game since the opening day of the season, Tony McMahon returned to right-back and Justin Hoyte moved to the right of midfield. Seb Hines replaced Wheater.
Kris Boyd came in for Kink and the other change made by Boro boss Tony Mowbray was the inclusion of O'Neil.
Burton, playing in the third round for the first time since famously taking Manchester United to a replay in 2006, made only one change as leading scorer Harrad replaced James Collins.
The two teams were separated by 45 league places prior to kick-off, but in an end-to-end first-half that was not immediately obvious.
Albion forced Boro back early on when a misdirected pass from Nick Bailey was seized upon, but Hines read the break well and cleared the danger.
At the other end, Boyd's free-kick was pushed away by Adam Legzdins.
Boro stepped up a gear after the restart and after McMahon struck the crossbar with an 18-yard shot at the beginning of the second half, O'Neil put Mowbray's men ahead in the 58th minute.
O'Neil, making his first appearance of 2011 after illness, put the ball in the top left-hand corner from 25 yards out after Leroy Lita was brought down by Russell Penn.
Lita and Body wasted good chances when well-placed inside the area and Boro were made to pay as Harrad stole the headlines.
The striker, who is out of contract at the end of the season and looking for a move to a Championship club, suggested he might be able to cut it at that level with two tidy finishes.
The first came after 82 minutes when he turned home Penn's cross from 12 yards out.
Harrad then won the tie for Burton in the fourth-minute of injury time, smashing a shot past Coyne after Scott Malone's free-kick was deflected into his path.
Burton manager Paul Peschisolido was "absolutely overjoyed" after guaranteeing the Brewers a place in the fourth round of the FA Cup for the first time in their history.
He said: "I feel fantastic, but if you'd have asked going into the last 10 minutes of the game I wouldn't have said so. I actually thought we were terrible today. We just didn't perform.
"Perhaps we didn't deserve it, but I'm not complaining because we're in the hat for the next round - and that's unbelievable.
"Obviously, I'm absolutely overjoyed that we've got through. It's a fantastic result for the club and the fans."