Pittsburgh has won five of its last eight road games and now trails first-place Cincinnati by just two games in the National League Central Division standings.
Thursday's game had its share of drama.
After Ryan Ludwick's second home run of the night, a 431-foot blast that tied the score off Pirates closer Joel Hanrahan in the ninth, in jogged Chapman, who already owned a Reds franchise record with 24 appearances without an earned run allowed to start the season.
"You knew it was going to happen sooner or later," said Reds manager Dusty Baker. "You just didn't want it to happen tonight. Not like that."
McKenry fell behind Chapman 0-2 before taking a 100-mph fastball for a ball. The next pitch was middle-away at 99 mph, and McKenry lined it for a double to put the Pirates ahead to stay. Chapman (4-1) lost for the first time this season.
"You still have butterflies after a game like that," McKenry said. "[Chapman] is a special arm. You just have to go up there with the mindset to battle and put the ball in play. You have to prepare for the 100-mph heater. I just reacted to it."
So did the Pirates' dugout, which collectively erupted when Barmes crossed the plate.
"I can't script these games," said Hurdle. "This is probably as satisfying a win as I've had here. I just like the way we were able to scratch and claw."
Lost in the drama was a solid outing by starter Kevin Correia, who allowed three runs on eight hits, walked one and struck out four in five innings. But he was long gone when the real drama commenced.
Early on, it was a pitcher's type game with Correia matching wits with Cincinnati's 24-year old right-hander Mike Leake, who retired the first nine batters he faced, five via strikeout.
The Reds made a bid to take the lead in the first inning when Joey Votto attempted to score from second on Brandon Phillips' single. But right fielder Jose Tabata, starting for the first time since Sunday due to left leg tightness, threw out Votto at the plate on a perfect throw to McKenry.
Jay Bruce hit his 13th home run on Correia's first pitch in the second inning to put Cincinnati ahead, 1-0.
In the fourth, Alex Presley led off with a double to right. Neil Walker's RBI double tied the score, 1-1. Garrett Jones' sacrifice fly to center scored Walker to put the Pirates ahead, 2-1.
Correia had retired eight straight following Bruce's home run. But for the second time in three innings, he surrendered a solo home run on his first pitch of an inning when Ludwick launched a 420-foot homer into the bleachers to tie the score, 2-2.
For the second straight night, Reds center fielder Chris Heisey robbed a Pirate of extra bases when he made a fully-extended diving grab of Andrew McCutchen's liner in the sixth. But Jones followed Heisey's defensive gem with a single to left, driving home Presley to put the Pirates ahead, 3-2.
Again the Reds jumped on Correia at the start of an inning with three straight singles to begin the sixth. Bruce's RBI hit tied the score, 3-3.
But Jared Hughes replaced Correia and escaped a first-and-third, nobody-out situation keeping the game tied. "That's good stuff by a rookie," Hurdle said.
Heads-up base running by Tabata helped the Pirates regain the lead in the seventh.
Tabata reached on an infield single. He advanced to second on a sacrifice bunt by Barmes, but with Phillips' back turned momentarily after retiring Barmes at first, Tabata raced toward third and slid in safely. He scored moments later on McKenry's sacrifice fly.
Three scoreless innings by the bullpen helped the Pirates maintain the one-run lead going into the ninth. Ludwick's solo shot off Hanrahan tied the score and electrified the crowd before McKenry's clutch hit off Chapman silenced them.
The Reds weren't done. Votto doubled leading off the 10th off Chris Resop. But with runners on first and second and two outs, Resop struck out Ludwick to earn his first save.
"We had chances, big-time, to at least tie that game up or possibly go ahead," said Baker. "It's tough to take, but they played us hard this series."
Hanrahan (3-0) earned the win, despite his blown save. "Good stuff all over the joint tonight," Hurdle said.