The go-ahead blast came on the first pitch of the eighth, an 88-mph cutter from Atlanta reliever Chad Durbin.
"He's been looking better and better every day," manager Davey Johnson said. "In Colorado, he started to look more relaxed, more comfortable in there. Shoot, he's back getting easy hits, so it's nice to have him back."
Tyler Clippard worked out of a jam in the ninth for his 13th save in as many opportunities. This one came in a game where the Nats knew the heat -- it was 101 degrees at the time of the first pitch -- was on.
"We've been playing a lot of close games, and we're comfortable in those games, but this atmosphere tonight was a lot like a playoff atmosphere," Clippard said. "It's huge for us, especially since we're a young team and we need these games to give us confidence down the stretch."
Washington left-hander Ross Detwiler, making his second start since rejoining the rotation after a month-long stint in the bullpen, was superb through six innings before unraveling in the seventh.
He allowed just two hits through the first six frames before he hit Freddie Freeman, misplayed a bunt and balked with one out in the seventh. Pinch-hitter Martin Prado then delivered an RBI single, breaking Detwiler's shutout. The next batter, Michael Bourn, hit a sacrifice fly to left, scoring Jack Wilson, to make it 4-2.
"The biggest thing that sticks out is I had [Freeman] with two strikes, and I tried to throw a curveball and hit him with it," Detwiler said.
Johnson left Detwiler in the game with two outs to face Andrelton Simmons, who knotted the game at 4 with his third home run of the year, a shot to left field that came on a 1-2 slider that Detwiler left over the plate.
"It's a shame he made one bad pitch and it cost him the ballgame," Johnson said. "But it was a strong effort. And we needed it so bad, I can't even tell you. Our bullpen was kind of beat up."
It negated the four runs that Washington scored in the first three frames -- the first coming on Detwiler's first hit of the season, an RBI single that made it 1-0 -- helped by some fielding miscues on the part of the Atlanta defense.
Washington scored twice in the third on an Ian Desmond single after two Braves errors and an infield single, then again in the fourth on a Jesus Flores home run.
But it set the table for Morse's heroics in the eighth.
"It's first pitch, and he's 3-for-3 at that point, he's going to be aggressive trying to get that fourth one, maybe try to leave the park," Durbin said. "You don't want to get behind, either, with a guy that's swinging. It was middle-middle, probably middle-away just a little bit. You want to make a little bit better pitch, but until they hit it out or hit one in the gap, it's not all that bad."
Morse, who hit .303 last year with 31 homers and 95 RBIs in 146 games, has slowly been working his way back into midseason form over the last week or so.
"I'm starting to have better at-bats," he said. "I'm starting to see pitches, recognize pitches. It comes with, basically, at-bats -- seeing the ball more and just being at the plate, up at the plate, watching the pitcher."
Atlanta starter Randall Delgado lasted just four frames, surrendering four runs (two earned) on eight hits while striking out three. Washington reliever Sean Burnett earned the victory, his first of the year.
With the win, Washington maintains its hold on the top spot in the National League East and moves to 5-1 against Atlanta this season, including 4-0 at Turner Field.
"Any time you play a division team, it's important -- especially when you're in first place like we are," Morse said. "Our division games are the most important games."