"It was certainly frustrating to give up a walk, a bloop and a blast [in the fifth]," Happ said.
There were frustrations on the offensive side of the ball, too, where the Astros didn't make good use of 10 hits and four walks. They had runners tagged out at the plate, third base and second base and hit into three double plays, going 2-for-8 with runners in scoring position.
The loss dropped the Astros to 9-27 on the road.
"We had runners all over the place, there's no doubt, and the guys were putting pretty good at-bats together off [Cubs starter Matt Garza]," Astros manager Brad Mills said.
The game could have been the last for Astros slugger Carlos Lee, who is mulling whether to accept a trade to the Dodgers. Lee went 0-for-4 and hit into a double play and said after the game he's going to decide by Sunday so he can clear his head.
"Either I'm going or I'm not," he said.
Happ carried a 2-0 lead into the fifth when he walked No. 8 hitter Luis Valbuena to start the inning. He was sacrificed to second and scored on a single to right by Starlin Castro one out later, and Rizzo then launched an 0-1 pitch into the right-field stands to put the Cubs ahead, 3-2.
"It just feels good to get that win, and get that monkey off my back to get that home run," said Rizzo, who hit his first homer as a member of the Cubs.
Happ singled to lead off the fifth and was on base for the entire inning on a hot afternoon, which Mills thought might have tired him out a little bit.
"He left off the inning with a single and had to stay out there on the bases and then came in and walked the leadoff hitter and that wound up really costing him," Mills said. "That was tough. Other than that, he threw the ball really well and battled all the way through and really did a good job."
The Astros had nine hits in 5 1/3 innings against Garza -- matching the team's hit total from the previous three games combined. Jordan Schafer's RBI single in the third scored J.D. Martinez, and Houston took a 2-0 lead in the fourth on an RBI single by Martinez.
"It was a day when you had to battle the elements," Garza said. "It was a warm one. I felt like in the second inning, I was wearing a suit made out of water."
But for the second game in a row, some bad base running hurt the Astros.
Martinez was tagged out to end the fourth trying to go from first to third base on a Scott Moore single, and Chris Johnson was thrown out at third trying to advance from first on a single by Jason Castro in the sixth.
Martinez went halfway to third and then turned back towards second, where he was tagged out easily.
"I'm not exactly sure what J.D. was doing," Mills said. "C.J., I don't mind that as much. He was trying to bust his tail to get to third with one out with [left fielder Alfonso] Soriano, who's known not to have the best arm around and he was going to his left. I'm not exactly sure what happened to J.D. because he was halfway to third and he stopped."
Johnson made a hard slide into third and appeared to beat the tag.
"It was an aggressive play, and I just tried to push the envelope a little bit," he said. "We want to be aggressive. We don't want to sit there and play station-to-station. We know who's out there and where guys are positioned. I wanted to take a chance because second and third with one out is a big difference."
The Astros managed only one base runner the rest of the way as five Cubs relievers combined to hold Houston to one hit and one walk in 4 2/3 innings. Carlos Marmol overcame a ninth-inning walk to secure his fifth save in his last eight outings.
"I think we swung the bats fairly well today," said Johnson, who went 2-for-3 with a walk. "We stuck with our plan of making Garza try to throw a lot of pitches and I think we did a pretty good job with that. I know Happ's not excited, but we think he threw well. Hopefully tomorrow we can come out and score some runs and get a W."