Dayan Viciedo singled off Happ to open the second inning and one out later, Marcus Semien hit his first career home run to left to give the White Sox a 2-0 lead.
Happ -- who had a 2.12 ERA (four earned runs in 17 innings pitched) over his previous three starts -- labored in that inning and escaped having allowed only two runs despite facing eight batters.
After striking out Josh Phegley, Happ allowed another single and two walks in the inning before getting Paul Konerko to fly out for the third out.
The White Sox put a runner in scoring position in each of the first four innings against Happ. In the fifth, he yielded another homer, this one a solo shot by Avisail Garcia to right-center that made it 3-1.
"The 0-2 [count], the first [home run], I just put it right there for him," Happ said of Semien's homer. "I made it easy on him."
"It's a great feeling," Semien said of hitting his first Major League home run. "I was really kind of nervous. Whenever you have a first-time anything in the big leagues, it's kind of a great feeling. I'm just happy about it."
Happ said the home run Garcia hit -- which proved to be the difference in the game -- was another mistake up in the zone.
"The second one I was behind in the count and left a fastball up," he said. "Flyballs tend to go out in this park, it seems like. [Garcia] definitely got enough of it and it turned out to be the winning run."
Happ scattered eight hits overall while walking three and striking out five. He said that even though the results show otherwise, he felt good and just made two mistakes on the season-high-tying two homers he surrendered.
He also said the new, lower arm slot he's been working on with pitching coach Pete Walker is still a work in progress, but that he still needs to execute his pitches.
"For the most part, I felt great," he said. "A couple of pitches over the plate and I guess that was enough [for the White Sox] tonight. … I felt good, but I just didn't execute a couple of times. It's tough to be 100 percent on when you're trying [a new arm slot] but there's no excuses. It's a big league game and you've got to be able to get guys out."
Blue Jays manager John Gibbons said he felt Happ pitched better than what his line reflected, but that he would've liked to have seen the left-hander mix in more of his repertoire.
"I thought his stuff was better than the results [showed]," Gibbons said. "That first inning he was really on, but I just think he needed to mix it up a little bit more, get his curveball going a little bit more, which makes his fastball that much better. … He was good early, and then he ran into trouble in [the second] inning. And then, of course, the big home run that Garcia put up."
The Blue Jays scored a run in the fifth on J.P. Arencibia's 21st home run of the season, his third of the year against the White Sox. In the eighth, Kevin Pillar led off with a double and Ryan Goins brought him in with a single to left to make it 3-2.
The Jays had a chance for more in the eighth, but with one out and runners at first and second, pinch-hitter Adam Lind grounded out softly to third and Brett Lawrie grounded out to short.
Toronto lost for only the fourth time in their last 11 road contests, and finished 3-4 in the season series against the White Sox. They have lost nine of their last 13 games overall.
The Blue Jays now head to Baltimore, where the Orioles are still hoping to sneak into the postseason. Gibbons said that while his club isn't going to the playoffs, it still means something when playing a team that still has a chance.
"We've still got a say in this thing," Gibbons said. "The next few series are against teams that are still in it. Baltimore's hanging on there. Of course, Tampa's in the lead for the Wild Card. So maybe we can still have a say."