It marked the sixth loss in the past seven games for Chicago, which is in a fight with Detroit for the rights to the American League Central crown.
Kluber has had a hand in the division race, considering his two wins this month have come against the Tigers and White Sox. In Tuesday's victory, Tribe first baseman Russ Canzler got into the act by going 3-for-4 at the plate with a home run.
The White Sox and Indians will square off four more times down the stretch.
"It's fun right now for us trying to play the spoiler role," Canzler said. "If we keep playing good baseball, the next week is going to be a lot of fun."
For the 26-year-old Kluber, the win was his second in 11 outings for the Indians (64-91) this season. The right-hander has endured an up-and-down stay in the big leagues -- Kluber has six starts with five innings or fewer logged to go along with five quality starts -- since being promoted from Triple-A Columbus in early August.
On Tuesday, Cleveland's offense spotted him a four-run lead with a solid showing against Chicago lefty Francisco Liriano. Canzler launched a solo home run off Liriano in the second inning and later doubled to help ignite a three-run push for the Tribe in the fourth inning.
Liriano (6-12) took the loss after yielding four runs on seven hits in 3 2/3 innings.
The first run in the fourth came on a would-be double-play grounder from Thomas Neal, who chopped a pitch from Liriano to second baseman Gordon Beckham. Neal narrowly beat the throw to first, though, and Cleveland went on to add two more runs on base hits from Ezequiel Carrera and Shin-Soo Choo.
"It happens," White Sox catcher A.J. Pierzynski said. "Have to get the next guy. We didn't do that and it led to two more runs. That's it. I didn't see a replay, but I know it was close. That's the way it goes and you just have to get the next guy."
In the fifth inning, Pierzynski put the White Sox (82-72) on the board by belting an 0-1 offering from Kluber out to right field for his 27th home run. After that leadoff shot, Dayan Viciedo sent a 1-1 pitch deep to left for his 22nd homer, trimming the Tribe's lead to 4-2.
Indians pitching coach Ruben Niebla jogged to the mound for a brief pep talk, and Kluber focused on finding a way not to cave under the pressure.
"Finishing off that inning was an important part of the game," Kluber said. "I was trying to tell myself -- and Ruben came out and said it, too, basically -- that those were the only two mistakes that I made up to that point.
"If I got back to what I was doing beforehand, I'd had good success with it so far. It was basically, 'Don't dwell on those. Go back to what you were doing earlier.'"
The righty issued a one-out walk to Beckham, but proceeded to strike out both Alejandro De Aza and Kevin Youkilis to escape further harm in the fifth. Kluber (2-4) then breezed through the next six batters he faced, ending with seven strong innings with only the two runs allowed on four hits.
"He gave up those two homers and then just settled down," Acta said. "He showed some moxie."
It was that type of performance in the Minors that convinced the Tribe to test Kluber in The Show in the first place.
"I've seen him pitch like that a bunch of time this year in Columbus," Canzler said, "where he just kind of shuts teams down and uses all his pitches for strikes. It seemed like that's what he was doing today."
Kluber's showing set the stage for setup man Vinnie Pestano to get a bit of redemption.
After surrendering a game-changing three-run home run to White Sox slugger Adam Dunn in the eighth inning of Monday's 5-4 loss, Pestano was asked to handle the eighth again on Tuesday. The righty did so without incident, even striking out Dunn with one runner aboard to end the inning.
"As a reliever, that's what you want," Indians closer Chris Perez said. "I know he had a really bad taste in his mouth last night. As a reliever, the good ones, that's what you want, to come back the next day and get the same guys out. He did that."
Perez allowed a solo home run to Paul Konerko and walked two in the ninth inning, but held on to collect his career-best 37th save.
It was Kluber who led the way, and the rookie showed something in the process.
"He pitched well," Acta said. "He was able to get through it."