"Nobody likes losing," White Sox manager Robin Ventura said. "I don't know if it takes any monkey off your back because the record still stinks. You just go and play for today and you're trying to do things to win games. There was a lot of good things today. Dunn hit the homer. We're seeing what the young guys can do. That where we're at right now."
Rienzo (2-1), who was called up from Triple-A Charlotte on July 30, allowed one run on four hits with four strikeouts and two walks over 6 2/3 innings.
Dunn got his 31st homer and the second of his career to land on Eutaw Street just beyond the right-field scoreboard at Oriole Park at Camden Yards.
Orioles starter Bud Norris (10-11) struggled against a patient White Sox lineup. The righty allowed four runs on nine hits with six strikeouts and one walk over 4 1/3 innings and 100 pitches.
Adam Jones picked up his 31st home run for the Orioles. Baltimore hit eight solo homers in the series, but went just 3-for-37 with runners in scoring position.
"He had a lot of baserunners out there," Orioles manager Buck Showalter said of Norris. "They left a bunch of people out there, too, and have all series. But it's not like we can just throw our hats out there and drive in runs and get a hit."
Chicago took a 1-0 lead in the second on a one-out double by Dayan Viciedo that scored Conor Gillaspie. The White Sox then took a 2-0 advantage in the fourth when Alejandro De Aza hit a sharp grounder with the bases loaded that Orioles second baseman Ryan Flaherty knocked down but could not get to quickly enough to make a throw to first.
"The one thing on my mind today is I wanted to make the losing streak over," Rienzo said. "Nobody likes to lose. The clubhouse is more happy when we win."
Jones pulled the Orioles to within a run in the fourth when he hit a pitch from Rienzo just below the knees over the right-center fence. However, Dunn answered an inning later when he crushed a changeup from Norris that travelled 424 feet for a 4-1 lead.
One day after allowing the winning runs in the 10th inning, Addison Reed loaded the bases and gave up a run in the ninth before closing out his 37th save.
"I think the best thing about the whole thing is that we came out of it with a win," Reed said. "Obviously, we've been kind of struggling as of late, and just to get a win shows that we're not giving up. We're still out here, we're trying to win every game and things worked out for us today. I had a little bit of command issue but found a way to get out of it and come out of here with a win."
The victory was key for the White Sox to try to gain some momentum with a schedule that does not get any easier. Chicago opens a three-game series with first-place Detroit on Monday.
"It always feels good to win," said third baseman Jeff Keppinger, who went 1-for-4 with a double and a run. "We're out here to compete and win baseball games. It was a really bad road trip for us, but to be able to go home knowing that we at least scratched one across is good."