The victory extended the Rays' season-best win streak to five games as they moved to 38-49. Tampa Bay will take a 6-1 record on its current road trip to Detroit for a four-game weekend series against the Tigers that begins Thursday night.
Sean Rodriguez, who had not played since Saturday in Baltimore, started at shortstop and quickly knocked off the rust with an RBI single in the fourth inning and a two-run home run in the sixth that traveled 445 feet over the center-field wall.
"How about Sean's homer? That's out of the old [Yankee Stadium] with regular dimensions in center field," Rays manager Joe Maddon said. "... He's hit home runs here in the past. He normally hits the ball pretty well in this ballpark."
Rodriguez's eighth home run of the season put the Rays up, 5-3.
"It felt good, I hit it clean," Rodriguez said. "... It feels good, not only to take advantage of it, but just being able to contribute. That's essentially what you're trying to do every game is just trying to find a way to contribute."
The Yankees opened the scoring when Brett Gardner led off their half of the first by homering to right off Rays starter Jake Odorizzi.
Against Yankees starter Vidal Nuno, who allowed four runs (three earned) on eight hits over five-plus innings, Kevin Kiermaier scored on a passed ball in the third to tie the score, but the Yankees answered in the bottom half of the inning when catcher Brian McCann redeemed himself for the passed ball with his 10th home run of the season.
Rodriguez's RBI single in the fourth tied the score at 2, but Gardner's RBI single in the bottom half of the inning put the Yankees up, 3-2.
"That was really frustrating," Odorizzi said of Gardner's hit. "Made my pitch, got the result I wanted, weak contact, it seemed to find [its] way through a lot of those times. Nothing you can do. Home runs happen, solos don't beat you, but [Gardner's RBI] was the one that kind of stung me the most."
Brandon Guyer's RBI single in the fifth drove home Desmond Jennings to tie the score at 3, and after Rodriguez's clutch blast, James Loney drove in the Rays' final run when he grounded out to first with the bases loaded in the ninth.
Odorizzi held the Yankees to three runs on eight hits in 5 2/3 innings to pick up his fourth win of the season.
"Jake pitched well enough," Maddon said. "He wasn't as sharp as he can be, but all of a sudden the curve became relevant with the game in progress, and that made it even better for him."
The Rays' bullpen was without Grant Balfour and Jake McGee, who were resting and unavailable Wednesday, but they certainly made it work.
Cesar Ramos retired Yangervis Solarte for the final out of the sixth. Juan Carlos Oviedo and Joel Peralta -- who made his club-record-tying 266th appearance as a pitcher -- posted scoreless frames in the seventh and eighth, respectively. Then Brad Boxberger tossed a scoreless ninth to preserve the win and earn his first save of the season.
"Bullpen: great," Maddon said. "Ramos got a big out of Solarte. Oviedo was outstanding, he had not pitched in four or five days, and then he came out and did that. Joel. Then, of course, Boxey at the end. ... We're building this combination with Jake and The Box."
The Yankees currently are on a different track than the Rays, losing for the fifth straight time and falling below .500.
"I thought we did swing the bats better today," Yankees manager Joe Girardi said. "Unfortunately, we didn't score enough runs, and some of our rallies were a couple outs and we never got a big hit."
Clearly, Maddon likes the way his team is playing and, particularly, the way everybody seems to be contributing.
"We're getting a different bus driver on a daily basis," Maddon said. "That's kind of neat. That's what we need. That's who we have to be. We're not necessarily built around one guy. We're built as a group, and everybody's got to do their job. And that's the concept I've been trying to promote among them."