It also marked the end of a seven-game road trip in which the Sox won the first and last games, but lost the five in between.
A day later, Ortiz got his 25th home run, the one that eluded him Saturday when a fan reached for his line drive and brought it over the wall before it could get there on its own for what was ruled a fan-interference double.
"I've never hit a ball like that, that creates controversy," said Ortiz. "Make sure it goes way over the fence so I don't have to deal with that [stuff]."
Tampa Bay starter Chris Archer, who served up the home run, did not enjoy Ortiz's emphatic bat flip, and added some fuel to the fire in the recent controversy between Ortiz and Rays ace David Price.
"I think it was a perfect example of what Price said," said Archer. "All of my interactions with him off the field have been good, but when it comes to him on the field, I don't know what makes him think that he can showboat the way he does, and then nobody can retaliate or look at him in a funny way or nobody can pitch him inside."
Made aware of Archer's words, Ortiz shrugged them off.
"What can I tell you, man? Players in today's game are too sensitive about things. I'll just leave it like that," said Ortiz. "I think he's a good pitcher. I think he's got great stuff. He's a guy that I think is going to be pretty good. But it takes some time to get to that level."
Ortiz's feud with Price has been ongoing. He didn't seen amused that a player fairly new to the Majors in Archer would jump in.
"Whatever, dude," said Ortiz. "There's always going to be comments out there. He's not the right guy to be saying that. He's got two days in the league, you can't be just complaining about [stuff] like that."
The Red Sox were just happy to leave Florida with a win, their first of the season at Tropicana Field in six tries.
Now they go back to Fenway Park, with Thursday's non-waiver Trade Deadline looming, hoping once more to fight back into contention.
Boston opens a seven-game homestand against the Blue Jays and Yankees on Monday night.
Allen Webster, who was called up to make the start in place of Jake Peavy, who was traded Saturday, competed well in his season debut.
The righty, who earned the victory, gave up three hits and two runs over 5 1/3 innings, walking five and striking out four.
"My body felt really good," said Webster. "My command wasn't as there as I wanted it to be, but I made the pitches when I needed."
After being shut out for the 11th time Saturday, the Red Sox needed a big swing, and they got one from Ortiz. The slugger ripped a three-run homer to right off Archer to break the scoreless tie in the third.
After getting booed by the fans at Tropicana Field all weekend, Ortiz gave his bat a hearty flip to the side before he rounded the bases on Sunday's blast.
"That's pretty much what I do," said Ortiz. "Pretty much a situation, tie game, something like that where you put your team ahead, you enjoy."
The Rays did rally in the bottom of the third. Curt Casali got it started with a one-out walk. Kevin Kiermaier reached on an infield single. Desmond Jennings mauled a two-run double to left-center to make it a one-run game.
Center fielder Jackie Bradley Jr. helped snuff out a rally in the sixth by making a brilliant running catch against the wall in left-center on a drive by Evan Longoria.
Did Bradley know he could get it off the bat?
"Yeah, I did," said Bradley, adding to his season-long highlight reel of catches. "I got a good read off of it and just tried to track it down."
Junichi Tazawa, who struggled in Friday night's loss, fired a dominant eighth.
"Today might've been the best stuff he had all year," said Sox manager John Farrell. "There was conviction to every pitch he threw. I think the other night might've been the result of too many days down, four days off, but he was outstanding. Good command, power to his stuff, good split, good breaking ball. He was very good. The bullpen, to record  outs, they did an outstanding job."
Koji Uehara came on for the ninth and earned his 21st save.