"Joel, whom I've got as much confidence in as I've ever had in any relief pitcher for us, did not have a very good night," manager Joe Maddon said. "I'm not 100 percent sure why he did not use his fastball more.
"I think sometimes, and I've talked about this, the guys will look at the gun reading and think their fastball's not good enough. But that's not true. Even if the numbers are not good, there's movement that can be good sometimes, as is the case with him."
The loss was Peralta's second.
Rays starter Chris Archer, who lasted a season-low 4 2/3 innings his previous time out, worked just five innings in this one before giving way to Brad Boxberger, who was recalled from Triple-A Durham prior to the game.
Archer, who threw 100 pitches and pitched his way out of two no-outs, bases-loaded jams, allowed seven hits and struck out six, but departed trailing, 3-2.
Rays' starting pitchers have now lasted five innings or fewer in 15 of their last 21 games.
Tampa Bay is 5-6 and has blown 12 leads in its last 11 games.
"Archie did OK, he kept us solvent there," Maddon said. "We kind of messed up a double play on him, too [in the fourth inning]. That really caused at least 10 more pitches and a lot of stressful pitches in that moment.
"He wasn't as sharp as he can be, but nevertheless, he pitched in a manner that could have permitted us to win that game. Overall, he threw too many sliders in bad spots. He's making some great pitches and some not so great. One walk and six punchouts -- that's not so bad."
"I think I did a lot better job, especially in tough situations -- bases-loaded, no outs," Archer said. "It's an improvement from last game, but I'm definitely not where I want to be as far as overall efficiency of pitches.
"I gave up a lot of hits with two strikes and if I don't put us in that hole -- only going five, giving up three runs -- then the complexion of the game is totally different."
After Matt Joyce singled to left off Orioles starter Chris Tillman with two outs in the first inning, Evan Longoria put the Rays on top, 2-0, with a towering home run -- his fourth -- over the 404-foot mark in center.
Longoria is hitting .407 (11-for-27) against Tillman in his career and his five homers off the right-hander are his second-most against any pitcher. Longoria has six homers against the Yankees' CC Sabathia.
Steve Pearce cut the Rays' lead in half with a leadoff homer in the third -- his first of the season.
The Orioles loaded the bases in the fourth before Archer recorded an out, thanks to singles from Nelson Cruz and Adam Jones and an error by Yunel Escobar on Matt Wieters' potential double-play grounder.
J.J. Hardy lofted a sacrifice fly to left to tie the game, but Archer struck out Steve Clevenger and got Pearce to ground out.
Again in the fifth, the Orioles loaded the bases with no outs on a double by Flaherty, a walk to Markakis and a single by Manny Machado.
And Archer allowed just one run on Jones' sacrifice fly.
Tillman settled down after serving up that first-inning homer to Longoria, holding the Rays to only two more hits until the seventh when Wil Myers singed and David DeJesus walked to open the inning.
After Ryan Hanigan was hit on the left hand to load the bases, Ben Zobrist flied to center to tie the game at 3. Zobrist's sacrifice fly gave him the 468th RBI of his career, tying him with Carlos Pena for the third-most in club history.
The Rays threatened in the ninth -- putting two men aboard with none out against Orioles closer Tommy Hunter.
Hanigan then bunted them over. Maddon challenged the play, checking to see if Flaherty was on the bag on the forceout, but the call on the field was confirmed.
After an intentional walk to Zobrist loaded the bases, Hunter got the final outs to secure his ninth save.
The Orioles are 3-0 against the Rays this season, outscoring them, 15-4.