In addition, they moved to 30-20 on the season, which represents the most games over .500 the team has ever found itself. Tampa Bay becomes just the fifth club in Major Leauge history to be 10 games or more over .500 on Memorial Day after losing 95-plus games the previous season.
"The record definitely indicates that we've been playing good, solid baseball," Rays manager Joe Maddon said.
Pena drew a one-out walk in the ninth to bring Longoria to the plate to face left-hander George Sherrill.
"I love being in that situation, not being the guy that the team is looking to per se ... you know I'm looking forward to getting up there and having a chance to win the game," Longoria said.
The rookie connected on a 2-2 fastball and doubled to right-center field to score Pena from first and give the Rays their fourth walk-off win of the season, setting off yet another mob scene that saw Pena and Longoria get pancaked.
"When I hit it, [Pena] was already on the move," Longoria said. "I hadn't seen him move like that [before]. He was going good."
Pena took off as soon as he saw the ball headed for the gap.
"I got up to full speed, and that's it," Pena said. "That was my highest gear. When I got to third base, I was actually yelling at [third-base coach Tom] Foley, 'Send me!' Because I can't stop right now. You can't just put it in park right there; you'll blow your transmission. I had it in my mind I was going to score."
Pena and Longoria have started to pick up the pace offensively.
Pena became the first Rays player this season to reach double digits for home runs when he hit his 10th dinger of the season in the third off O's starter Daniel Cabrera. Only the night before, he'd collected a single, double and a triple. Meanwhile, Longoria had two home runs and six RBIs Saturday night before going 2-for-5 Sunday with two RBIs.
"I made an adjustment with my stance, just a little bit," Longoria said. "I've just been a lot more patient waiting to get my pitch. Not missing my pitch has been the main thing."
James Shields started for the Rays and got immediate assistance from his defense with one out in the first inning. Carl Crawford dove toward the line in left field to steal a hit from Melvin Mora. After Shields struck out Nick Markakis to end the inning, he waited for Crawford outside the Rays' dugout and told him, "Now that you're all stretched out, are you ready to go?"
"He made a great play and that started the whole thing [Sunday]," Shields said. "We had some good hits and some good timely hitting today. ... Our hitters are starting to come alive and our defense is staying strong."
Even though Shields let a 4-1 lead get away, he never gave in to the O's hitters and managed to throw two additional scoreless innings to keep the game tied at 4 before Trever Miller took over to start the ninth.
"I'm just glad I got to keep them in the game as long as I did, and these guys came through," Shields said.
Miller retired Aubrey Huff to start the ninth, and then Troy Percival entered the game and got the final two outs of the inning, earning him his first win of the season.
While the excitement is palpable inside Tropicana Field, Maddon sounded like the manager of a young team who understands there are 112 games left in the season.
So he talked about being more fundamentally sound, being sharper on the bases and paying better attention to the signals.
"There are definitely areas we can get better at in regard to fundamentals," Maddon said. "How good is this team? We're 10 games over .500 after 50 games. That's nice. But I'd like us to continue to push and take, because nobody's going to give us anything."