The seven-run loss marked the third shutout and the largest margin of defeat all season for the A's, who have now lost three of their last four.
"Sometimes it goes that way," said A's manager Bob Melvin. "You'll get hits in bunches and everybody hot, and then other times, it cools off as a club. We've been in a little bit of a funk for maybe a week or so."
Sonny Gray settled in after allowing three runs in the first two innings, but the A's simply could not figure out Phelps, who pitched 6 2/3 frames and surrendered both Oakland hits.
Despite their best efforts to work the count, the A's went hitless against Phelps until the fifth, and they didn't get a baserunner past second all night.
"It's definitely one of the best starts of my career, probably," Phelps said. "Coming against a team that is first in its division and has one of the best records in baseball -- my biggest thing is trying to go out and give us a chance to win every time out. Our offense gave me a chance to be successful today."
The A's lone hits came from Derek Norris and Jed Lowrie, while the middle of the lineup stayed cold. John Jaso is now hitless in his last 17 at-bats, Josh Donaldson is mired in an 0-for-27 slump, and Brandon Moss is 1-for-his-last-17.
For a team that has made a habit of scoring runs in bunches, Friday's game was frustrating.
"For as well as [Phelps] pitched, he threw a lot of balls, which usually we take advantage of that," Moss said. "But any time when we would get the pitch count up, we always seemed to have two strikes. We'd get a guy on and just couldn't get anything going."
The Yankees opened the game with three straight singles by Brett Gardner, Derek Jeter and Jacoby Ellsbury, with Ellsbury's hit scoring Gardner. Mark Teixeira followed with a sacrifice fly to make it 2-0, and Gardner's RBI in the second increased the A's deficit to three runs.
Gray found his rhythm after seven of the Yankees' first 11 batters reached base, retiring 13 of his last 14 to put together a quality start. But the Yankees put the game away in the eighth with four runs off Jeff Francis, all with two outs.
"They put the ball in play early, and I left a few balls up, and they were able to take advantage of it," Gray said. "They were aggressive, and once we were able to start making pitches down in the zone, we got them out. It was just a little too late."
After posting a 1.99 ERA in his first 10 starts, Gray has surrendered 15 runs in 24 innings over his last four outings. He has also allowed eight first-inning runs in 14 starts.
"That's probably been his problem the last few times in the starts, whether it's ball-strike ratios, or giving up some hits [early] like he did tonight," Melvin said. "The good news is he settled in pretty well. This was probably his most difficult start, as far as giving up hits early on, but he was able to give us six innings and settled in and threw the ball well after the second inning."
Phelps exited with two outs in the seventh, and Dellin Betances and Jose Ramirez held Oakland hitless over the final 2 1/3 frames.
It was the third time in 2014 that the A's have been kept to two hits or fewer.
"We've sort of had a little bit of a stretch here where we haven't taken the kinds of swings that we want to take, or have the at-bats that we want to have," Moss said. "But that's the ups-and-downs of the season. You're going to have your hot times, and you're going to have your bad times, and you've just got to fight through those times and try to find a way to be productive, at least sometimes, during that."
New York snapped Oakland's streak of seven straight wins against the Yankees at the Coliseum, improving to 5-2 on its nine-game road trip.