Duffy gave up homers to Dioner Navarro, Josh Donaldson and Russell Martin. After the game, Duffy was simply shaking his head.
"I was trying to go away on a couple of those pitches," Duffy said. "I felt great today for the most part. The fastball felt great, but my slider wasn't where I wanted it. The changeup to Donaldson, I need to get through that more.
"I just need to make better pitches. They capitalize on mistakes and my hat's off to them. They're a really good team. But I'm better than that. I'm a lot better than that."
Duffy was staked to a 2-0 lead after Kendrys Morales doubled in two runs.
But Duffy gave one back on Jose Bautista's sacrifice fly in the first, and then Navarro belted a two-run homer in the second that gave the Jays the lead for good.
"I had better stuff tonight than I had at The K [a 3-0 win] against them," Duffy said. "They squared up some balls on some good pitches. My team played great tonight. I just didn't execute. I gave it my best out there.
"Like I said, hat's off to them. They're a really great team. There are a lot of wouldas and couldas. I had a good fastball. There are times you don't have anything on your fastball and you win. And then tonight, you have a really great fastball and you lose.
"I just know I'm capable of so much more than that."
Jeffrey Flanagan is a reporter for MLB.com. Follow him on Twitter at @FlannyMLB. This story was not subject to the approval of Major League Baseball or its clubs.