"But I went to the service for three years," he said. "When I came back in 1946, I got an ear infection, and it took me a while to regain my former talent."
Irvin is 92 years old and living in a Houston retirement community. Two years ago, when I spent a couple of delightful hours with him as a columnist for the Houston Chronicle, Irvin was still clear of mind and lively of spirit. He had the arms, hands and presence of a great athlete.
These days, he has good days and bad, and was not up to sitting for another interview. When we talked, he opened a door into a fascinating, painful, important time in American history.
"May I just say this? We weren't jealous of Jackie Robinson, but we were envious," Irvin said. "We knew this was a big thing. Most of us had never heard of him. He only played for Kansas City for maybe a month or two. He was better known in California as a track star and a football player."
The stories of Irvin, Robinson, Larry Doby and other pioneers are uniquely American. They're not just baseball stories, either, because they helped change the world.
Dr. Martin Luther King Jr. said many times that the men who broke baseball's color line made everything he worked for easier because it forced white America to see the world in a way they never had before.
Things worked out just fine for Irvin, even though it was Robinson who became the first African-American to play in the Major Leagues.
Irvin will be remembered as much for being one of baseball's true gentlemen as for being a great player. And he was a great player, both during eight seasons in the Negro Leagues and eight more in the Majors, including seven with the New York Giants.
He played in two World Series and was inducted into the Hall of Fame in 1973. He was the first black player in Giants history, and in June 2010, they retired his No. 20 during a ceremony at AT&T Park.
He has lived a life that has given him a front-row seat to amazing change. He was a mentor to Willie Mays and a friend to Ted Williams. Howard Cosell was Irvin's attorney when he and Mays opened a Harlem liquor store.
Irvin was in the dugout that 1951 afternoon at the Polo Grounds when Bobby Thomson hit the most dramatic home run in baseball history.
"Bobby made us $5,000," Irvin said. "That's how much winning the pennant was worth."
The Giants were 13 games behind the Dodgers on Aug. 11, but won 37 of 44 to force a best-of-three playoff series. New York entered the bottom of the ninth inning of the third playoff game trailing, 4-1.
"I made the only out of the inning," Irvin said. "We're just hoping against hope that something would happen. Most of us are thinking, 'Well, we'd played great, but maybe we didn't play good enough.' [Giants owner] Horace Stoneham was in the dugout. Stoneham and [pitcher] Sal Maglie went back to his office, all the way in center field. They never saw the home run.
"In the dugout, we're sitting there pulling for [Thomson]. All of a sudden, we looked at each other and realized we'd won the pennant."
Out in the visiting clubhouse in center field, Dodgers pitcher Don Newcombe, who'd been removed from the game in the ninth inning, was coming out of the shower and wondering why the photographers were dismantling their equipment and hurrying out the door.
"What the hell are you guys doing?" Irvin quoted Newcombe as saying. "They said, 'The Giants won the pennant.' He thought they were full of it. Now, over in our clubhouse, Logan, the clubhouse custodian, had put the champagne away. He figured it was wait 'til next year. When we won, he tried to chill it in 10-15 minutes.
"I got a bottle of the champagne and took a taste. Willie said, 'Hey, Irv, what are you drinking?' I said, 'Champagne. Want a taste?' He spit it out and said, 'Give me a Coke.'"
Irvin celebrated by going home and having dinner with his family. Dee, his wife of 67 years, died in June 2008.
During our 2010 visit, he laughed a lot as he remembered the old days, the way things used to be. He never made more than $25,000, but there were days when it felt like stealing.
"We had so much fun," he said, "and we had a little money. [Roy Campanella] would say, 'They're paying us for playing a game we'd play for free.'"
He and Campanella once played against one another in a Mexican league championship series.
"We're playing the last game in front of about 15,000 people," Irvin said. "I come to the plate with two outs. I'm getting ready to get in the batter's box, and our owner, Jorge Pasqual, yelled, 'Monte, come here!'
"I go over, and he put his arm around me and said, 'Hit a home run for me.' I said, 'Jorge, damn, don't you see how hard Salazar is throwing?'
"He was the George Steinbrenner of Mexico. Worth about $50 million. Impulsive. Demanding. I told him, 'I can't promise you a home run, Jorge, but I'll try to keep the rally going.'
"I go to the plate, and Campy said, 'What did Jorge want?' I told him he wanted me to hit a home run.
'Are you crazy?' He called everybody dude. He said, 'Not today, dude.' I did hit a home run, and by the time I rounded the bases, Jorge was at home plate. We won the game, 2-1. When he shook my hand, he handed me $500."
Irvin laughed a deep, belly laugh, a laugh from another time and place, a laugh that resonates all these years later.