When his former coach at Long Beach State, Dave Snow, called and offered him a job as a volunteer coach -- making a scant $400 a month -- Madison ditched the suit and jumped at the chance for a re-entry into baseball.
It wasn't without sacrifice, though.
"It was a really tough decision," Madison said. "I went from making good money as a 23-year old to making next to nothing. That's where family help came in. I stayed with a few [former] teammates, slept on their couches. I drove a Frito Lay truck to make money and sold office supplies. I worked all kinds of crazy jobs to make money.
"But I had a huge passion for baseball."
That much is still evident today for Madison, who you can find busily preparing for what will be his second First-Year Player Draft after being named the Padres director of scouting in December 2009.
It's been quite a path, starting with humble beginnings. But, as Madison is frequently reminded, it was the right decision for him all along.
"People always ask me if I'm doing my dream job," Madison said. "I tell them playing baseball for a living was my dream job, but scouting is a close second."
To get into the scouting game, Madison had considerable help along the way -- a lot of which came from Twins scouting director Deron Johnson. Johnson introduced Madison to then-Padres scouting director Bill Gayton, who subsequently offered to send Madison to Major League Baseball's scout development program.
For this, he's forever grateful to Johnson and Gayton.
"Not only has [Johnson] impacted my life and career, but he has also hired and mentored several African-American scouts that are working toward successful careers in the game," Madison said.
He worked in that capacity for two seasons before taking a similar job with the Cardinals. He spent two seasons with the Cardinals before returning for a third stint with the Padres, reuniting him with Jason McLeod (assistant general manager in charge of scouting and player development) and Chris Gwynn (director of player personnel). Madison worked closely with McLeod and Gwynn in his first stint with the Padres.
"I love being back here in San Diego. It's an outstanding organization to work for, and I wouldn't want to be anywhere else," Madison said. "Because I started my career in the game here in San Diego, I have always considered it home and looked forward to the day that I could come back and work.
"The Padres are loaded with first-class people, and you truly feel a sense of family here."
Madison, who cites his parents as the greatest influences in his life, estimates he spent about 250 nights on the road last season scouting players. He figures to be every bit as busy this year, preparing for an important Draft, one that will see the Padres select five players in the first 60 overall picks.
It's a busy job and life for Madison, but one that he wouldn't trade for anything.
"I originally got into scouting as a way to learn the business and get my foot in the door, with hopes of moving into a front office quickly," Madison said. "After learning about baseball and seeing the challenge of scouting, I fell in love.
"There is something thrilling about watching amateur players when they are 17-21 and projecting what they will be when they are 23 and in the Major Leagues. Watching them develop and grow as men and players is something that fulfills me daily. Being able to help a young man realize his dream of playing professional baseball is extremely gratifying."