The panel discussion followed a dinner in which all proceeds went to providing children with a safe place in school to discuss and talk freely. Torre himself is an outspoken survivor of domestic violence, although until about 15 years ago, he kept much of his childhood struggle hidden.
"If I had a safe room in the school, I would have been tempted to talk about it with somebody," said Torre, who was announced as Team USA's manager for the 2013 World Baseball Classic earlier Thursday. "I felt like my house was the only house in which it was going on. I felt like I was the cause of it as the youngest of five children and the child that my dad didn't want."
Sitting on the stage, the five managers each told stories about the stress of managing, disciplining players, ejection tales and feuding with other managers.
La Russa kidded that he only had qualms with one of the four other skippers on stage, and hinted that it was Lasorda when the 1988 World Series was brought up. Lasorda's Dodgers upset La Russa's A's in five games.
To that, Scully rolled up his sleeve and showed the crowd the '88 World Series ring on his left hand, garnering plenty of laughter and applause.
La Russa called Torre the "poster boy" for surviving domestic abuse and going on to bigger and better things.
"I applaud Joe," La Russa said. "I think anybody like Joe that gets behind the cause is admirable. But when you realize how well he knows what he's supporting, that really hits you."
Torre said he isn't sure how his childhood would have been different had he been given opportunites like the ones his charity has provided to thousands of students.
But for reference, he only needed to look to the right of him on stage, where before the dinner two students from Mark Twain Middle School gave impassioned speeches about their experiences in Margaret's Place.
Margaret's Place is the name of what Torre refers to as the "safe room" at the school where no topics are off-limits -- from domestic violence, to gang violence, to young relationship abuse and bullying. The room is named after Torre's mother, Margaret, who suffered the brunt of the domestic abuse in his family.
"I was an 8-, or 9- or 10-year-old when [my father] went for his revolver, because he was a police officer," Torre said. "He went for his revolver because my oldest sister was trying to protect my mom with a kitchen knife. He went for his gun. I remember that, and I was a witness to that."
Thursday's event raised enough money to start a Margaret's Place in John Adams Middle School in Los Angeles. The foundation also has aims on starting a "Margaret's Place in two other middle schools in the area.
"I grew up with somewhat perfect parents, and you kind of think, 'Well everybody has that,'" said Mattingly, who succeded Torre as manager of the Dodgers in 2011. "And then when you get out into the world, you find out that there's a lot of stuff that goes on. ... Here's someone that overcame all of this in Joe."
During the panel discussion, Mattingly told the story of getting hit by one of La Russa's pitchers when he was playing first base for the Yankees and La Russa was managing Oakland. Mattingly kidded that he knew it was coming and thought, "I'm not going to like this very much."
As for his favorite part about Thursday night's dinner, Mattingly was succinct.
"Just listening," he said.
And there was plenty to listen to. From Torre reaching across the stage to shake Scioscia's hand and finally forgive him for two AL Division Series defeats to Lasorda rehashing a fist-fight with Giants manager Charlie Fox in 1973.
"It's a fun night," Scioscia said. "It's baseball and we all have a passion for it. I brought my autograph book."
Scioscia wasn't the only one, and by the time the event had ended, the money was raised for thousands more children to have a place to openly discuss what silently tormented Torre for years.
As the name of the charity implied, those children will be safe at home, indeed.