The recent NBA Hall of Fame inductee sat down with Shannon Sharpe and revisited a “rough” chapter in his relationship with the actress

Dwyane Wade is looking back at a painful moment in his past with wife Gabrielle Union — one that almost led to the end of their relationship.

While on Shannon Sharpe’s Club Shay Shay podcast, the NBA Hall of Famer, 41, talked about the time when he had to tell Union, 50, that he had fathered his son Xavier with another woman during a break in their relationship.

Wade said that at the time, back in 2013, he moved to end their relationship, thinking it would be best for both of them.

“I tried to pussyfoot around it, I tried to break up with her,” Wade told Sharpe regarding Union. “‘Hey, things have been bad lately,’ ‘Hey, we’ve been having a little distance in our relationship anyway,’ I tried all of that. She kept showing up.”

And Wade knew it was imperative to have the “hard conversation” with the Bring It On star and talk to her about fathering a child with another woman.

“You’re thinking about it all, it’s all scary,” Wade said. “One, the whole situation is scary enough, you’re a public figure. But you know that this is going to hurt someone that you’ve been building a relationship with and a life with.”

He continued, “No matter what people say on the outside, or what people want to think, ultimately you gotta sit with you, and you gotta sit with this person, and I had to sit with my wife and have this conversation.”

Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

Wade added that tumult coincided with the playoffs and was a “rough” time, but that he couldn’t have gotten through the situation without Union by his side.

Ultimately, the couple elected to move through their relationship difficulties and were engaged in Dec. 2013, marrying the following August. After dealing with years of infertility — which they frequently spoke openly about — they went on to welcome daughter Kaavia James Union-Wade via surrogate in 2018.

Wade shares two children with ex-wife Siovaughn Funches; Zaire, 21, and Zaya, 16, as well as Xavier, now 9, with Aja Metoyer. Wade is also the legal guardian of his nephew, Dahveon Morris, 21.

“It hasn’t been perfect, it will never be perfect but that was nine years ago,” Wade said of their past relationship struggles. “We go to therapy, we’ve had shouts about it, we’ve had regular conversations about, and so it’s been something that going to be something that I have to work at and work on.”

He added, “It doesn’t go away because years come or because I say ‘sorry.’ ”

In an essay for ­Time in 2021, the actress shared her own perspective on the couple’s emotional journey.

“It should go without saying that we were not in a good place at the time that child was conceived,” Union wrote. “But we were doing much better when he finally told me about the pregnancy. To say I was devastated is to pick a word on a low shelf for convenience.”

She added, “There are people — strangers I will never meet — who have been upset that I have not previously talked about that trauma. I have not had words, and even after untold amounts of therapy I am not sure I have them now.”

Still, Wade believes the experience is all part of their “story.”

“At the end of this day, this is my family, this is my story,” Wade told Sharpe. “We gonna get through this how we gonna get through it, but we gonna get through it.”