Anderson Cooperhad a pointed question forPrince Harryabout his current chapter since leaving royal life.

In an interview that aired on CBS'60 Minuteson Sunday to promoteSpare, the Duke of Sussex’s memoir, Cooper noted that Harry and his wife Meghan Markle stepped back from their senior royal roles in January 2020 and relocated to North America, then inked a deal with Netflix and gave a tell-all interview to Oprah Winfrey months later. The couple has continued to use the royal Duke and Duchess of Sussex titles they received fromQueen Elizabethon their2018 wedding day, and Cooper asked why.

“Why not renounce your titles as Duke and Duchess?” Cooper, 55, asked.

Harry, 38, replied: “And what difference would that make?”

60 Minutes Twitter

Prince Harry Tells Anderson Cooper He Can’t See Himself Returning as a Working Senior Royal

“One of the criticisms that you’ve received is that okay, fine, you wanna move to California, you wanna step back from the institutional role. Why be so public? Why reveal conversations you’ve had with your father or with your brother? You say you tried to do this privately,” the60 Minutescorrespondent said.

For more on Prince Harry’s memoir, listen below to our daily podcast PEOPLE Every Day.

“And every single time I’ve tried to do it privately there have been briefings and leakings and planting of stories against me and my wife. You know, the family motto is ‘never complain, never explain,’ " Harry said of the adage. “But it’s just a motto. And it doesn’t really hold.”

Prince Harry and Meghan Markle.Chris Jackson/Getty Images

Prince Harry, Duke of Sussex, and Meghan, Duchess of Sussex arrive on the long Walk at Windsor Castle

Chris Jackson - Pool/Getty

Prince Harry, Duke of Sussex and Meghan, Duchess of Sussex visit a local farming family, the Woodleys, on October 17, 2018 in Dubbo, Australia.

“We’re six years into it now and I have spent every single year of those six doing everything I can privately to get through to my family. And the thing that is the saddest about this, Tom, is it never needed to be this way. It never needed to get to this point,” he said. “I’ve had conversations, I’ve written letters, I’ve written emails, and everything is just, no, you, this is not what’s happening. You are imagining it.' And that’s really hard to take.”

The book jacket of Prince Harry’s memoir ‘Spare’.PENGUIN RANDOM HOUSE

Prince Harry book

Can’t get enough ofPEOPLE’s Royals coverage?Sign up for our free Royals newsletterto get the latest updates onKate Middleton,Meghan Markleand more!

Prince Harry’s memoir, which will be released on Jan. 10, is an “intimate” and “heartfelt” look into the “experiences, adventures, losses, and life lessons that have helped shape him.”

source: people.com