Katherine Heigl is done apologizing for her "difficult" reputation.

The Firefly Lane star recently reunited with former Grey's Anatomy costar Ellen Pompeo for Variety's Actors on Actors series, where they got candid about the scrutiny that followed Katherine after her departure from the medical drama in 2010. And the 44-year-old admitted that it took years for the comments to stop getting to her.

"It took me until probably my mid- to late-30s to really get back to tuning out all of the noise and going, 'Are you this bad person? Are you unprofessional? Are you difficult?', she told Ellen in the conversation published June 6. "Because I was confused!  I literally believed that version, and felt such shame for such a long time, and then had to go, 'Wait. Who am I listening to? I'm not even listening to myself. I know who I am.'" 

As a refresher, Katherine stirred up controversy during her almost six seasons on the show when in 2008 she withdrew her name from Emmys consideration—after winning the previous year—saying she didn't feel the material warranted it. She further called out the show's working conditions in 2009, a claim Ellen has also spoken about.

 The 27 Dresses star also recalled her headspace at the time.

"I was up here in my head, in my gut, in my mind, in my life. I was just vibrating at way too high of a level of anxiety," she said. "I got on my soapbox and I had some things to say, and I felt really passionate about this stuff. There was no part of me that imagined a bad reaction."

Katherine added that she never meant to hurt anyone with her comments, saying, "It's really disconcerting when you feel like you have really displeased everybody. It was not my intention to do so."

The two stars also reflected on the ways in which they're learned to stand up for working conditions on their sets over the years. Specifically for Firefly Lane, Katherine was grateful for the presence of an intimacy coach to help guide the younger actresses on set. 

"I felt this weight off of me in a way that I didn't feel like I had to find a way to fight those battles for these girls," she explained. "I'm always the bad guy. People like me to be the bad guy."

To which Ellen responded, "There's two roles women fit into, victim or villain. And the women who are victims are only victims because they don't have the guts to be the villain."

Acknowledging how social media has shifted the way people share opinions, the 53-year-old added, "You were just one person. And somehow now collectively the whole world gets to criticize everything and it's okay but when you did it, they didn't like you."

Katherine agreed, nonchalantly adding, "They didn't like me. Oh, well. What are you going to do?"

But Ellen has her own suspicions as to why people responded negatively to Katherine. She tells her former costar, "You're too hot, Heigl."

Katherine laughed, responding, "That must be it! That's what I tell myself when I go to sleep at night."

For the latest breaking news updates, click here to download the E! News App