Gabriel “Gabe” Brown is letting his boundaries be known.

As Gabe made and enjoyed breakfast with his mom Janelle Brown during the Sunday, Oct. 13 episode of Sister Wives, the 22-year-old revealed that his father Kody Brown, 55, had “hit me up” to chat. “It was like strange because he was like, ‘Hey, I miss you.’ And then, he was like, he kept, like, phrasing it like I owed him an apology,” Gabe recalled.

“Eventually, I was just like, ‘Hey, unless you’re, like, actually ready to have a relationship and fix things, then we’re not going to talk anymore,'” Gabe continued. “He came back a couple of days later and texted me. He’s like, ‘Hey, I’ve been like thinking about what you’ve been saying. I forgive you, Please forgive me.’ I was like, ‘Forgive me for what?’ Like, what’s going on here?”

At one point, Gabe said: “I told dad that if he doesn’t change and he can’t take accountability, then I just won’t ever see him again. And I’m perfectly OK with that.”

Janelle, meanwhile, noted that she doesn’t “know what it would take at this point” to get things back on track, especially with everybody in the family being “so busy and they’re starting to spread out across the country.” But Gabe said, “I think it would take Dad really setting things straight. And, I think that that would take an act of God.”

From there, Janelle, 55, then expressed in a confessional that she believes the COVID-19 quarantine period, which jumpstarted Kody’s estrangement from much of the family, had “just presented an opportunity for their dad to do what he’s been wanting to do for a long time, which was live and make a life with his favorite wife,” she said, referring to Kody’s fourth wife Robyn Brown, 44.

“Their dad had been expressing things to them just off the cuff about, you know, he wished that he would have waited and just married Robyn. I don’t know what dad says that to their kids, but my kids have many stories like that,” Janelle continued, though Kody denied it and called the “gossip” surrounding this a “misunderstanding.”

As Gabe and Janelle’s conversation shifted to discussing Robyn, Gabe said: “I think that Robyn definitely has a victim complex. I mean, to put it bluntly, and I don’t necessarily blame her for that.”

Gabe also said he missed being close to Robyn and Kody’s children but he can’t “blame them at all for following on along party lines.”

“If she actually believes that we were mistreating her or her children in any way when she was constantly getting favorited by dad and we were always working on our relationship with her kids. If she actually believes that, then there’s no chance of me having a relationship with Robin ever again,” Gabe declared. “I don’t know, I just, I don’t have a lot of respect for Robyn, so it’s hard for me to rationalize some of her actions. Maybe that’s the reason that she’s like, ‘Oh, you always hated us,’ is because she doesn’t have our respect anymore because she does s— like that.”

Continuing to candidly address how he feels, Gabe added that he is “not hopeful for a relationship with dad and Robyn.”

Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

Sister Wives airs Sundays at 10 p.m. ET on TLC.