Suzanne Somers’ Husband Recalls ‘Talking to Her’ in Final Days (Exclusive)

When Alan Hamel and Suzanne Somers returned home to Palm Springs, Calif., on Thursday, following a six-week trip to Chicago to address her mobility issues caused by a fall, there was little indication something was deeply wrong.

“The first few days we were back, she ate, she had a great appetite and she was taking her meds,” Hamel, 87, exclusively tells PEOPLE.

But by Saturday, Hamel knew things had taken a turn as Somers stopped eating and taking her medications, and was unresponsive to his questions.

“I was staying up most of the night just talking to her,” he recalls. “Then I kissed her, and she responded to that, and it was a physical response with her lips. I knew she was still with us, and I knew that she could probably hear what I was saying.”

Hamel says he continued “just talking to her for hours at a time” until his wife of 46 years took her final breath.

“Then finally it was over,” he recalls, noting that the couple were in bed at the time. “Normally, when people are not responding, stop eating and stop taking their meds, you call 911 and they take you to the hospital. I knew that she did not want to be in a hospital setting. That was the difference. So it was very peaceful, and that’s the way it all went.”

Somers died early Sunday morning at 76 from an aggressive form of breast cancer she had struggled with over 23 years, her longtime publicist R. Couri Hay confirmed to PEOPLE on behalf of Somers’s family. In addition to Hamel, the Three’s Company actress was also surrounded by her son Bruce and other members of her immediate family.

The actress’s death occurred one day before her 77th birthday. Several members of the family had already been gathered to celebrate the occasion, while others drove in the day of. But after her death, they decided to mark the day in another way, Hamel tells PEOPLE.

“We actually toasted last night,” Hamel recalls. “We had, I think, 12 of us here, and all toasted Suzanne. I’m positive we’ll do something today. I don’t know what it will be, but it will involve a lot of cake, because Suzanne was crazy for cake. All cake. But there was one kind of cake she really liked. Red velvet.”

Somers’ death marked the end of their extraordinary marriage, one that spanned 46 years. Hamel recalls being struck by Somers when he met the model on the set of the marriage competition show, The Anniversary Game, in 1969.

“We were going to be shooting in a couple of hours and I had my crew around me and we’re talking about last minute touch-ups,” he explains. “I look over on the other side of the stage, and I see the most incredible woman I’ve ever seen in my life. And I thought, ‘I have to go over there and talk to her.'”

“So I left the crew, and I started walking over and I’ve never been good with come-on lines at all,” he continues, adding that he was feeling a “little panicky” about what to say. “I still didn’t figure out what I was going to say to her. So out blurted, ‘Would you mind getting me a cup of coffee?’ How’s that for stupid? And she did. She did get me a cup of coffee, and there was an immediate connection like I had never felt before in my life.”

The couple tied the knot in 1977 and remained inseparable. Throughout the course of their marriage, the pair worked closely to build Somers’ lifestyle brand and wellness empire, with Hamel even serving as Somers’ manager. They weren’t afraid to talk about the inner workings of their relationship, including opening up about their sex life and how they never spent a day apart. And when Somers struggled on-and-off with breast cancer, their work took a backseat to her health.

“The past 42 years, we have not spent even one hour apart. And one of our sons refers to our relationship as ‘functionally codependent,’ which it probably is,” Hamel quips.

Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

Despite Somers’ decades-long contributions to TV, film and fitness, he believes his wife would still want to be remembered for portraying Three’s Company’s iconic and “delicious character,” Chrissy Snow, all these years later.

“She brought joy into everybody’s heart with that character. It was lovable and beautiful, and it was exactly what she wanted,” he says, comparing the show to fellow legendary series I Love Lucy. “It’ll play forever.”