When should friends stop being friends?

I have a friend, let's call him S.
When he and his friends first came to my building I was very happy; no much more than that, euphoric, because I never had any friends on this block. I was finally not alone. And true enough, 3. Semester was the happiest Semester of my life. I had people to cook and share stories with and for the first time in my life, I looked forward to going home, and my bed wasn't the reason.
But here's the thing. This S person, he likes to make 'crude jokes' or 'gurau kasar', and I'm sure he didn't mean what he usually says and I, knowing this, despite being a sensitive girl who cries at the tiniest bit of insult, was never offended by any of it. Because I trust that he doesn't mean it and it was just a way of him having fun, I guess?
But sometimes (or maybe three times to be exact). It just went too far. There are words that my heart can't take even as a joke.
He once compared me to a pig. A fat hideous disgusting creature. It triggered my insecurities because I do think I'm fat and hideous and someone happened to blatantly point that out. But I let it go. Thinking it was just a one-time harmless joke; but guess what, he repeated it and burn me all over again.
My music taste didn't suit him either. Kpop to him is just a contemptuous music genre that he will never give a chance to. I understand that we can't force people to like what we like but I just wanted him to respect it and not look at me like I'm a disgrace.
Looking back, I realized he never sincerely apologized. I just forgave him 20 minutes later and that's it. [I'm a coward who's too afraid to lose a friend]

But here comes the breaking point. When things were all going swimmingly, and we were having a normal conversation, he blurted out this one sentence and the world just stopped. He said something in the lines of "Honestly I don't like talking to you; you might think I'm joking but it's a fact".
Upon hearing it time felt frozen and I can almost hear my heart break. My chest were heavy, I just couldn't believe he said that.
From what I can interpret from this was 'he didn't like talking to me' because of my immature thoughts, I'm not a knowledgeable nor pious person, I don't watch Nouman Ali Khan videos nor I have his books or maybe I'm just boring.
This really hits me so hard like a stinging slap on my face because to me, our conversations were my warmest comfort and I was genuinely happy to be able to share my stories and opinions to someone other than my diary.
My brain were flooded with all the things we talked about;
"If you have a genius child, what would you do?"
"Try hearing this song, what do you think?"
"I was right, there WAS a reason why he always talks to me; he wants me in his religious group!"
"From now on speak only English or German"
"It's good that the Prime Minister didn't let the Israeli athletes compete in Malaysia"
"I bet you wouldn't meet me here if I can sing"

Did he really hate all of it?

I always thought of him as one of my most important friends but i guess it wasn't mutual.
Even now, after a week has passed, my heart still aches and tears well up every time I recall that memory. Sometimes I couldn't sleep at night. The scene keeps replaying in my head and I so badly want it to stop but I can't. The food I crave didn't taste the same and I feel like I have a bullet hole in my heart.
So this is his impact, huh? Those few words. Meaningless to him but an avalanche to me. What did I ever do to him to deserve this? I tried to be nothing but good to him but I guess I was never enough.
Why was he nice to everyone else but me? Did he hate me from the start?
For his birthday I went to 6 different Rewe Supermarkets across the city to find that one Oreo flavour that he wants to try (but to no avail). I always buy extra biscuits so he wouldn't be hungry at night. I tell a lot of nice stories of him to the girl he likes.

Maybe I cared too much. It wouldn't hurt as much if I didn't. But I did.

I don't know how it will turn out in the future. It probably won't affect him whether I'm in his life or not. I just wish I can stop being sad. And maybe I will return to writing my heart out in a book.

And I don't resent him. For anything. He is entitled to his opinions and I guess I just have to accept it. I shouldn't act like I matter because I don't.