I 20F don’t know how to feel about situation with 21M bf, How should I approach this?

For context my boyfriend and I have been together for four years now and we’ve had our equal share of moments. currently we’re in a medium distance relationship. recently he came to visit and we decided to go out for drinks one night. Everything was fine up until he started ordering more and more drinks. I ended up having too much and majorly blacked out for my first time. this was also the first time i ever had a amf. i know that night was tough, we talked about it and I had thought we both got over it since it was my first time. mind you he’s also had his moments throughout the relationship, but vice versa we took care of eachother during those times.Fast forward a couple of weeks later, i’ve tried initiating spicy conversations and anything that has to do with intimacy, he just ignores or changes the topic. I brought this up to him and he proceeded to say “That night was a lot and it made me look at you differently as if you couldn’t hangout with my friends and I or do the activities i like to do”. Which really hurt because he’s never even tried to invite me to anything even though he’s had plenty of opportunities. i’ve taken care of him during his blackouts and for my first time, i got that as a response. It just made me feel kind of shitty and sad, especially since i’m supposed to live with him this summer at his college. i feel really hurt because i’ve never thought of anything else after his moments. I know that night was rough and now i don’t feel like i want to drink again or anything really. i just feel terrible. What would you do in this situation? Should i even drink or how should i approach this?