Insane hangovers at 22 (M)

My hangovers usually last 2 days, better on the 2nd one, first day it kicks in at like 5-6pm, insides shaking, heart palpitations, hot flushes, cold sweat, inability to focus just pure suffering, and no sleep on the first night due to palpitations waking me up. No palpitations when im sober tho, i feel good. Work, go to gym. Even after 3 beers i feel awful the next day. I was diagnosed with anxiety/panic disorder 4 years ago. But ive managed it an 95% it doesnt affect me when im sober anymore. What could be the issue, is alcohol just not for me?