Weird Lag Problem Only in Combat

[SauronTheGreat]

 I normally have 100-110 in-game ping. As a destroyer, my wrath skill is quite fast 3x/sec when i'm solo or duo pve. But, in a 6-man party, when everyone around, my in-game ping jumps to 160-170 and it's nearly 1xwrath/sec. I already use a ping booster, but i can not fix that high ping in combat, when it actually matters. Tried ctrl-f, but still ping jumps directly to 160 and my attack speed decreases around 3x ... how come my latency increase that much, just because there are some people around me ?

 Is there a workaround for this ? So i can fight like i'm solo ( Ctrl-f does nothing ).