My own queue position doesn't move accordingly

[Riyak]

So I've been in queue a while and just recently paying more attention I noticed for every 50 or so users that disappear in queue my position only moves by 1. For example there was 3000 users in queue and I had position 54 then when the queue is at 2900 I have position 52 o 51