Of course it would be, but we are talking about N between 1000 and 3500.
I see your point :
"I finished first in a multiplayer contest with 1000 contestants. If there had been 3000 more contestants, there is big probability that at least one person among these extra 3k had beaten me".
Maybe you would have finished 3rd or 5th ? This a little ranking shift (for the top players at least) that shouldn't impact earned points imo.