And those that are in elite fit condition (endurance athletes Cyclists and LD Runners etc) from intense long term HR, VO2 and Lactate Thresh training, acclimate quicker and in turn climb at a far faster pace than any sedate individual.
I have personally seen ultra marathon runner suffer terribly at altitude compared to less super-human individuals. Yes, I totally agree that fitter people will climb faster and perform better once acclimitised but from what I have read and witnessed, will not necessarily acclimitise quicker.