Sameer
Although I can't offer a medical opinion, I have exactly the same experience.
I spent 3 years training for the Air Force and I daresay I was in excellent medical shape.
However I always experienced that in events like cross country which required endurance, I fell short.
Was never really able to come to terms with this, specially because I was extremely good at all physical activities