I took my son to RMNP about 1 month before his 4th birthday. The little guy made it up the Lilly Mountain trail, parking lot to summit and back, without needing carried.
Just this past weekend (age 6 now) I took him for his first overnight backpacking trip and he did great. He hiked better on day 2 than on day 1, and at the end he asked if we could spend another night out there.
A healthy kid (mind and body) will rise to the (sane) challenges you put before them. You've got to realize that complaining is a phase, a stage, and you can persist, get them past it, and continue the hike. Keeping them interested is another challenge, though, and it depends entirely ont he kid how you do that.
I'm very proud of my son, especially now that we're in scouts and I can see what utter poofs the other kids his age are.