Thanks for the replies everybody. Sorry for the fairly lengthy post that follows, but it's kind of hard to explain my situation concisely. I type one thing, then want to type another to give that thing context, and I kind of just end up rambling. So read at your own risk. For a quick answer to a couple direct questions though:
Tom: The doctor did examine my knee for structural problems and found none, and x-rays also failed to show anything abnormal. No MRIs.
ksolem: The closest I get to exercise currently, and this has been the same since June, is just walking 1-3 miles around my neighborhood most days, followed by stretching and then using an ice pack.
Just a bit more info on the specific problem I have now; after a hike with a lot of elevation change on Mt Diablo in mid March my right knee was tight, and it felt particularly so if I squatted with most of my weight on that leg. When I did this there was a pain that felt like it was in the back of my knee. The knee also just felt generally weird, with mild discomfort both while sitting and walking, and walking down hills or stairs was uncomfortable, with the discomfort centered on the lateral side of the knee around what I believe is the end of the femur. I don’t know what to compare it to, but I think my IT bands may be a bit too tight as well. Sometimes if I squat on this knee, and occasionally after just walking for a while, some piece of tissue in this area swells up to the point of feeling like a little squishy ball under the skin that I can shift back and forth. This tissue wraps over the end of the femur. All of these symptoms apart from the squatting tightness/pain (which has since subsided) are mild; they're definitely noticeable, but they're not what I would call painful.
After just resting it for a while, which has been my standard approach when things get sore even though it doesn't seem to work very well, my symptoms had lessened a bit but were still there so I found a doctor in mid May (moved out here in late January). He did do what seemed like a decent exam of my knee, he moved my leg in all different directions and motions and also palpated the knee, and none of those things caused any pain and he found nothing abnormal. Also had x-rays taken, and those didn't show any signs of arthritis or any other abnormalities. He settled on an aggravated plica due to overuse, but pretty much told me there wasn't much I could do about it...his first attempt at treatment was to proscribe some beefy NSAIDs but this didn't accomplish anything.
I asked about physcial therapy, he said he didn't think it would help much, but he had one of his assistants show me a few hamstring stretches and a quad/hip exercise that had me sit on the floor and lift my legs, extended either straight in front of me or at a 30-45 deg angle to the side. Even doing less than the recommended 3x10 daily program aggravated the front of my hip (I think it's the hip flexor) that I originally bothered last winter while pool running. Ha, what kind of clown gets hurt while pool running? I was pulling my leg forward too aggressively against the resistance of the water apparently. This soreness also took a couple months to go away but it hadn't bothered me for a while before doing the exercises. Oh, and I was pool running because I had a sore achilles from cycling and hiking on hills.
All of the doctor stuff was during the early part of the summer, first visit at the end of May, and saw him a couple more times through July. As far as current activity level goes, the only thing I do is walking 1-3 miles a day, just around the local neighborhood usually, which is completely flat. I usually follow this with simple stretches of the hamstrings, quads, calves, and hips/IT band. I've done this all through the summer and this fall so far, only hiking I've done was the first 3 weekends of July. The doc recommended I try to increase my activity level a bit as he thought I couldn't really do any damage to my knee.
So I went up to western slopes of the Sierra and did a few very low key overnighters, hiking about 2 miles a day plus whatever bit of walking around the area of my camps taking photos and such. I picked places that wouldn't be very hilly, but apparently even the couple hundred feet of gain and loss was too much, and even my left knee was aggravated after the third outing, so I stopped doing that stuff (and while it was nice to spend some time in the woods, it sucked to be just basically camping while looking at the high peaks on the horizon that I knew I could have been climbing all summer). Now my left knee exhibits more discomfort than my right, again it’s just a sort of vague discomfort around the lateral end of the femur but now there is also some kind of under the knee cap. For example, normally when your knees are bent and your pants are putting pressure on your knee cap you don’t notice it, but I do, so I’ll frequently shift my legs around when I’m sitting to make that feeling go away. I also notice this in my left knee while walking, but not in my right.
So the thing that's frustrating to me is that my symptoms aren't intense at all, but they just linger around and prevent me from doing anything that I fear will make them worse. The "variety of issues" I spoke of have, over the past 5 years, included
-A sore left ankle, around the bottom and back edge of the bony part on the lateral side, was diagnosed as peroneal tendonitis. I did some exercises for this and it eventually got better, but that area is still sensitive and if my footwear puts too much pressure on it, it will get tender pretty quickly.
-Sore achilles tendons in both legs, down around the insertion points. This has bothered me on two separate occasions, once induced by running, the other time by cycling and hiking on hills
-A weird fatigue and tightness of my shin muscles when running, basically after running just a couple miles the shin muscles would become completely pumped out and exhausted to the point that I couldn't maintain smooth form from foot strike through toe off. Seemed kind of like compartment syndrome, but I lacked some of the other symptoms like the intense pain or tingling. This is only a factor with running, never bothered me just walking or even briskly hiking.
-Knee soreness similar to what I have now, around the lateral side of the knee, brought on by hard cycling, either climbing or pushing a big gear. I’ve thought it was IT Band related, but I never had that specific issue looked at.
-The hip flexor from too much pool running
-A sore spot on the bone on the outside of my left foot
-Not in my legs, but both elbows are always just a bit of tight gripping away from having golfer's elbow flare up. This started after my first trip to a climbing gym a couple years ago, where I climbed for a couple hours until my forearms were toast. Comes back any time I work the forearms at all, if I just squeeze one of those rubber grip rings a few times I'm asking for less fun turning door knobs for a while.
So the past five years for me have gone pretty much like: aggravate something, spend a ton of time waiting for it to go away, come back to activity and progressively build it up, then aggravate something else. I think my biggest problem is that I've let this effectively make me sedentary, as the "recovery" periods take up the bulk of the time, yet I still carry the mindset of an athlete so I think I just overdo it repeatedly, even when I try to take it slow. The more I sit and think about all of this the more I think I just consistently make foolish decisions and end up over worked, and I just get impatient with the recovery, but it kind of seems like I'm a bit too susceptible to this stuff and my body has a hard time dealing with and getting over these things once they have set in.
But even if there isn’t really anything out of the ordinary with me and I’m just a dumbass who constantly overdoes things, I just think I need some guidance in building up my fitness effectively. My hope is that I'm just in a big rut where many parts of my body are just not in the shape to do what I demand of them, and that if I very carefully get them into shape I should be able to eventually train and stay consistently strong and fit like a normal guy in his 20s.
Anyways, I think my next action will be to make another, better attempt at just talking all of this stuff over with my current doctor, and that doesn't get me anywhere I will look somewhere else. And keeping a diary is a great idea, as I do find it difficult to remember exactly how some issues in the past have started, progressed, and concluded.
Thanks again for the ideas everybody, perhaps sometime in the not too distant future your efforts will be repaid with a new partner for adventures!