I know the desert mountains are cool and beautiful, and not to be overlooked, but, don't think you should overlook the beach, either, which it seems, no one's really mentioned, except in passing. Yeah, if the weather is no good, certainly head into the desert. But, if it's one of those sparkling San Diego days, you should definitely take a look at a beach.
Torrey Pines is one such magical place.
http://www.torreypine.org/