Do you have a route picked out? It seems like the two basic options are a) the JMT from Happy Isles to Sunrise and then on to Tuolumne and b) starting off the same way, but then branching off to Merced Lake and up to Vogelsang, then down to Tuolumne. Four Days should be more than enough for either of those - personally I'd probably budget 3 days for either one, so four days would allow a more leisurely pace. For route A, you'd probably get to at least the Little Yosemite Valley on day 1, Sunrise High Sierra Camp on day 2, and roll into Tuloumne on day 3. For route B, you might be able to get to Merced Lake on day 1, Vogelsang on day 2, and then Tuolumne on day 3 - day 1 would be pretty long though, so for the full 4 days you could stop day 1 in Little Yosemite.