You don't need an Adventure Pass when you park above the "End maintained road" sign on Lytle Creek Road.
Map of Lytle Creek Fee Area
Or most of it....If you start the hike aprox. 1.2 miles before you reach the end of the paved road and follow the river bed up stream. There is a stream bed that flows down from the canyon between Chalk Peak and it's neighboring peak to the east. Follow this stream bed up and to the left until you reach the eastern talus slopes of Chalk Peak, at just about the point where you start to see tall pine trees. Then it's up to your right on very loose rocky slopes. Continue on up towards the obvious ridge and then on to the summit. Only proplem is that there really isn't a turn out along side that portion of the road. Have fun.