it gets on your skin, and water cannot take it off.
It's an oil, so it's not water-soluble. Lots of scrubbing with ordinary soap and hot water will basically work, but not as well as some of the specialty soaps on the market. Your skin is sensitive to incredibly tiny amounts of this stuff, so just removing most of it just isn't good enough.
After the oil is gone, it's up to your own body to stop reacting.
That's where strong astringents come in. Jewelweed sap is a great example.
In the absence of jewelweed, you can try tannins: make a poultice from oak bark or fir bark or witch hazel bark [many other plants work too]).
My theory is that salt water has an astringent-like effect. The alternative theory would be some kind of catalysis/reaction involving trace minerals in seawater (zinc or copper, maybe?). I think I know how to do a controlled experiment (won't be even single-blind though), stay tuned.