First, pick your soap carefully. My niece ended up being allergic to the bar soap I used.

Grate up the bar. Watch your fingers when you get near the end!

Very slowly add about two(2) tablespoons of water. You don't want it to end up too watery. You can add more, but you can't take it out! Mix it up with a fork.

Split the soap into three equal sections (Or less if you want more of just one color of soap)

Mix in a good squeeze of food coloring. I would warn against using red, because it does have a tendency to stain the tub more so than other colors.

Shape it into a crayon like shape, or press it into a mold. Let dry for about two days in a cool dry place.

Just a warning that it ends up being super slippery and hard to grab a hold of. So something other than a crayon like shape might be easier to hold!

And they will change the water to whatever color the crayon is :)