From Wooljersey

Scratch refers to handicap marks in scratch racing, which some running races like the Dipsea still use. These are distance markers, but you can also run a handicap with a time bonus. This is where "On your mark" comes from. A scratch racer would be the fastest, "starting from scratch," i.e. the line in the sand or dirt.