Just 23 miles from Interstate 95, Varnville was originally known as "Dixie." One of the original founders of this quaint town was James Varn, who operated a saw mill. It was said that the aristocrats settled in Varnville.