Actually, Sepia is often used as a base when coloring, and prefered over b&w. I never leave a single pixel uncolored/gray. White, black, and even gray are all colored.
Perhaps you're getting gray because there is not enough color blocking it out. Could also be that the image has too much contrast, and/or is too dark.