Color superconducting state has been known as a possible phase of quark matter with sufficiently large baryon number density so as for perturbative analysis to be valid. We point out that a color ferromagnetic state is another possible phase of such a sufficiently dense quark matter. Furthermore, we show under reasonable choices of parameters that the color ferromagnetic phase is energetically more favored than the color superconducting phase in the quark matter with smaller baryon number density. Supposing that increasing baryon density in neutron stars transforms nuclear matter into the quark matter of the color ferromagnetic phase, not color superconducting phase, we find that a critical mass of the neutron star with such an internal structure is about 1.6M⊙.