Breast cancer is the most common cancer among women worldwide. Breast cancer is caused by the overexpression of genes that facilitate breast cell proliferation. Estrogen receptor alpha (ERα) mediation is mostly responsible for the development of malignant tumors by regulating the transcription of various genes as a transcription factor. ERα is regarded as an important receptor for the proliferation of this disease and its inhibition is necessary for the treatment of breast cancer. In our study, the aim is to find out potential inhibitors for ERα by docking 100 anticancer constituents of different halogen-based derivatives against ERα. Among the 100 X-based derivatives, 20 ligands were selected based on the interaction energy ranging from −6.7kcal/mol to −8.2kcal/mol and lower values of inhibition constant (0.92–11.77μmol). We have performed a comprehensive analysis of five most popular ligands (L17, L57, L61, L67, and L70) among these 20 selected derivatives. The interaction analysis is mainly stabilized by making different interactions including hydrogen bonding, hydrophobic, electrostatic, and halogen bonding with the active site of ERα. A quantum study based on frontier molecular orbitals (FMOs), molecular electrostatic potential (MEP), and global chemical reactivity descriptor (GCRD) is carried out to explore the structural chemistry of our most popular ligands which indicates that they are quite reactive and kinetically stable. Additionally, we also assess the ADMET profiles to predict the toxicity and drug-likeness of our lead compounds. Our selected ligands have the highest absorption and good clearance rate and have no AMES toxicity, skin sensitization, and hepatotoxicity. Molecular dynamics simulations were run to check the conformational stability of apo form of ERα and complex state, at 60-ns time scale. The molecular dynamics simulations are based on the plots of RMSD, RMSF, Rg, SASA, and the number of H-bonds. The results of RMSD and RMSF showed that the lead compounds L17, L57, L61, L67, and L70 are most stable and have no significant residual fluctuations and deviation. The analysis of the plots of Rg, SASA, and the number of H-bonds revealed that the complexes L17, L57, and L70 are most stable. So, the lead anticancer compounds L17, L57, L61, L67, and L70 are the most promising inhibitors against ERα of breast cancer. The comprehensive analysis of all studied parameters highlighted that our selected ligands have great potential to inhibit ERα. So, it can be concluded that the selected halogen-based derivatives can promote rational drug design for the target therapies of ERα of breast cancer.