The article presents an application of combined classifier in the medical decision support system for breast cancer diagnosis. Apart from the canonical malignant vs. non-malignant problem we introduced a third class — fibroadenoma, which is a benign tumor of the breast often occurring in women. Medical images are delivered by the Regional Hospital in Zielona Góra, Poland. For the process of segmentation and feature extraction, adaptive thresholding and competitive neural networks are used. To increase the overall accuracy of the pattern recognition step we selected the classifiers using diversity measures to achieve a heterogeneous ensemble. A two-step selection, combining the advantages of pairwise and non-pairwise diversity measures is proposed. Experimental investigation proves that the introduced method is more accurate than previously used classification approaches.