Modularity Matrices for software systems can be put in block-diagonal form, where blocks are higher-level software modules, in a hierarchy of modules. But the exact module boundaries are often blurred by the uncertainty whether given matrix elements are module members or outliers. This paper provides an algorithm to determine module sizes. As a consequence the algorithm also decides which matrix elements are outliers. Matrix elements are weighted by their Affinity — an exponential function of the off-diagonality. The module size is given by the positive consecutive elements of the eigenvectors corresponding to the largest eigenvalues of this weighted symmetrized Modularity Matrix. By means of case studies, we illustrate the idea that outliers not only indicate the need for system redesign, but explicitly point out to problematic design spots. This work extends the applicability of linear algebra spectral methods to Modularity Matrices, at higher software abstraction levels than previously shown.