The universal sl2sl2 invariant of string links has a universality property for the colored Jones polynomial of links, and takes values in the ℏℏ-adic completed tensor powers of the quantized enveloping algebra of sl2sl2. In this paper, we exhibit explicit relationships between the universal sl2sl2 invariant and Milnor invariants, which are classical invariants generalizing the linking number, providing some new topological insight into quantum invariants. More precisely, we define a reduction of the universal sl2sl2 invariant, and show how it is captured by Milnor concordance invariants. We also show how a stronger reduction corresponds to Milnor link-homotopy invariants. As a byproduct, we give explicit criterions for invariance under concordance and link-homotopy of the universal sl2sl2 invariant, and in particular for sliceness. Our results also provide partial constructions for the still-unknown weight system of the universal sl2sl2 invariant.