Generalising the shuffle product over a vector space, we define the sticky shuffle product over an associative algebra
and endow it with a Hopf algebra structure. In various examples we exhibit connections with iterated sums, integrals and stochastic integrals and examine the significance of the Hopf algebra coproduct and antipode in these contexts. Finally we show that there is a homomorphism from the Yangian Hopf algebra Y(glN) to the corresponding sticky shuffle product Hopf algebra.