Recently, a method called Meta-SR has solved the problem of super-resolution of arbitrary scale factor with only one single model. However, it has a limited reconstruction accuracy compared with RDN+ and EDSR+. Inspired by Meta-SR, we noticed that by combining the core idea of Meta-SR and D-DBPN, we might construct a network that has as good image reconstruction accuracy as D-DBPN’s, at the same time, keeps arbitrary scaling function. According to Meta-SR’s Meta-Upscale Module, we designed a different structure called Meta-Downscale Module. By using these two different modules and back-projection structure, we construct an arbitrary back-projection network, which has the ability to enlarge images with arbitrary scale factor by using only one single model, meanwhile, obtains state-of-the-art reconstruction results. Through extensive experiments, our proposed method performs better reconstruction effect than Meta-SR and more efficient than D-DBPN. Besides that, we also evaluated the proposed method on widely used benchmark dataset on single image super-resolution. The experimental results show the superiority of our model compared to RDN+ and EDSR+.