Quantum technologies offer a promising route to the efficient sampling and analysis of stochastic processes, with potential applications across the sciences. Such quantum advantages rely on the preparation of a quantum sample state of the stochastic process, which requires a memory system to propagate correlations between the past and future of the process. Here, we introduce a method of lossy quantum dimension reduction that allows this memory to be compressed, not just beyond classical limits, but also beyond current state-of-the-art quantum stochastic sampling approaches. We investigate the trade-off between the saving in memory resources from this compression, and the distortion it introduces. We show that our approach can be highly effective in low distortion compression of both Markovian and strongly non-Markovian processes alike. We further discuss the application of our results to quantum stochastic modelling more broadly.