We propose a double-parameter regularization scheme for dealing with the backward diffusion process. Considering the smoothing effect of Yosida approximation for PDE, we propose to regularize this ill-posed problem by modifying original governed system in terms of a pseudoparabolic equation together with a quasi-boundary condition simultaneously, which consequently contains two regularizing parameters. Theoretically, we establish the optimal error estimates between the regularizing solution and the exact one in terms of suitable choice strategy for the regularizing parameters, under a-priori regularity assumptions on the exact solution. The a-posteriori choice strategy for the regularizing parameters based on the discrepancy principle is also studied. To weaken the heavy computational cost for solving the discrete nonsymmetric linear regularizing system by finite difference scheme, especially in higher spatial dimensional cases, the block divide-and-conquer method together with the properties of the Schur complement is applied to decompose the linear system into two half-size linear systems, one of which can be solved by the diagonalization technique, and consequently an efficient parallel-in-time algorithm originally developed for direct problem is applicable. Our proposed method is of much lower complexity than the standard solver for the corresponding linear system. Finally, some numerical examples are presented to verify the efficiency of our proposed method.