We propose a reduced algebraic cost based on pairwise epipolar constraints for the iterative refinement of a multiple view 3D reconstruction. The aim is to accelerate the intermediate steps required when incrementally building a reconstruction from scratch. Though the proposed error is algebraic, careful input data normalization makes it a good approximation to the true geometric epipolar distance. Its minimization is significantly faster and obtains a geometric reprojection error very close to the optimum value, requiring very few iterations of final standard BA refinement. Smart usage of a reduced measurement matrix for each pair of views allows elimination of the variables corresponding to the 3D points prior to nonlinear optimization, subsequently reducing computation, memory usage, and considerably accelerating convergence. This approach has been tested in a wide range of real and synthetic problems, consistently obtaining significant robustness and convergence improvements even when starting from rough initial solutions. Its efficiency and scalability make it thus an ideal choice for incremental SfM in real-time tracking applications or scene modelling from large image databases.