This paper is concerned with accurate and robust estimation of the fundamental matrix. We show that, given certain conditions, a basic linear algorithm can yield excellent accuracy, in some cases two orders of magnitude better than sophisticated algorithms. The key element of the success is the relative accuracy of displacement estimates used as input and the use of statistical distribution of the errors. We propose a low-level, gradient-based (as opposed to feature-based) algorithm based on the Hough Transform to extract the low-level measurements. We show that it is much more efficient, both in terms of computational expense and accuracy of the final estimate, to remove the errors in the intermediate representation (optic-flow) than to attempt to improve the final estimate by complicated nonlinear algorithms. Experimental results are also included.