Photometric redshift estimation is becoming an increasingly important technique, although the currently existing methods present several shortcomings which hinder their application. Here it is shown that most of those drawbacks are efficiently eliminated when Bayesian probability is consistently applied to this problem. The use of prior probabilities and Bayesian marginalization allows the inclusion of valuable information, e.g. the redshift distributions or the galaxy type mix, which is often ignored by other methods. In those cases when the a priori information is insufficient, it is shown how to `calibrate' the prior distributions, using even the data under consideration. There is an excellent agreement between the 108 HDF spectroscopic redshifts and the predictions of the method, with a rms error Delta z/(1+z_spec) = 0.08 up to z<6 and no systematic biases nor outliers. The reliability of the method is further tested by restricting the color information to the UBVI filters. The results thus obtained are shown to be more reliable than those of standard techniques even when the latter include near-IR colors. The Bayesian formalism developed here can be generalized to deal with a wide range of problems which make use of photometric redshifts. Several applications are outlined, e.g. the estimation of individual galaxy characteristics as the metallicity, dust content, etc., or the study of galaxy evolution and the cosmological parameters from large multicolor surveys. Finally, using Bayesian probability it is possible to develop an integrated statistical method for cluster mass reconstruction which simultaneously considers the information provided by gravitational lensing and photometric redshifts.

36 pages, AAS Latex, submitted to ApJ