We analyze a simple model of elastic failure in randomly inhomogeneous materials such as minerals and ceramics. We study a two-dimensional triangular lattice with nearest-neighbor harmonic springs. The springs are present with probability p. The springs can only withstand a small strain before they fail completely and irreversibly. The applied breakdown stress in a large, but finite, sample tends to zero as the fraction of springs in the material approaches the rigidity percolation threshold. The average initial breakdown stress, ${\ensuremath{\sigma}}_{b}$, behaves as ${\mathrm{\ensuremath{\sigma}}}_{\mathrm{b}}^{\mathrm{\ensuremath{\mu}}}$\ensuremath{\approxeq}[A(p)+B(p)ln(L)${]}^{\mathrm{\ensuremath{-}}1}$, where L is the linear dimension of the system and the exponent \ensuremath{\mu} is between 1 and 2. The coefficient B(p) diverges as p approaches the rigidity percolation threshold. The breakdown-stress distribution function ${F}_{L}$(\ensuremath{\sigma}) has the form ${\mathit{F}}_{\mathit{L}}$(\ensuremath{\sigma})\ensuremath{\approxeq}1-exp[-${\mathit{cL}}^{2}$exp(-k/${\mathrm{\ensuremath{\sigma}}}^{\mathrm{\ensuremath{\mu}}}$)]. The parameters c and k are constants characteristic of the microscopic properties of the system. The parameter k tends to zero at the rigidity percolation threshold. These predictions are verified by computer simulations of random lattices. The breakdown process can continue until a macroscopic elastic failure occurs in the system. The failure occurs in two steps. First, a number of springs fail at approximately the strain which causes the initial failure. This results in a system which has zero elastic modulus. Finally, at a considerably larger strain a macroscopic crack forms across the entire sample.