Although strong nonlinear interactions between quantized excitations are an important resource for quantum technologies based on bosonic oscillator modes, most electromagnetic and mechanical nonlinearities are far too weak to allow for nonlinear effects to be observed at the single-quantum level. This limitation has been overcome in electromagnetic resonators by coupling them to other strongly nonlinear quantum systems such as atoms and superconducting qubits. We demonstrate the realization of the single-phonon nonlinear regime in a solid-state mechanical system. The single-phonon anharmonicity in our system exceeds the decoherence rate by a factor of 6.8, allowing us to use it as a mechanical qubit and demonstrate initialization, readout, and single-qubit gates. Our approach provides a powerful quantum acoustics platform for quantum simulations, sensing, and information processing.