We present a simple analytic gravitational solution which describes the holographic dual of a (2+1)-dimensional conductor which goes beyond the usual linear response. In particular it includes Joule heating. We find that the nonlinear frequency-dependent conductivity is a constant. Surprisingly, the pressure remains isotropic. We also apply an electric field to a holographic insulator and show that there is a maximum electric field below which it can remain an insulator. Above this critical value, we argue that it becomes a conductor due to pair creation of charged particles. Finally, we study (1+1)- and (3+1)-dimensional conductors at the nonlinear level; here exact solutions are not available and a perturbative analysis shows that the current becomes time dependent, but in a way that is captured by a time-dependent effective temperature.