We have measured the ac nonlinear susceptibilities ${\ensuremath{\chi}}_{3}^{\mathcal{'}}$, ${\ensuremath{\chi}}_{5}^{\mathcal{'}}$, and ${\ensuremath{\chi}}_{7}^{\mathcal{'}}$ of very dilute AgMn spin glasses above and below the transition temperature ${T}_{g}$ as a function of frequency, temperature and magnetic field. In the static limit, these quantities display well-defined critical singularities at ${T}_{g}$ and imply a divergence of the spin-glass correlation length. The nonlinear susceptibilities can be fitted above ${T}_{g}$ to powers of frequency with weakly temperature-dependent exponents. While the effective exponents satisfy the relations imposed by static and dynamic scaling, the observed temperature and frequency roundings suggest that the spin-glass correlation length does not actually become infinite and is cut off at a length scale of 2000 A\r{}. Activated behavior in the vicinity of ${T}_{g}$ with a logarithmic dynamic scaling also accounts for the measured dynamical nonlinear susceptibilities. The measured critical exponents (\ensuremath{\beta}=0.9, \ensuremath{\gamma}=2.3, \ensuremath{\delta}=3.3, \ensuremath{\nu}=1.3, z=5.4) are not mean field, and differ from the exponents obtained by Monte Carlo simulation of short-range Ising systems.