This paper builds on the notion of the so-called orthogonal derivative, where an n-th order derivative is approximated by an integral involving an orthogonal polynomial of degree n. This notion was reviewed in great detail in a paper by the author and Koornwinder in 2012. Here, an approximation of the Weyl or Riemann–Liouville fractional derivative is considered by replacing the n-th derivative by its approximation in the formula for the fractional derivative. In the case of, for instance, Jacobi polynomials, an explicit formula for the kernel of this approximate fractional derivative can be given. Next, we consider the fractional derivative as a filter and compute the frequency response in the continuous case for the Jacobi polynomials and in the discrete case for the Hahn polynomials. The frequency response in this case is a confluent hypergeometric function. A different approach is discussed, which starts with this explicit frequency response and then obtains the approximate fractional derivative by taking the inverse Fourier transform.