For some time now the first two authors have been developing a theory of ``higher order tensors'' which they call (derivative) strings. This work has been motivated by examples from statistics where such objects occur naturally. By a ``higher order tensor'' is meant an object like a tensor but whose transformation rule under a change of co-ordinates involves arbitrarily high derivatives of the co-ordinate change map. For those familiar with the language higher order-tensors can be described succinctly as sections of some vector bundle associated to the principal bundle of infinite order frames. The present paper looks at the concept of higher order differentiation. The authors are interested in the idea of repeated (possibly covariant) differentiation of vector or other tensor fields. This gives examples of certain types of strings which are not ``structurally symmetric''. These are called differentiation strings. The paper gives both co-ordinate free and co-ordinate based definitions of these strings. The former is of interest as these are examples of strings that don't fall readily in to the jet formalism. The latter allows calculations using the powerful index notation which the authors have developed in previous papers. This notation handles the type of complicated combinatorics which arises when, for instance, two Taylor series are composed. References to other work of the authors in this area and to its applications to statistics can be found in the bibliography of this paper.