Recent research has shown that words or morphemes that are closer to each other in linear order tend to have higher statistical inter-predictability, measured as mutual information. We offer an explanation for this in terms of holistic chunking of inter-predictable symbols, providing an efficiency gain in the retrieval of stored symbols to encode a message. Inter-predictable chunking then interacts with structural priming, to produce the schematic linear structures at the heart of both syntax and morphology. We therefore argue that predictability and efficiency play a key role in the emergence of grammatical structure, going beyond previous information-theoretic analyses of natural language. In this paper we articulate these key principles at the heart of grammatical linearisation, and use a simple computational implementation to show that they do indeed produce natural-language-like structures, using NP-internal ordering as a case study.