The existence of both water and sediment at the bed of ice streams is well documented, but there is a lack of fundamental understanding about the mechanisms of ice, water and sediment interaction. We pose a model to describe subglacial water flow below ice sheets, in the presence of a deformable sediment layer. Water flows in a rough-bedded film; the ice is supported by larger clasts, but there is a millimetric water layer submerging the smaller particles. Partial differential equations describing the water film are derived from a description of the dynamics of ice, water and mobile sediment. We assume that sediment transport is possible, either as fluvial bedload, but more significantly by ice-driven shearing and by internal squeezing. This provides an instability mechanism for rivulet formation; in the model, downstream sediment transport is compensated by lateral squeezing of till towards the incipient streams. We show that the model predicts the formation of shallow, swamp-like streams, with a typical depth of the order of centimetres. The swamps are stable features, typically with a width of the order of tens to hundreds of metres.