Increasing interest in using machine learning systems for decision making and support in the public sector has raised questions as to how these technologies can be designed, implemented and managed responsibly. This short discussion paper describes some relevant social and technical potentials and perils of machine learning by relating them to different groups of public sector values outlined in the public administration literature. Practitioners may find this structure useful to help them understand different dimensions of responsibility they may wish to consider if they are considering using these technologies, and how they link to developing work and tools in the field.