In this paper we examine a natural language processing and machine learning approach to help assess the quality of railway hazard logs. The focus is on highlighting red flags in the hazard log content to help improve the accuracy and quality of the contents and so the speed of risk reviews. Data is presented that indicate the approach has potential for significant savings in time and increased quality. The tool is one of a number that we are developing as part of an initiative to improve rail system development and operation by employing artificial intelligence (AI) to augment existing methods in the context of a wider system engineering approach. This will in turn lead to rail systems becoming more sustainable and resilient.