Tiny cracks that exist in steel beams have poor continuity and low contrast in images, posing a huge challenge to crack detection using image-based approaches. When complex backgrounds exist, the existing deep learning methods are usually unable to perform effective feature transfer and fusion for crack feature mapping, and they cannot accurately distinguish crack features from similar backgrounds. In this article, we propose a fusion segmentation algorithm, using the fully convolutional network (FCN) and structured forests with wavelet transform (SFW) to detect tiny cracks in steel beams. First, five neural networks based on the FCN framework are constructed to extend the global characteristics of tiny cracks. Second, a fine edge detection approach using multi-scale structured forests and wavelet maximum modulus edge detection to refine the characteristics of tiny cracks are proposed. Here, a competitive training strategy is used to address the SFW parameter optimization problem. Finally, we fuse the multiple probability maps, acquired from both the optimal FCN model and the SFW classifier, into a merged map, which can segment tiny cracks with robustness better than the comparison approaches. The experimental results show that compared with state-of-the-art algorithms and other segmentation approaches, the proposed algorithm realizes better segmentation in terms of quantitative metrics.