Abstract The propagation characteristics of curved detonation have been extensively investigated in the present study using three types of stoichiometric test mixtures (2H2+O2+3Ar, C2H4+3O2, and CH4+2O2) in a new experimental facility consisting of a straight channel joined to a helical channel. The helical channel with different inner radii (Ri = 60 mm, 70 mm, and 80 mm) and uniform outer radius (Ro = 100 mm) were used. Flame propagation through the helical channel was observed by high-speed CCD camera, and the trajectories of triple points on the detonation waves were obtained using a soot-deposition plate. The results clearly identify three detonation propagation modes, namely, a stable mode, critical mode, and unstable mode. A definite flame shape, which is perpendicular to the inner wall of the channel, is observed for the stable mode. The smooth flame front becomes more wrinkled as p0 and/or Ri decreases. Three types of patterns of cellular structure are observed for unstable mode, which present specific features of the periodical variation with various mixtures and curvatures, such as periodical disappearance near the inner wall, “petals”-like structure, and intermittent structure. Moreover, the ratio Ri/λ is introduced as a sensitivity parameter to characterize the ability of a detonation. The detonation limits are found to be at the range of 2.6 ≤ Ri/λ ≤ 4.8 in the present study.