SUMMARYTravel time reliability is an emerging evaluation measure for highway network service levels. This paper focuses on travel time reliability indexes for a highway network and proposes new travel time indexes for users. The paper also discusses and compares several previously and newly proposed indexes for use by both highway users and operators. Because this comparative study requires travel time variation, two models are developed for estimating travel time variation. One is the multi‐hierarchical stochastic model for estimating travel time variation under uncertainty in demand and service. The other is the temporal and spatial transition model using actual traffic detector data. Then, new indexes for assessing travel time reliability are proposed. Because operators of highway network are interested in the degree of congestion and delay, the previously proposed indexes are regarded as operator‐side indexes. The highway users are, however, interested in the accuracy of their travel time and in the earlier arrival at their destination. Thus, this paper proposes the user‐side indexes. Travel time reliability indexes, including previously proposed indexes, are calculated on the basis of the cumulative distribution function outcome from these models and are compared and discussed. In two sections, 11 indexes with different characteristics are compared. One of the results is that these indexes behave differently for the same route. This suggests that understanding an index's formulation and characteristics is important for selecting the appropriate index according to both the purpose of use and the characteristics of the study route. Another important result is that indexes useful for the user side do not always agree with the operator‐side indexes. Finally, the results of the comparative study suggest that the combination of average travel time and an appropriate travel time reliability index is very important for both users and operators for assessing the route reliability properly. Copyright © 2012 John Wiley & Sons, Ltd.