Background. The aim of this study was to confirm the effectiveness of open-label placebo (OLP) in Japanese patients with chronic low back pain (CLBP), similar to previous reports, and to investigate its short- and medium-term effects in this study population. Methods. Fifty-two patients with CLBP were randomized into a treatment as usual (TAU) group (n = 26) or an OLP + TAU group (n = 26) for 12 weeks. The TAU included advice to remain active and exercise in conjunction with recent psychological education based on a self-management strategy. In contrast, participants in the OLP + TAU group were instructed to take two OLP capsules a day. Outcome measures were assessed at baseline and at weeks 3 and 12 using the Roland–Morris Disability Questionnaire (RMDQ), Numerical Rating Scale (NRS) for pain intensity, and the Timed-Up-and-Go (TUG) test. Difference in outcomes between the two groups was compared at the two follow-up points. Results. Although all participants completed the 3-week follow-up, four patients (two in each group) were lost to follow-up beyond week 3. There were no significant intergroup differences in changes in the RMDQ score ( p = 0.40 ), pain-NRS score ( p = 0.19 ), and TUG time ( p = 0.98 ) at week 3. Two-way repeated measure analyses of covariance showed significant time-course effects but did not show group effects or any interactions between the time-course and group in terms of the RMDQ score. However, it did not show any effects in the pain-NRS score and TUG time at week 12. Conclusions. The OLP + TAU group showed no superior findings in comparison with the TAU group after 3 weeks and 12 weeks for Japanese patients with CLBP. Nonetheless, significant improvements in functional disability were observed in both groups.