AbstractIn a remote lecture photographed by multiple cameras, the task of selecting the video streams to be transmitted to a remote location is now performed mainly by the lecturer, and represents a large burden. This paper proposes a method for automating this task. The lecture context is expressed as a predefined sequence of transitions between lecture states, and the relationship between the lecture context and the video streams to be transmitted is implemented as rules. Since the lecture states are constructed as combinations of easily observable information from real lecture scenes, the appropriate video streams can be selected without an understanding of lecture content by matching the extracted lecture state sequences against the predefined rules. Experiments with simulated lectures using a blackboard and overhead projector have shown that the proposed method is effective in real‐time selection of the video streams to be transmitted. © 2003 Wiley Periodicals, Inc. Syst Comp Jpn, 34(3): 64–75, 2003; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.1195