In previous years, an interest has developed in the coded representations of video signals allowing independent manipulation of semantically independent elements (objects). Along these lines, the ISO standard MPEG-4 enhances the traditional concept of the video sequence to convert it into a synchronized set of visual objects organized in a flexible way. The real-time generation of a bitstream according to this new paradigm, and suitable for its transmission through either fixed- or variable-rate channels, results in a challenging new bit allocation and rate control problem, which has to satisfy complex application requirements. This paper formalizes this new issue by focusing on the design of rate control systems for real-time applications. The proposed approach relies on the modelization of the source and the optimization of a cost criterion based on signal quality parameters. Different cost criteria are provided, corresponding to a set of relevant definitions of the object priority concept. Algorithms are introduced to minimize the average distortion of the objects, to guarantee desired qualities to the most relevant ones, and to keep constant ratios among the object qualities. The techniques have been applied to a coder implementing the MPEG-4 video verification model, showing good properties in terms of achievement of the control objectives.