In this paper, two level handwriting recognition concept is presented, where writer identification is used in order to increase handwriting recognition accuracy. On the upper level, author identification is performed. Lower level consists of a classifiers set trained on samples coming from individual writers. Recognition from upper level is used on the lower level for selecting or combining classifiers trained for identified writers. The feature set used on the upper level contains directional features as well as the features characteristic for general writing style as line spacing, tendency to line skewing and proportions of text line elements, which are usually lost in typical process of handwritten text normalization. The proposed method can be used in applications, where texts subject to recognizing come form relatively small set of known writers.