Audio Chord Estimation (ACE) holds a pivotal role in music information research, having garnered attention for over two decades due to its relevance for music transcription and analysis. Despite notable advancements, challenges persist in the task, particularly concerning unique characteristics of harmonic content, which have resulted in existing systems' performances reaching a glass ceiling. These challenges include annotator subjectivity, where varying interpretations among annotators lead to inconsistencies, and class imbalance within chord datasets, where certain chord classes are over-represented compared to others, posing difficulties in model training and evaluation. As a first contribution, this paper presents a novel methodology for assessing inter-annotator agreement in chord annotations, using metrics that extend beyond traditional binary measures. Our analysis demonstrates that incorporating the distance metrics based on perceptual concepts of consonance significantly enhances agreement scores. Expanding on these findings, we introduce a novel ACE conformer-based model that integrates consonance concepts into the model through consonance-based label smoothing. The proposed model also addresses class imbalance by separately training models to detect root, bass, and all note activations, enabling the reconstruction of chord labels from this information.