During the last two years the *metrics project (MEasuring The Reliability and perceptions of Indicators for interactions with sCientific productS) analyzed open metrics regarding perception and reliability. The vision of using altmetrics promises a variety of benefits especially in promoting open access and open science. While new opportunities are being discussed among *metrics experts, one essential question is how these possibilities are being perceived in the community of researchers - whose activities these metrics are supposed to evaluate. The fact that altmetrics are to a large extent derived from interactions on social media platforms leads to questions about their reliability: signals happen on a variety of online platforms that can vastly differ regarding their aims, functionalities, modes of communication as well as topics, and other characteristics. Also, the varying degrees with which different user groups are represented by different kinds of altmetrics have an influence on their meaning. And even beyond the involved platforms and actors, the individual types of interactions that are counted as altmetrics themselves emerge under deeply heterogeneous circumstances and express substantially different things. Our project results reveal patterns regarding career stages, choice of platforms, use of functions, etc. that need to be taken into account when constructing meaningful metrics as an addition to established bibliometrics. Only if we know what altmetrics can and can’t be used for we may apply and develop any kind of *metrics in a responsible way. Ready-to-go-solutions may be easy to implement, but just because data from scholarly communication and social media platforms is readily available it does not mean that it can be universally used for every purpose. The poster will summarize results and recommendations from the project and relate them to the wider field of open science.