With the development of online applications based on social networks, many different approaches have emerged to evaluate the service that these applications provide. Reports made by end users regarding the consumer's experience or opinion are commonly used to rate the quality of different online services. Therefore, ensuring the authenticity of the users' reports, and the detection of malicious users' dishonest reports, have both become important issues to achieve accuracy in the rating of such services. In this paper, we propose and evaluate a private-prior peer prediction-based trustworthy service rating system, which requires users to report their prior and posterior beliefs regarding whether their peers will report a high-quality opinion of the service. The reports are made to a data processing center which evaluates the users' trustworthiness by applying a strictly proper scoring rule, and removes reports received from users whose trustworthiness rating is low. This peer prediction method is compatible with incentives to motivate users to report honestly. In addition, an unreliability index is proposed to identify malicious users, and malfunctioning or unreliable users who have a high error rate in making judgments about quality. Thus, reports with high unreliability values will also be excluded from the service rating system. By combining trustworthiness and unreliability, malicious users face the dilemma that they cannot receive both a high trustworthiness and low unreliability rating simultaneously when their reports are false. Simulation results indicate that the proposed peer prediction-based trustworthy service rating can identify malicious and unreliable behaviors effectively and motivate users to report truthfully, and that a relatively high service rating accuracy is achieved by the proposed system.