Does 360-degree feedback work in different industries?:A between-industry comparison of the reliability and validity of multi-source performance ratings

In order to determine the usefulness of multi-source rating in different types of organizations, this study explored differences among organization types in four areas: leniency, interrater agreement, relationships between these ratings and effectiveness, and the relationship between agreement and effectiveness. Used self, subordinate, peer, and supervisor ratings for 1,080 target managers in six types of organizations: education, military, government, manufacturing, finance, and health. Interrater agreement was measured in three ways: an index of variance, a point-difference categorization method, and categories of self-other agreement. Results indicated that a leniency bias was present in educational institutions, after controlling for demographic characteristics. Interrater agreement was lowest in government agencies and highest in education and manufacturing organizations. In private sector organizations, more poor-performing managers tended to over-estimate their performance relative to the perceptions of others. Interrater agreement was positively related to effectiveness especially in education and finance organizations. Results suggest that multi-source feedback may work differently in different types of organizations, and such differences may need to be taken into account by researchers, practitioners, and feedback recipients.