Importance Ratings in Multi-Source Feedback

Most of the research on multi-source feedback focused on skill rating. This study investigated both skill rating and importance rating. It was found that interrater correlations on both skill rating and importance rating are low. But withinlearner interrater correlations were higher than between-learners interrater correlations. Skill rating was correlated to importance rating. Follow-up analysis indicated that within -learner skill-importance correlation is related to the overall skill level of the learners, suggesting that learners who fit with their jobs as perceived by a particular rater constituency group are generally rated high on skill rating scale by the same rater constituency group.