The Importance of a Reputation Model
Reputation is an important part of the People Profile. We not only want to work with someone based on his or her expertise, but also on their reputation regarding teamwork and their history of value creation.
The reputation model is still a frontier. We are not aware of any organization that has the perfect model; we continue to do research in this area.
We have learned a lot from online community reputation models. Most of us have seen Amazon’s product ratings and have used them in our considerations to purchase. Communities use reputation systems to recognize the contribution of its members. Wikipedia’s WikiTrust, for example, relies on an analysis of user contributions to articles, and calculates positive or negative increments of reputation when a new contribution is made.
One definition of reputation is “information used to make a value judgment about an object or person” (Building Web Reputation Systems).
A reputation model has several functions. It helps build trust. It helps us identify and recognize those who are creating value. It provides insight to who delivers quality help, and it can be used for performance assessment purposes. It may also help to select with whom we want to collaborate. In this way, a reputation model can be a motivator and stimulate the right behavior.
Designing a Reputation Model
Designing a good reputation model is not easy. There are many factors that should be input to the model. Simple ways to recognize people's contributions are often done by star ratings, thumbs up/thumbs down voting, or flags. These are explicit ways to indicate the helpfulness of content or the value of an interaction. Explicit input is helpful but it can easily be compromised as explicit feedback is always negotiable. We need to include implicit indicators, which are more powerful and more complicated. Implicit indicators come from patterns and trends of people's behavior, including but not limited to:
- the frequency at which I am invited to collaborate
- the frequency at which I opt-in to help
- the success of swarms I participate in
Those who participate in a collaborative effort to resolve a request should inherit the customer sat rating and/or the customer effort score for that request. Over time, a pattern will emerge that is telling about the value one is creating. Reputation can also be inherited from the content I am associated with. Dr. Marc Smith, a sociologist who studies online communities, proposes that a reputation model should be based on 20% explicit input and 80% implicit indicators.
Therefore, we need to look at different indicators for reputation building in a collaborative environment. We use the following design principles for building a reputation model:
- No arbitrary limits
- Criteria-based recognition (not competition-based)
- Acknowledge all the skills
- Include a mix of implicit and explicit feedback
- Evolves over time
- Unique to each individual
- Integrity (not “gameable”)
There is one exception to the principle of abundance in this case. An explicit feedback mechanism such as kudos or karma points can be an element of the reputation model. Experience with these feedback mechanisms has shown that these work best when they are limited to a certain number per person per time period. In this case, a scarcity model helps create value around the explicit feedback.
The reputation model should use the same classification model as the skills profile. The difference between the skills profile and the reputation model is the skills profile describes my areas of expertise. My reputation is my history of value creation or what I have done with the skills I have. Some organizations use badges to reflect contribution in skills areas.
When designing the reputation model, take the following considerations into account:
- Decide what skills are needed for organizational success
- Deep skills, technical expertise
- Broad or transferable skills
- How many levels of competency do we want / need
- Major, minor
- Interested, capable, master
- 1- 5 (1= novice.... 5= master)
- Define the criteria for each competency level
As we mentioned the reputation model is an area that the Consortium members continue to evolve. There will be more to come on this topic.