Reputation Models & Badging
As Consortium Member companies continue to evolve how we think about and recognize the contribution people make to the success of an organization, Reputation Models and Badging have proven to be helpful approaches.
- Reputation
- Represents the history of contribution (value) a person has created with their skills and abilities, as seen by the people receiving the value.
- Reflects the way a person is applying those skills and abilities, and if they are doing so in a way that contributes to the team's overall success.
- Badges
- Represent a person's skills and abilities and the application of those skills and abilities. Badges are criteria-based.
- Badges can be achieved by gaining skills and abilities and applying them to a job function, but this does not mean that the skills and abilities gained are seen as valuable.
Like so much of Intelligent Swarming, these models may challenge the traditional ways we have thought about our organizations and the way people contribute to our overall success. We continue to learn how Reputation models and Badges work in support organizations as Member companies test ideas.
Take this scenario as an example:
A knowledge worker has been with a company for many years and is a deep technical expert in a specific, highly impactful feature of the company's software product. This knowledge worker has gained many defined skills, competencies, and certifications in this feature and closes the most service requests related to it. These achievements are reflected in the Platinum Badge they have been granted in this feature. At the same time, this person is difficult to communicate with, has little empathy for the customer's position, and rarely helps others who are trying to learn. While they have the opportunity to use the skills they demonstrated to receive a Platinum Badge, the way they are applying that knowledge results in a very low reputation score. While having the required skills, the knowledge worker is not seen as adding valuable contributions by customers and peers.
Using both a Reputation model and Badges can be a powerful way to tap into intrinsic motivators. When designing Reputation Models and Badging, we want to leverage the work we've already done by using the attributes defined in our Common Classification Model.
The Importance of a Reputation Model
Reputation is an important part of the People Profile. We not only want to work with someone based on their expertise, but also on their reputation regarding teamwork and history of valuable contribution.
One definition of reputation from Stack Overflow is “a rough measurement of how much the community trusts you; it is earned by convincing your peers that you know what you’re talking about.” (Stack Overflow: what is reputation?)
A reputation model has several functions. It helps us:
- Build trust
- Identify and recognize those who are contributing to success
- Gain insight to who delivers quality help
- Assess performance
- Select with whom we want to collaborate
In this way, a reputation model can be a motivator and stimulate the right behavior. The ways to develop a functioning Reputation Model for an Intelligent Swarming system are still emerging, but we have learned a lot from online community reputation models. Most of us have seen Amazon’s product ratings and have used them in our considerations to purchase. Communities use reputation systems to recognize the contribution of its members. Wikipedia’s WikiTrust, for example, relies on an analysis of user contributions to articles, and calculates positive or negative increments of reputation when a new contribution is made.
Design Considerations for a Reputation Model
Designing a good reputation model is not easy. There are many factors that could be inputs to the model. Simple ways to recognize people's contributions are often done by star ratings, thumbs up/thumbs down voting, or flags. These are explicit ways to indicate the helpfulness of content or the value of an interaction. Explicit input is helpful, but it can be easily compromised as explicit feedback is always negotiable. We need to include implicit indicators, which are more powerful and more complicated. Dr. Marc Smith, a sociologist who studies online communities, proposes that a reputation model should be based on 20% explicit input and 80% implicit indicators. Implicit indicators come from patterns and trends of people's behavior, including but not limited to:
- the frequency at which I am invited to collaborate
- the frequency at which I opt-in to help
- the success of swarms I participate in
Those who participate in a collaborative effort to resolve a request should inherit the customer satisfaction rating and/or the customer effort score for that request. Over time, a pattern will emerge that is telling about the value of contribution one is creating. Reputation can also be inherited from the content I am associated with.
Therefore, we need to look at different indicators for reputation building in a collaborative environment. We use the following design principles for building a reputation model:
- Abundant
- No arbitrary limits
- Criteria-based recognition (not competition-based)
- Diverse
- Acknowledge all the skills
- Include a mix of implicit and explicit feedback
- Dynamic
- Evolves over time
- Unique to each individual
- Integral
- Multi-faceted
- Not “gameable”
There is one exception to the Principle of Abundance in this case. An explicit feedback mechanism such as kudos or karma points can be an element of the reputation model. Experience with these feedback mechanisms has shown that these work best when they are limited to a certain number per person per time period. In this case, a scarcity model helps create value around the explicit feedback.
The reputation model should use the same classification model as the skills element of the People Profile. The difference between this element and the reputation model is the skills element of the People Profile describes my areas of expertise. My reputation is my history of contribution, or what I have done, with the skills I have.
Badging
There are many excellent examples of badging; the belts of Six Sigma being one. Each belt level has criteria that cover the skills and abilities needed to reach that level, along with a certification. This is a way to identify that someone has achieved a certain level of knowledge. This model supports the Principle of Abundance since there is no limit to the number of people who can achieve any given level of knowledge. As you move to more advanced belts, the criteria becomes harder and more demanding, but there is no cap on the number of available Black Belts, for example.
There should be no mystery around how a badge is earned. Additional considerations to keep in mind are that badges need to:
- Operate on a principle of abundance (not scarcity)
- Based on specific criteria (not competition)
- Anyone can earn any badge; there is no limit on how many people can earn any given badge
- Promote the desired behaviors
- Learning, collaborating, sharing and improving
- Collaboration (not competition)
- Reinforcing our brand promise
- Provide recognition for accomplishments (not ranking)
- Account for decay in skill and abilities if not used over time
Badges are a way to reflect the skills and abilities someone has obtained; when designing them, we should leverage the same skills and competencies we defined when creating People Profiles.
- Decide and be clear on what skills and abilities are needed for the organization to be successful
- Transferable Skills (broad)
- Subject Matter Skills (deep)
- Define the criteria for achieving any specific badge
- Document the criteria
- Define how many levels or if it is a singular badge (Bronze, Silver, Gold as an example)
Stack Overflow: An Example
Stack Overflow is an impressive example of badging and reputation models, and is one Consortium Members reference often.
From Wikipedia:
"[Stack Overflow] serves as a platform for users to ask and answer questions, and, through membership and active participation, to vote questions and answers up or down similar to Reddit and edit questions and answers in a fashion similar to a wiki. Users of Stack Overflow can earn reputation points and "badges"; for example, a person is awarded 10 reputation points for receiving an "up" vote on a question or an answer to a question, and can receive badges for their valued contributions, which represents a gamification of the traditional Q&A website. Users unlock new privileges with an increase in reputation like the ability to vote, comment, and even edit other people's posts."
Stack Overflow's model resonates so well with Intelligent Swarming because it recognizes people for their skills, ability, and history of contribution based on Stack Overflow's "three most important activities...Asking, Answering and Editing." In the description of how Badges and Reputation are determined, it is very clear how each are calculated and assigned. We highly recommend spending some time looking at these examples:
Considerations
We look forward to learning more about the way we can leverage Reputation Models and Badging in support organizations. Two interesting lessons learned in this area:
- Recognition is more powerful than rewards. Attaching tangible rewards (money) to the reputation model does not change people's behavior in a desirable way. Rewards are not a motivator in cognitive or intellectual work and in fact, tangible rewards can diminish the impact of recognition (video: summary of Drive by Daniel Pink).
- Do not create leaderboards as they promote competition, not collaboration. We should create recognition boards that recognize people's mastery in critical skill areas (there should be lots of skill areas defined). Recognition is criteria-based, not ranking-based. It is not about recognizing the one person who is the best at this skill, it is about recognizing all the knowledge workers who meet the criteria for mastery in that skill area.