_(1).svg?revision=1)
Technique 5.4: Content Health Indicators
While there are many checks and balances in the KCS methodology to ensure content health, there are five key elements that contribute to article quality:
- A content standard that defines the organization's requirements for good articles
- Content Standard Checklist, which determines an article's alignment with the content standard
- Process Alignment Review (PAR), which identifies how well the KCS workflow is being followed
- The proficiency model, supported by coaching
- A broad and balanced performance model.
KCS proposes a proficiency model that uses the Content Standard Checklist and the Process Alignment Review (PAR) to assess the knowledge worker's ability to consistently create articles that align to the content standard and follow the KCS workflow. Knowledge workers are recognized for their KCS understanding and capability, thereby earning rights and privileges in the system. This program contributes to the level of quality and consistency of the articles in the knowledge base.
As the organization gets started with KCS, the KCS Coach plays a major role in the quality of the knowledge base content by reviewing the articles created by KCS Candidates who do not yet have the permissions to put articles in a Validated state. The Coach's goal is to support the KCS Candidates in learning to do the Solve Loop, adhering to the content standard, and using the structured problem solving process.
The Content Standard Checklist
The Content Standard Checklist is a coaching tool designed to help knowledge workers understand and remember to align articles with the content standard. The output may also be used to indicate how well a group of knowledge workers are understanding and applying the content standard.
A content standard and the Content Standard Checklist provide important guidance when prompting AI tools to generate articles from unstructured notes.
The Content Standard Checklist should be consistent with the content standard for a "good article," quantifiable to facilitate reporting, and shared with both management and the individual (as part of a conversation about understanding and behavior - not about the number). Because we are addressing technical accuracy with the practice of "reuse is review," the Content Standard Checklist is not a technical review.
As a starting point, apply these checks in the form of yes/no questions:
- Is the article Unique? - not a duplicate article, no other article with same content whose create date preceded this articles created date (this is a critical part of the Content Standard Checklist)
- Is the article Complete? - complete problem/environment/cause/resolution description and types
- Is the Content Clear? - statements are complete thoughts, not sentences (as appropriate)
- Does the Title Relate to Article? - title contains description of main environment, and main issue (cause if available)
- Are the Links Valid? - hyperlinks are persistently available to the intended future audience
- Is the Metadata Correct? - metadata set appropriately: article state, audience, type or other key metadata defined in the content standard
Capture the answers to these questions in a Content Standard Checklist spreadsheet. In the example below, we review a handful of articles that each knowledge worker has linked or created in a specified period of time, and tally the number of "yes" answers in each column. The resulting percentage reflects the frequency with which a knowledge worker (or group of knowledge workers) followed the content standard while interacting with those articles.
While we can use this as an indicator of the general health of the articles in the knowledge base, it is best used as an indicator of the understanding of the KCS content standard. Tangible, quantified information like this improves the quality of feedback we can provide to individual knowledge workers to enhance skills development and drive article health.
Start simple. Here is an example of an Content Standard Checklist spreadsheet focused on the big six items:
Percentages are calculated using (total possible points minus total points times 2) divided by total possible points. In Excel, the formula for Al's Content Standard Checklist % = 1-(2*(L5-K5))/L5
In both example spreadsheets here, the errors carry a weight of 2. This is done to enable the percentages to better reflect differentiation between those doing well and those who need some help. In these examples, anyone with an Content Standard Checklist percentage below 90 should get some attention from a Coach. If they are consistently below 80, they are at risk of losing their KCS role. It is important to monitor trends over time on the Content Standard Checklist percentages for both teams and individuals.
This matrix can be customized to suit an organization's requirements. A consumer product may need more emphasis on usability and formatting compared to a highly technical audience. But: don't over-engineer it! Start simple and evolve it based on experience. Some of the factors in the criteria will be influenced by the knowledge management technology being used in the environment. The content standard sets the criteria for the checklist, and must be tailored to the environment and the tools being used.
As an organization matures in its use and confidence in KCS, it becomes easier to pay attention to more granular or refined content considerations like versioning, global distribution, use of multimedia, and measuring team-based contribution in addition to individual contribution. So over time, we might add some additional areas of focus.
Following is a sample Content Standard Checklist spreadsheet in a mature KCS environment, for example purposes only (click to view full size).
Some key ideas to note in the mature example:
- Compare the number of articles reviewed for each creator. A legitimate sample size is important. Creators Chuck and Irene may have too few articles to be fairly weighed.
- In the first row (# of articles with issues in this area), organizational performance is visible. This is a great area to focus on with groups of KCS Coaches. In this example, many articles are duplicates, incomplete, or unusable. This result could mean knowledge workers need more training on searching and documenting content.
- One contributor, Kim, is a prolific contributor, but also leads in the top three categories of problems. Attention from a KCS Coach is merited.
Many mature organizations develop a weighting system for this more complex criterion as the violations do not all have the same impact. For example, a duplicate article is a more serious error than an article that is too wordy. (See the Progress Software case study for an example in practice.) Again, the criteria and weighting should be done based on the needs of the organization and should be considered only after the organization has had some experience with the Content Standard Checklist process.
Knowledge Sampling for the Content Standard Checklist
To complete a Content Standard Checklist spreadsheet, a group of qualified reviewers (usually the KCS Coaches) participate in regular knowledge sampling of articles from the knowledge base. While the articles are selected randomly (and assigned randomly to reviewers, so coaches are not often evaluating their coachee's articles), it is important to be sure to sample articles from each individual.
Here is a typical process:
- Develop a Content Standard Checklist and criteria
- Evaluate a sample of articles
- Calculate the content standard percentage met and develop summary reports
- Provide regular feedback to the knowledge workers on possible content standard areas of focus with comments from the Coach who did the evaluation
- Provide periodic feedback to leadership
The Content Standard Checklist should be used for learning and growth of the knowledge workers. During rollout and training, the frequency of this monitoring should be weekly. It will take more time due to the high number of KCS Candidates (people learning KCS). Once the organization has matured, the frequency is typically monthly and should not consume more than a few hours of time per month per reviewer. Note that what the organization focuses on around the content standard should change over time. The elements for assessment at the beginning of a KCS adoption will be more basic than those things the organization will focus on two years into the knowledge journey.
Feedback to the Knowledge Worker
Knowledge workers must have visibility to their Content Standard Checklist evaluations so they understand where to self-correct. Again, this is as part of a conversation about understanding and behavior - not about a percentage or number. Content Standard Checklist results are also a key tool for the Coaches as it helps them identify opportunities for learning and growth.
Content Health is for Everyone
The KCS Principle of demand driven and the Core Concept of collective ownership combine to create efficiency. The idea that people feel a sense of responsibility for the quality of the articles they interact with is critical. The cost and delay of the alternative: that someone else owns article quality and that it is someone else's responsibility to review it, is prohibitive. This sense of collective responsibility is reinforced through coaching, the proficiency model, communications from the leaders, the performance model, and the recognition programs. The new hero in the organization is the person who creates value through their contribution to the knowledge base, not the person who knows the most and has the longest line outside their cube.