Home > Measurement Matters

Measurement Matters

Abstract

Successful adoption of the Knowledge-Centered Support (KCSsm) methodology offers profound benefits for any information-intensive organization. For most groups, KCS is transformational in that it changes and increases the value proposition of support. The benefits realized in the short term can be tracked using traditional support metrics. The longer-term benefits are in new areas of value creation and, therefore, require new measures.

 

Extending ideas in the KCS Practices Guide, such as the Strategic Framework, this paper helps KCS leaders:

  • Distinguish the four phases of a KCS adoption
  • Recognize the key organizational trends, relevant milestones, and exit criteria that denote progress through these phases, and
  • Articulate the different internal and external measurements appropriate for each phase

Equipped with these concepts and the measurements that explain the support organization’s impact on the broader business, KCS leaders can educate executives outside the support organization and more easily sustain the financial and organizational commitment required for to maximize the benefits.

Introduction

The Consortium members recognized early on that “KCS is a journey, not a destination.” Built into the KCS model is a process for continuous learning and improvement. As in any journey, there are milestones along the way: intermediate goals. And, as with any journey, there are periods of travel and periods of rest. However, if we rest too long or in the wrong place, the initiative loses momentum and the benefits are not sustained.

GOOD NEWS AND BAD NEWS: KCS THRIVES ON CHANGE. THIS IS GOOD NEWS, SINCE MOST OF US WORK IN ENVIRONMENTS WHERE THE RATE OF CHANGE IS ACCELERATING. THE BAD NEWS IS THAT, IF THE KCS EVOLUTION STALLS, WHAT HAPPENS? SUPPORT ANALYSTS WILL NOT MAINTAIN ENGAGEMENT

This paper supports the successful management of KCS programs by representing key milestones in context as phases of the KCS journey. For each phase, we describe the relevant benefits and measures and how these factors change as we mature from adoption through proficiency to leverage of the knowledge base. We specifically note the telltale conditions, the exit criteria, which help leaders know they are entering the next phase.

 

Understanding of the phases is critical, because the level and type of benefits vary through each phase. Sensitivity to these phases optimizes and sustains the benefits of KCS, appropriately shifting the focus to measuring new and different indicators at each phase.

 

A challenge for KCS leaders is managing expectations of KCS ROI as the KCS program matures with in the organization. The near term benefit seen at the end of Phase 3 (increase in support capacity) is usually more than enough to justify the adoption program cost. The longer-term benefits realized in Phase 4 (self-service success and product improvements) may or may not be appropriate to talk about at the outset.

 

The timing of that discussion depends on the perspective of executives and how willing they are to see an expanded role and contribution from support. The longer-term benefits are in areas that are not usually viewed as being in scope for support and so are often not measured today. The pragmatic “show me the money” types may struggle to relate to the broader value proposition.

 

To lay the groundwork for this new relationship-based valuation, Phase 1 describes baseline measurements to capture, while the Phase 4 discussion offers ideas for quantifying contributions outside the traditional support parameters that typically focus only on operational efficiency.

 

Using a phase-based discussion framework for the focus, measures, and benefits allows KCS leaders to describe the different competencies, efficiencies, and contributions in a way that is relevant to the organization.

 

A note on process: We offer specific numbers for many of the key measures. This precision provokes discussion and helps make the dynamics at play more obvious. Please recognize that the results will vary based on the nature of the customers and products supported.

The Four Phases

Although we describe KCS adoption as a journey, it may be more accurate to think of it as a continuous process of learning and improvement. As KCS is adopted, an organization enjoys an evolution of its people and their skills, its processes and the supporting technology, and its relationships with its customers and other internal organizations. The maturing knowledge base, talent base, and self-service support model contribute new organizational value.

 

Four distinct phases mark progress and must be measured with relevant measurements:
 

Phase Focus Sample Organizational Measurements

1: Planning and design

  • Build tools required for successful adoption
  • Gather baseline measurements
  • Set realistic internal and external expectations
  • Executive sponsor buy in
  • First draft of project deliverables

2: Adoption

  • Create internal understanding and excitement through initial competency
  • Establish internal referenceability
  • Ratio of known to new incidents
  • Participation rate
  • Article quality index
  • Competency profile

3: Proficiency

  • Create and mature the knowledge base
  • Increase process efficiency
  • Reduce time to proficiency
  • Improve collaboration and analyst satisfaction
  • Cost per incident
  • Resolution capacity
  • Percentage first contact resolution
  • Time to proficiency for new employees and new technologies
  • Time to publish

4: Leverage of the knowledge base

  • Optimize resource utilization
  • Reduce support cost
  • Increase customer success
  • Improve employee satisfaction
  • Improve products and services
  • Support cost as a percentage of revenue
  • Customer loyalty (renewal rate; new product/upgrade adoption rate)
  • Customer satisfaction (speed to resolution; first call resolution)
  • Employee satisfaction (survey)
  • Employee turnover rate
  • Self-service use (call deflection or issues resolved without assistance
  • Ratio of known to new incidents being handled by the support organization
  • Product Improvements (Number of Requests for Enhancements (RFEs) accepted by product development)
  • Cultural health
  • Time to adopt new/upgraded products

 

Measurements Matter

We use phase-appropriate measurements to assess when we are ready to move to the next phase. The right measurements for each phase will provide honest feedback and promote the correct behaviors. Getting measurements wrong means a premature advance to the next phase, and the result may be failure for the KCS initiative.

 

Take the example of customer self-service. While KCS techniques improve the efficiency and quality of assisted support, most teams are also using it to enable a shift from the assisted model to a self-service model. By capturing knowledge and making it available broadly, KCS helps support analysts shorten resolution times and improve capacity in Phase 2 and Phase 3. The measures here are familiar: average work time to resolve, cost per incident, and number of incidents handled per analyst per month. Trends in these measures accurately reflect improvement.

 

The picture changes dramatically in Phase 4, however, as customers gain access to the well-developed knowledge base. Customers help themselves to information earlier in the exception process. Many of their questions and concerns are answered quickly, on demand, without support center assistance or the need to escalate.

 

When escalations to assisted support are necessary, resolution of these unique and complex incidents may require significant time and multiple resources. This successful self-service model changes the nature of the work in the assisted model:

  • The easy and known issues are now being resolved through self-service.
  • The new and complex issues are the ones entering the assisted support process.

 

 

[PHASES OF ADOPTION GRAPHIC]

 

 

Unfortunately, the traditional metrics—average work time to resolve, cost per incident, and number of incidents handled per analyst per month—start to look terrible. From the great results in Phase 3, as compared to initial baselines, the numbers go the wrong way.

 

But the customer experience is vastly improved. What has gone wrong? Nothing. Phase 4 simply needs different measurements—web self-service results, product improvements based on the patterns in the knowledge base, and the impact on customer and employee loyalty—not just the transaction speed assessed in the traditional assisted model. Support is transformed from a transaction-based model to a highly leveraged relationship-based model. We do not measure the same things we once measured. A clear understanding and interpretation of measurements is key to successful implementation of KCS initiatives.

Positive Trends, not Absolutes

KCS phases follow the same sequence for each adoption, but each organization has a different experience. The length of time each phase occupies depends on the culture, the underlying technology, and the complexity and nature of the business.

 

At any single point in time, different groups within a single organization may also be at different phases. This variation is normal, but challenging for leaders who need to capture and communicate each team’s progress and contribution. Consistent measurements support an informed assessment and provide a meaningful way to communicate progress and value.

 

As we look at applying measurements, we must keep in mind that absolute numbers are not critical. Positive changes in the trends are critical. As with the timeline for each phase, the actual results and benchmarks vary by industry and the complexity of support issues. We must establish measurements relevant to our businesses and technologies and then use them to confirm that trends are moving in the right direction. Depending on the business, even a modest change in a particular measurement can represent significant ROI.

Phase 1: Design and Planning

Phase 1 provides time to develop the roadmap and foundation of our KCS program. The foundation has seven key elements: Strategic Framework, Content Standard, Workflow Map, Communication Plan, Performance Assessment model, Technology Plan, and the Adoption Roadmap. Attending a KCS Foundations Workshop and using the KCS Practices Guide can help the KCS adoption team create these building blocks for success. In terms of measures and benefits, the Strategic Framework is the primary document.

Strategic Framework

The Strategic Framework unifies the planning and design work of Phase 1. It documents the higher-level goals for the organization with respect to the key stakeholders: the customers, employees, and business. Understanding how KCS aligns with and supports these goals assures the capture of relevant baseline measures for benchmarking and for communication outside the support organization. Generally, there is one Strategic Framework for the entire organization.

 

Of customer, employee, and business goals, the customer goals are especially important. Alignment with customer goals, starting with more efficient support transactions and evolving to improvements in the customer experience, articulates a business value that is easy to explain to people outside support. Well-defined benefits to the customer help sustain commitment to the KCS program across executive level management changes and communicate value to new organizational players, inside or outside the support organization.

 

Research may be required to set meaningful baselines and to expose the real customer demand for support (which is much greater than the incident volume--see the Demand Based View of Support paper from the Consortium website) and more customer choices or channels for resolution.

 

The Strategic Framework captures a long-term view of the organization’s goals. However, since it is anchored in assumptions about the business, the market, and the technology environment, the framework should undergo review and renewal at least annually. Perhaps little will change; perhaps we will re-prioritize. The evaluation process is constructive and can be an extremely valuable source of content for our ongoing communications.

 

From these higher-level objectives, we derive internal group and individual performance measures. Clear alignment among these tiered goals helps every team member make sense of their own contributions to group and business success. Internal goal and performance management is an important step in KCS planning, but is not the emphasis of this paper. For details, refer to the Practices Guide sections covering Balanced Scorecards, Performance Assessment, and the Strategic Framework.

 

Baseline Operational Benchmarks

Phase 1 benchmarks should include quantitative operational measures as well as a qualitative cultural baseline. The baseline measures document the pre-KCS state of the organization and should reflect the generic balanced scorecard quadrants:

 

Goal Type Measurement
Customer
  • Customer loyalty
  • Customer satisfaction (Speed of resolution)
  • Time to adopt new/upgraded products
Employee
  • Employee satisfaction / loyalty
  • Employee turnover rate
  • Time to proficiency for new employees and new technologies
Financial
  • Support cost as a percentage of total revenue (or products shipped, licenses sold)
  • Cost per incident (or exception) (cost/incident)
  • Resolution capacity (analyst productivity measured in incidents/month/analyst)
  • Average work time to resolve
  • Product Improvements (Number of RFEs accepted by product development)
Process
  • Incident volume
  • Percentage first contact resolution
  • Competency Profile (% at each level KCSI, KCSII, KCSIII)
  • Participation rate
  • Self-service use (issues resolved without assistance or escalation)
  • Article quality index
  • Ratio of known to new incidents
  • Time to publish
  • Average work time to resolve

 

The list above reflects standard measurements. We select those most relevant to the business, but keep it simple by selecting just 1-3 measurements per category. What is important is capturing a “present state” baseline that allows us to demonstrate progress against the traditional metrics of success, as well as to measure customer and employee loyalty factors that people may not expect KCS to affect.

 

Consider the rich article data captured in the knowledge base. Armed with the knowledge base’s detailed use cases and related incident volumes, the support team can justify product improvements. When these requests are built into new releases, the support team is delivering value to the business. By comparing the Phase 4 Product Improvements number with the Phase 1 Product Improvements baseline, we can document the contribution made by KCS.

Cultural Baseline

The other type of benchmark gauges teamwork attitudes using a qualitative cultural baseline. It aids adoption investments and highlights the significant cultural shift that KCS usually requires for success. We often refer to the work of Patrick Lencioni and his survey in The Five Dysfunctions of a Team, which provides a quick assessment of a group’s level of trust, ability to resolve conflicts, willingness to commit, accountability, and focus on results. Employee surveys are helpful if they document the current state of the culture, employee morale and loyalty to the company.

 

Initially, we measure the organization’s ideas about itself. We use these levels to guide appropriate investments in coaching, mentoring, and communication during Phase 2 and Phase 3, as well as longer-term trends.

 

For example, many organizations suffer from the “programme du jour” syndrome, introducing quality improvement programs without following through or attempting and failing in serial, “knowledge engineering” initiatives. If the organization has a history of tepid, poorly conceived and spottily implemented programs, the attitude of the members is likely to be distrustful and noncommittal toward any fresh quality improvement initiative. If our survey reveals this cynicism, we must invest extra effort to identify what is different about KCS, confirm individual engagement, ensure alignment, and follow through.

 

The Strategic Framework and balanced scorecards can help with alignment. We may also choose to increase the planned frequency of communication and raise the visibility of ongoing executive support, especially in the case of reorganization. To ensure that trust and collaboration are improving, members retake the survey after three months.

 

If the program appears to be stalling, a cultural survey may reveal attitude challenges we must tackle. We might find surprises that could lead us to rethink assumptions or try a new approach.

 

By Phase 4, through the knowledge sharing and collaboration of KCS, we should be able to chart material improvements in teamwork. As in the case of operational measurements, the initial cultural baseline enables these comparisons.

Assessing the Investment Required

Most organizations expect costs to be associated with a KCS program, but many miss some critical investment areas that affect success. Phase 1 provides the opportunity to request the support we really need when the budgets are most readily adjusted.

 

Phase Investment Considerations
Phase 1
  • Project and program management
  • KCS Foundations and Design Workshops
Phase 2
  • Project and program management
  • KCS training for end-users, coaches, and manager
  • Marketing costs for Communication Plan
Phase 3
  • Tools and integration – for KCS and for self-service (web, forums)
  • Coaching time (usually an opportunity cost for existing team members)
  • Marketing costs to support the Communication Plan
Phase 4
  • Tools and integration for self-service (web portals, forums)
  • Marketing costs for promoting self-service to customers)

 

A general rule of thumb (circa 2007) is that we will spend $1500-$2500 per person on technology and a similar amount on the people and processes of the program itself. This is a one time, first year expense. Planned expenses may cover some of this budget (annual training allowances or existing project management staff, for example); some cost may be incremental. The good news is that operational headcount expenses may not need to change. Rather than hiring additional staff, we transform the skills of our existing personnel to increase their efficiency and their contribution in enabling self-service success.

Exit Criteria for Phase 1

We will be ready to move from Phase 1 to Phase 2 when an executive sponsor has bought in and we have completed the planning deliverables. The chart below recaps the deliverables and the readiness criteria. Refer to the Practices Guide and the KCS Foundations workshop for implementation details.

 

Phase 1 Activities Benefits Readiness Evidence
Executive Sponsor Buy In
  • Ensures champion who can launch and maintain visibility externally and provide vision and leadership internally
  • Kickoff meeting for core team with executive
  • Budget approval
  • Team-wide communication from executive about plan and goals
Strategic Framework Development
  • Sets clear expectations internally and externally; show how contributions align and fit into a bigger picture
  • Separate Customer, Employee, and Business Views with related activities and anticipated results
Baseline Measures
  • Allows us to benchmark against ourselves as the KCS program matures
  • 1-3 baseline measures captured for each Customer, Employee and Business View
  • Cultural baseline established at group level
Workflow and content standard
  • Enables consistent article quality and rapid improvements in proficiency
  • Workflow has been simulated and documented
  • Content standard is understandable and easily accessible
Communication Plan
  • Captures key messages for each audience
  • Educates stakeholders and maintain commitment to project
  • Written communication plan with project owner (possibly a marketing owner as well as a team lead)
  • Review and signoff of plan by executive sponsor
  • Process for testing communications effectiveness
  • Process for feedback and improvement
Performance assessment and reports
  • Guides individual development
  • Enables understanding of group progress
  • Draft radar chart developed with baseline competency profiles for team members
Technology Plan
  • Helps us understand the ability of our existing technology to support new KCS processes
  • Minimizes technology investment
  • Technology assessment complete
  • Technology accurately implements workflow and content standard
Adoption roadmap and training plan
  • Helps set expectations about time and cost commitments
  • Simplifies project management
  • KCS core team and first group of adopters identified
  • Training scheduled for KCS participants, managers, and coaches

 

Phase 2: Adoption

Phase 2 establishes a team-wide understanding of KCS concepts, including the dynamic collective experience that is captured in the knowledge base, while it builds KCS skills. Activities include:

  • Training of the team members, their managers, and the coaches
  • Validation of the foundation elements built in Phase 1
  • Evolution of the technology to better support the practices
  • KCS competency development to the point that the KCS practices become second nature for the support analysts.

 

The desired outcomes of this phase include internal referenceability and participant enthusiasm about using the KCS Practices.

 

As we work through Phase 2, our focus is on individual and team proficiency measurements rather than organizational measures. The key indicators below reveal how well the team understands and embraces the KCS A Loop practices. Some of these baselines may have been captured in Phase 1, but we will need to gather any additional activity measurements specific to this phase. We will sample them to test our increasing proficiency during the phase.

Exit Criteria for Phase 2

If we achieve certain thresholds of maturity in the adoption phase, we are ready to move on to Phase 3, the proficiency phase of KCS. These levels attest to individuals’ competence, but also validate that underlying use of the KCS fundamentals is sound, and knowledge base quality is improving.

 

If we have a useful content standard, then people will use it to build quality articles. If the content in the knowledge base is findable and usable, analysts will search for, reuse, modify, and cite articles. If the collection of content is big enough, Knowledge Domain Experts will organically emerge.

 

Phase 2 Measurements Target for Exit to Phase 3
Competency profile

80-90% of analysts are KCS Contributors or KCS Publishers

Ratio of known to new incidents

Reuse rate is equal to or greater than creation rate

Participation

Participation rates level off between 65 and 85% (this rough range represents enough use of the knowledge base to sustain the methodology)

Technology requirements

Technology must support the workflow (through modifications to existing tools or acquisition of new tools)

Article Quality Index

Meeting defined targets

 

Because participants are reusing content successfully, Phase 2 should produce moderate efficiency gains. However, we do not set targets for these goals. Efficiency targets would distract us from the critical adoption measurements, and we tend to get what we measure.

 

Noting positive trends benefits both individuals and the organization as a whole. Improvement in traditional resolution capacity measurements, coupled with the analysts’ growing sense of confidence in supporting a broader range of products, should contribute to the participants’ excitement about their progress and build referenceability.

Phase 3: Proficiency

With positive momentum and confidence from Phase 2, we move to the proficiency phase, building people and process competence and a mature knowledge base. Phase 3 includes mainstream use and evolution of the KCS foundation elements, the beginning of B Loop content development (patterns, clusters and trends), as well as a conscious development of the knowledge base.

 

In Phase 3, we make major efficiency gains, so we can assess organization-level measurements, such as resolution capacity and average work time to resolve. By the end of Phase 3, we should be able to document the kind of improvements in cost per transaction ROI that business managers understand. We should also see improvements in our cultural baselines of collaboration and trust and in employee job satisfaction.

Exit Criteria for Phase 3

Traditional business measures should look great by the end of Phase 3, justifying continuing support for the KCS program. We refer to the Phase 1 baselines to see if we have gained improvements in these KCS group-level measurements:

 

Phase 3 Measurements

Target for Exit to Phase 4

Ratio of known to new incidents

Reuse rates are two to three times creation rates (70-85% of resolutions come from reused knowledge)

Participation

Participation rate trend has stabilized

Percentage first contact resolution

Increase compared to baseline

Time to proficiency for new employees and new technologies

Decrease compared to baseline

Resolution capacity

Increased by at least 25% (measured in improved average work time to resolve or incidents/analyst/month) (Use the benefits simulator on the Consortium website.)

Article Quality Index

80-90% of articles are consistently rated at least “acceptable” according to the Article Quality Index, and the process for evaluation is stable

Cost per Incident

Decreased since Phase 1 baseline (this is dependent on a growth in incident volume or the ability to deploy analysts to other high value projects)

Completion of Phase 3 should mean celebration for the project team and the support organization. The executive sponsor should feature the quantified improvements in internal and external communication.

 

Although self-congratulations are justified, we must make sure that the stakeholders understand the need to maintain forward progress. Phase 4 not only provides enormous operational improvements for the support organization, it is also where we see improvements in customer success and loyalty. The expanded application of the knowledge base provides a crucial shift in the nature of the work. This is what serves as the fundamental motivation for the support analysts to continue doing KCS. It is in Phase 4 that the mix of new and known issues begins to shift from mostly known to mostly new. The support analysts are working on far fewer known issues; much of the redundant work is now being handled through self-service. This not only reinforces the KCS behaviors, but improves employee morale and job satisfaction.

 

If you stop here, the process will atrophy and die. No amount of tee shirts, coffee cups, or certificates can substitute for the inherent motivation of more meaningful work for the support analysts.

 

Phase 4: Leverage of the Knowledge Base

As we enter Phase 4, our mature knowledge base allows us to deliver additional value to the company by leveraging the knowledge into new applications. First, we extend access to our articles beyond the internal support group, exponentially improving web self-service for customers and partners. Second, we share patterns, trends, and insights from the knowledge base with product and service development teams to eliminate root causes and improve customer satisfaction.

Delivering Value to the Customer through Web Self-Service

We emphasize web-based self-service rather than other self-service models, such as automation and integration of help functions into the product, because it is the most mature and measurable. Most of the myths about “why my customers won’t use the web” have been proven wrong. However, there are a few environments (very few) where web-based self-service does not make sense. The organization has to find other creative ways to enable self-service. The goal must be to resolve known issues outside of the assisted model.

 

With web self-service, many customers resolve their own problems, and they answer questions they would not have bothered to call about. Overall transaction volume on the web are likely to be at least 3 to 10 times higher than the incident volume in the baseline. This increase is indicative of the pent-up demand for support information. We are improving customers’ access to information, and, therefore, their success with our products, at nearly zero incremental cost.

Delivering Value to the Customer through Product Improvement

CUSTOMER SATISFACTION ON THE WEB

THERE ARE FIVE CRITICAL ELEMENTS TO WEB SUCCESS:

  1. CONTEXT—FINDABILITY AND USABILITY ARE HIGH, BECAUSE WE ARE CREATING STRUCTURED CONTENT IN THE CONTEXT OF THE CUSTOMER
  2. COMPLETENESS—MOST OF WHAT WE COLLECTIVELY KNOW IN SUPPORT IS ACCESSIBLE TO THE CUSTOMERS
  3. TIMELINESS—WHAT WE KNOW IS ACCESSIBLE TO CUSTOMERS WITHIN MINUTES OF WHEN WE KNOW IT
  4. PORTAL DESIGN—WE PROVIDE CHOICES FOR CONTENT ACCESS, INCORPORATE QUALITY SEARCH, AND ENSURE ZERO DEAD ENDS (INTEGRATION OF SUPPORT CHANNELS)
  5. MARKETING—THE “BUILD IT AND THEY WILL COME” MODEL DOES NOT WORK; WE HAVE TO HAVE A WAY TO ENCOURAGE CUSTOMERS TO USE THE WEB. 

WHILE KCS ADDRESSES ITEMS 1 THROUGH 3, SUCCESSFUL SELF-SERVICE DEPENDS ON ALL FIVE.

External to support but internal to the company, we can also channel insight from customers into the product management and development groups. Patterns, trends, and clusters of issues in the knowledge base all identify opportunities to improve the products in the areas that are most disruptive to the customer. This data powers new ways of forecasting and correcting product performance. Access to this data provides opportunity to develop new cross-functional customer-based measures. Our broader impact on key business measurements enables organizational learning at the corporate level: a primary benefit of the KCS B Loop processes.

The End of Tradition

With these two initiatives, we see striking changes in the positive value created by support, so it is time to make major changes in our measures. Phase 4 marks an inflection point in the evolution of the support organization and its relationship to the customer and product development.

 

The traditional metrics that were useful in assessing the benefits in Phase 2 and Phase 3 are now all going in the wrong direction:

  • Average work time to resolve has increased
  • First contact resolution rate is down
  • Cost/incident is up

 

Yikes! What have we done?

We have done well.

 

When improvements to self-service make these “old” measures start heading in what we would traditionally think is the wrong direction, it is a sign that self-service is successful. And it is an indication that these measures are no longer useful measures of the health and productivity of the support organization.

 

The value that support creates cannot be measured inside the support organization. Once upon a time, the support transaction (the incident) was everything; it was the primary unit of measure for support. Now, in Phase 4, it is still important, but its value is relatively small in comparison with the value from the use of the knowledge base. We must now characterize new measures for support contribution: customer success with self-service, and the organizational learning that takes place because of capturing the collective experience of the support organization in a reusable way.

 

For these concepts, we need a unit of measure that is bigger than support and, therefore, cross-functional. This is why we say KCS is transformational; it truly transforms the value proposition of the support organization.

New Self-Service Measurements

This change can be tough to internalize. Analysis of some lower level measures can help us understand this change in the area of web self-service:

  • Call deflection—the value of solving customer issues on the web for which they would otherwise have opened an incident. (This represents a small subset of the total customer success on the web.)
  • Time to publish—how long it takes new issues to be posted to the web
  • Self service use—percentage of customers who use the web before opening an incident
  • Self service success—percentage of time customers find what they need on the web

CALCULATING CALL DEFLECTION

CUSTOMER ISSUES THAT ONCE RESULTED IN INCIDENTS BUT ARE NOW RESOLVED WITHOUT ANALYST INTERVENTION--SO CALLED "CALL DEFLECTION"--MUST BE PERCEIVED AS A SOURCE OF COST SAVINGS, AND THOSE SAVINGS MUST BE IDENTIFIED AND REPORTED. WHILE EXCELLENT BENCHMARKS MAY BE WELL OVER 50% FOR CALL DEFLECTION, EVEN AN INITIAL ASSUMPTION OF 10% CALL DEFLECTION WILL DEMONSTRATE IMPRESSIVE ROI. SEVERAL METHODS SHOULD BE COMBINED TO DETERMINE THE CALL DEFLECTION RATE:

  1. CAPTURE THE TOTAL NUMBER OF PAGE VIEWS. IDEALLY, THE ARTICLE PAGE WILL HAVE A FEEDBACK MECHANISM ASKING CUSTOMERS IF THE INFORMATION SOLVED THEIR PROBLEM. THIS COUPLED WITH CUSTOMER INTERVIEWS WILL HELP US ARRIVE AT A SUBJECTIVE RATE OF SUCCESS
  2. ANALYTICS CAN IDENTIFY WHAT PERCENT OF USERS MOVE ON TO ANOTHER PAGE, IMPLYING THAT THE FIRST DOCUMENT VIEWED DID NOT SOLVE THE PROBLEM. (THIS IS A SPECIFIC EXAMPLE OF CLICKSTREAM ANALYSIS)
  3. IMPLEMENTING SELF-SERVICE RESULTS IN THE INCIDENT SUBMISSION PROCESS PROVIDES ACCURATE COUNTS OF DE-ESCALATIONS. WHEN A CUSTOMER SUBMITS A WEB-BASED TROUBLE TICKET THE DESCRIPTION IS PLACED INTO A SEARCH ENGINE AND THE RESULTS ARE RETURNED TO THE CUSTOMER. IF THE RESULTS CONTAIN THE ARTICLE, THE CUSTOMER CLOSES THE TICKET. OTHERWISE, THE TICKET CONTINUES IN THE WORKFLOW TO LAUNCH AN INCIDENT.

Customer Interaction Maps help us see and communicate the benefits gained in Phase 4. These maps plot the numbers of customer issues that come through self-service vs. the support center. They show the percentage of successful self-service interactions and the percentage of those that avoid incidents. They also show how interactions flow from self-service to the support center (escalations) and from the center to self-service (de-escalations).

 

Over time, Phase 4 activities should significantly shift the Customer Interaction Map to web-first, web success, de-escalation, and incident avoidance. Additionally, it is valuable to quantify the previously unserved demand: the volume of issues customers are solving on the web that they would not have called about.

 

We do not typically see support organizations with measures like these in the pre-KCS environment. These are new measures for the health and value of web self-service.

The Big Target: Ratio of support costs to revenue

All of these benefits are important, but for executives outside of support, it is usually helpful to boil results down to one factor: money, in the form of cost savings and cost avoidance.

Top Level Indicator of Support's Contribution

Figure 1: We can quantify support value by expressing support costs as a percentage of revenue.

 

If we can show that we are changing the ratio of support costs to revenue, the delta equals profit. For the top line, we may use products shipped or licenses sold. The goal is to normalize support costs against something that represents the top-level results. If we can reduce the ratio of support costs to revenue as well as improve customer and employee loyalty, what’s not to like?

 

This appeal is why the executive buy-in can create a problem: not because it is hard to understand (they love this picture) but because we must temper their excitement and inevitable impatience for results. For most organizations, Phase 4 benefits are 18-30 months from the beginning of the journey.

Success in Phase 4

Our new knowledge base initiatives should contribute to significant organizational benefits by the time we are 3 to 6 months into Phase 4. Again, it is necessary to refer to the baseline measurements captured in Phase 1 to show overall progress.

 

In addition to lower-level measures such as call deflection, we can introduce new measures that reflect the higher-level business value of customer and employee loyalty. Our Strategic Framework and the Communication Plan should have laid the groundwork for organizational understanding of this sea change. If these are new concepts for an organization, the Reichheld book, Loyalty Rules, describes this new relationship-based approach to value.

 

Phase 4 Measurement Indicator of Success

Customer loyalty (renewal rate; new product/upgrade adoption rate)*

Increase compared to baseline

Customer satisfaction

Increase compared to baseline

Employee satisfaction / loyalty

Increase from baseline

Employee turnover rate

Decrease from baseline

Time to publish

90/90 rule—90% of what the support organization learns from resolving issues is on the web within 90 minutes of becoming known.

Self-service use (call deflection or issues resolved without assistance or escalation)

85/85 rule—at least 85% of the time customers are using self-service first and at least 85% of the time they are finding what they need (Customer interaction maps help here.)

Ratio of known to new incidents

30/70 rule—the work in the support organization shifts from 70% known/30% new, to 30% known/70% new. Support analysts spend the majority of their time on new issues

Support cost as a percentage of revenue

Support costs have dropped by 25-50%, and the volume of customer issues resolved is up at least 100% (web success combined with incidents closed)

Incident volume

Number of incidents open declines (this often needs to be normalized to installed base or revenue in order to account for the dynamics of the business)

Product improvement (Number of RFEs accepted by product development) *

Increase from the baseline

Time to adopt new/enhanced products *

Decrease from the baseline

* Denotes cross-functional departmental measurements

 

Many organizations will stay in Phase 4 indefinitely, gradually improving the customer experience in an environment ripe with employee satisfaction. At the same time, most organizations will also start additional KCS projects at Phase 1. The tools and experiences developed in one project may be reusable in new projects, but each kickoff should have the benefit of a fresh look at customer, employee, business, and process goals and factor these goals into a selection of relevant measurements.

Implications of “Mostly New”

One of the key goals of KCS is to solve a problem once and use the article often. As we have mentioned, customer success with self-service and our influence with product development (often a longer term impact) should result in a dramatic change in the ratio of known to new incidents. As the support organization’s work shifts to mostly new, inevitably the traditional tiered support model becomes ineffective. Solving new problems is fundamentally a collaborative process and the old escalation model, moving incidents from level 1 to level 2 and so on, becomes unproductive. The organization moves from an escalation model to a collaboration model; we call this swarming. The measures for swarming are focused on assessing the health and effectiveness of collaboration. This is a new way to look at the organization and is an area of research and innovation currently under way in the Consortium. For more information on this project, please see the Adaptive Organization on the Consortium web site.

Summary

KCS is a journey. If we keep moving, the landscape will keep changing along the way. That change makes for an exciting, engaging journey. By using relevant measures for each phase of the KCS evolution, we can successfully track our progress and document the dynamic range of contributions to the organization. We can demonstrate tangible customer, employee, financial, and process benefits—if we create relevant operational and cultural baselines at the beginning and map those to the organizational goals, or the Strategic Framework.

 

The benefits do not appear equally during each phase. Phase 1 and Phase 2 are investment phases, Phase 3 benefits are big, and Phase 4 benefits are huge. Since KCS adoptions may extend across years, support for KCS can fade if the numbers do not look attractive. This is the reason the phases are so helpful in explaining the KCS adoption process and its evolving benefits. When the stakeholders understand how the benefits evolve and accrue, it is easier to gain and sustain their enthusiasm for and commitment to Knowledge-Centered Support.

 

References

Please take advantage of the Consortium website for detailed information: www.serviceinnovation.org/kcs

·       Practices Guide

·       Case studies

·       Demand Based View of Support (Funnel in the Cloud paper)

·       KCS Benefits Calculator

 

Books referenced in the text:

·       Kaplan, Robert S., and Norton, David P. The Balanced Scorecard: Translating Strategy into Action. Harvard Business School Press, 1996. http://www.balancedscorecard.org/

·       Lencioni, Patrick M. The Five Dysfunctions of a Team: A Leadership Fable. Jossey-Bass, 2002. http://www.tablegroup.com

·       Reichheld, Frederick F. Loyalty Rules. Harvard Business School Press, 2003. http://www.loyaltyrules.com/

·       Spitzer, Dean R. Transforming Performance Measurement: Rethinking the Way We Measure and Drive Organizational Success, AMACOM, 2007. http://www.transformingperformancemeasurement.com/

Acknowledgements

Contributing Members:

Amanda Roberts--Stone Cobra

David Bohl--Dell

David Kay--DB Kay & Associates

Writer--Barbara Kay

Consortium Staff:

Melissa George and Greg Oxton

v5.2 Updates—Kelly Murray

 

Quick Reference: Exit Criteria for Phases

Note: These seemingly exact numbers are offered as an example of the scope and dynamics of the KCS impact. Actual results will vary based on the characteristics of the company culture, products, and customers.

Phase 1: Design and Planning

Phase 1 Activities Benefits Readiness Evidence
Executive Sponsor Buy In
  • Ensures champion who can launch and maintain visibility externally and provide vision and leadership internally
  • Kickoff meeting for core team with executive
  • Budget approval
  • Team-wide communication from executive about plan and goals
Strategic Framework Development
  • Sets clear expectations internally and externally; show how contributions align and fit into a bigger picture
  • Separate Customer, Employee, and Business Views with related activities and anticipated results
Baseline Measures
  • Allows us to benchmark against ourselves as the KCS program matures
  • 1-3 baseline measures captured for each Customer, Employee and Business View
  • Cultural baseline established at group level
Workflow and content standard
  • Enables consistent article quality and rapid improvements in proficiency
  • Workflow has been simulated and documented
  • Content standard is understandable and easily accessible
Communication Plan
  • Captures key messages for each audience
  • Educates stakeholders and maintain commitment to project
  • Written communication plan with project owner (possibly a marketing owner as well as a team lead)
  • Review and signoff of plan by executive sponsor
  • Process for testing communications effectiveness
  • Process for feedback and improvement
Performance assessment and reports
  • Guides individual development
  • Enables understanding of group progress
  • Draft radar chart developed with baseline competency profiles for team members
Technology Plan
  • Helps us understand the ability of our existing technology to support new KCS processes
  • Minimizes technology investment
  • Technology assessment complete
  • Technology accurately implements workflow and content standard
Adoption roadmap and training plan
  • Helps set expectations about time and cost commitments
  • Simplifies project management
  • KCS core team and first group of adopters identified
  • Training scheduled for KCS participants, managers, and coaches

 

Phase 2: Adoption

Phase 2 Measurements Target for Exit to Phase 3
Competency profile

80-90% of analysts are KCS Contributors or KCS Publishers

Ratio of known to new incidents

Reuse rate is equal to or greater than creation rate

Participation

Participation rates level off between 65 and 85% (this rough range represents enough use of the knowledge base to sustain the methodology)

Technology requirements

Technology must support the workflow (through modifications to existing tools or acquisition of new tools)

Article Quality Index

Meeting defined targets

 

Phase 3: Proficiency

Phase 3 Measurements

Target for Exit to Phase 4

Ratio of known to new incidents

Reuse rates are two to three times creation rates (70-85% of resolutions come from reused knowledge)

Participation

Participation rate trend has stabilized

Percentage first contact resolution

Increase compared to baseline

Time to proficiency for new employees and new technologies

Decrease compared to baseline

Resolution capacity

Increased by at least 25% (measured in improved average work time to resolve or incidents/analyst/month) (Use the benefits simulator on the Consortium website.)

Article Quality Index

80-90% of articles are consistently rated at least “acceptable” according to the Article Quality Index, and the process for evaluation is stable

Cost per Incident

Decreased since Phase 1 baseline (this is dependent on a growth in incident volume or the ability to deploy analysts to other high value projects)

 

Phase 4: Leverage of the Knowledge Base

Phase 4 Measurement Indicator of Success

Customer loyalty (renewal rate; new product/upgrade adoption rate)*

Increase compared to baseline

Customer satisfaction

Increase compared to baseline

Employee satisfaction / loyalty

Increase from baseline

Employee turnover rate

Decrease from baseline

Time to publish

90/90 rule—90% of what the support organization learns from resolving issues is on the web within 90 minutes of becoming known.

Self-service use (call deflection or issues resolved without assistance or escalation)

85/85 rule—at least 85% of the time customers are using self-service first and at least 85% of the time they are finding what they need (Customer interaction maps help here.)

Ratio of known to new incidents

30/70 rule—the work in the support organization shifts from 70% known/30% new, to 30% known/70% new. Support analysts spend the majority of their time on new issues

Support cost as a percentage of revenue

Support costs have dropped by 25-50%, and the volume of customer issues resolved is up at least 100% (web success combined with incidents closed)

Incident volume

Number of incidents open declines (this often needs to be normalized to installed base or revenue in order to account for the dynamics of the business)

Product improvement (Number of RFEs accepted by product development) *

Increase from the baseline

Time to adopt new/enhanced products *

Decrease from the baseline

* Denotes cross-functional departmental measurements

 

Glossary of Measurements

Average work time to resolve – number of minutes consumed per incident in developing an answer, fix, bypass or workaround. Determined by dividing the total minutes worked by the number of incidents resolved

Call deflection— the number of customer issues solved through self-service that would have become incidents (this is a subset of self service customer success)

Competency profile—percentage of analysts at each level of the KCS competencies: KCSI, KCSII, KCSIII

Cost per incident— total support costs divided by the number of incidents closed

Cross-functional measures--measures to which multiple functions within the organization contribute. For example, product improvements require support to capture the interactions and recognize trends to give development credible input on high leverage opportunities for product improvement. Development must execute on these opportunities. The measure is shared by support and development. (See Transforming Performance Measurement by Spitzer)

Cultural health—support analysts’ attitude with respect to trust, commitment, conflict resolution, accountability, and focus on results. See Patrick Lencioni, The Five Dysfunctions of a Team

Customer loyalty— the level of emotional connection a customer feels towards the company, a longer-term measure of overall relationship. Indicators include renewal rate, new product/upgrade adoption rate, and referenceability

Customer satisfaction—transaction-based measure of the degree to which we have met the customer expectations. This is a short-term measure of the customer experience with support. Indicators are speed or average work time to resolve, “percentage first contact resolution”, technical knowledge, and politeness of the support analyst

Employee loyalty— the level of emotional connection employees feel towards the company, gauged through surveys or in the cultural baseline

Employee turnover rate—internal and external attrition, the rate at which support analysts are leaving the support organization

Executive sponsor buy-in—the executive champion for the KCS program understands KCS and is vocally committed. This is a qualitative measurement, but may be judged by the willingness of the champion to present the plan for the project to executive management, to host a kickoff with the project team, and to support communication efforts with email and other outreach.

Incident volume—number of incidents, cases, or tickets opened

Participation rate—the number of incidents closed with a article linked or cited, includes both creation and reuse of articles

Percentage first contact resolution—percentage of incidents resolved the support center on the first interaction. Used as a customer satisfaction indicator as well as an employee proficiency or process goal

Product improvements (Number of RFEs accepted by product development)—the rate at which suggestions for product, documentation or service offering improvements are implemented by development, an indicator of influence

Ratio of known to new—new articles created in the knowledge base vs. reuse of existing articles

RFE—Request for product enhancement

Resolution capacity— how many incidents can the support organization handle in a period of time; indicators are incidents/month/analyst or average work time to resolve (work minutes, not elapsed time)

Self service success--the percentage of time customers find what they need by using self-service (most often but not always use of the web)

Self-service use— the percentage of time customers use self-service before they open an incident

Article Quality Index—AQI is based on adherence to criteria defined in the KCS Content Standard (see the KCS Practices Guide)

Support cost as a percentage of total revenue—the ratio of support costs to total company revenue; used to normalize the cost of support in a dynamic environment. Other possible ways to normalize the support costs include against products shipped, licenses sold, customers subscribed (cross functional measure)

Time to adopt new/upgraded products—rate at which customers adopt new releases or products

Time to close – the elapsed time from incident open to incident closed

Time to proficiency for new employees and new technologies—the number of weeks or months required for an analyst to work with a high degree of independence; the learning curve

Time to publish—time from initial issue discovery to the time information is available to customer

Time to resolution—See “average work time to resolve.” Elapsed time from opening of an incident to offering an answer, fix, bypass or workaround 

Viewing 4 of 4 comments: view all
"Time to adopt new/enhanced products" This needs some clarification, perhaps some examples to illustrate the concept. Also, some suggestions on how to measure this
Posted 12:19, 1 Sep 2016
Right before the Phase 1 section, (in the Measurement Matter section), the graphic for Phases of Adoption is missing.
Posted 06:27, 12 Oct 2016
Will this be updated to Knowledge - Centered Service?
Posted 08:22, 18 Apr 2017
Indeed this whole paper will be updated - it's on the list!
Posted 17:06, 6 Jun 2017
Viewing 4 of 4 comments: view all
You must to post a comment.
Last modified
14:53, 21 Dec 2015

Tags

This page has no custom tags.

Classifications

This page has no classifications.