Cloud Outsourcing Metrics

Template includes a sample contract with a Service Level Agreement

When outsourcing, metrics should be used to measure the service provider's performance and determine whether the service provider is meeting its commitments. When properly chosen and implemented, the SLA metrics:

• Measure the right performance characteristics to ensure that ENTERPRISE is receiving its required level of service and the service provider is achieving an acceptable level of profitability
  
  
• Can be easily collected with an appropriate level of detail but without costly overhead, and
  
  
• Tie all commitments to reasonable, attainable performance levels so that "good" service can be easily differentiated from "bad" service, and giving the service provider a fair opportunity to satisfy ENTERPRISE.

Common sense is the rule of thumb that should be followed when selecting metrics. The selection process is made difficult because of the large number of potential metrics and must consider factors such as an enterprise’s organizational experience with metrics, the type of behaviors to be motivated and cost and effort of collection. Remember that the goal is to ensure a successful and positive working relationship between the service provider and ENTERPRISE.

 

Best practices for outsourcing metrics

Choose metrics that motivate the right behavior

Once you monitor a factor that by itself will modify behavior.  Each side of the relationship will attempt to optimize their actions to meet the performance objectives defined by the metrics. If the wrong metrics are selected, the relationship can go astray quickly. For example, paying programmers by the number of lines of code they produce will certainly lead to an increase in production, but may play havoc with quality and the true quantity of real work accomplished.

To motivate the right behavior, each side must understand the other side, its expectations and its goals, and the factors that are within its control. Realism must prevail. ENTERPRISE has to anticipate that service providers will want to make a profit; service providers have to expect that ENTERPRISE will want to control costs. When choosing metrics, focus on the behavior that you want to motivate. What factors are most important to your organization? Reducing costs and/or defects? Increasing  production or speeding time-to-market? Which factors are you willing to trade for improvements in another area? Pick an initial set of metrics that measure performance to these behaviors.

Often, secondary metrics are needed to provide checks and balances to avoid missteps. Also, consider whether the metrics are truly objective or are subjective enough to leave room for interpretation. Metrics that are based upon a subjective evaluation are open to different interpretations, and will likely lead to disagreement over whether a service provider has met its commitments. For example, state that "all printed invoices will be delivered to the post office within 4 hours after completion" rather than "all printed invoices will be delivered to the post office in a timely manner."

Ensure metrics reflect factors within the outsource service provider's control

Service providers should ensure that the Service Level Agreement (SLA) is two-sided. If the service provider's ability to meet objectives is dependent on an action from enterprise, the enterprise's performance must also be measured. For example, a service provider may be held accountable for the speed and quality of a system enhancement, but the quality is affected by the accuracy of client-developed specifications, and speed/delivery is held up by enterprise's approval cycle.

Choose metrics that are easily collected

If the metrics in the SLA cannot be easily gathered, then they will quickly lose favor, and eventually be ignored completely. No one is going to spend an excessive amount of time to collect metrics manually. Ideally, all metrics will be captured automatically, in the background, with minimal overhead; however, few organizations will have the tools and processes in place to do so. A metric should not require a heavy investment of time and money; instead use metrics that are readily available, compromising where possible. In some cases, it will be necessary to devise alternative metrics if the required data is not easily obtainable.

Limit metrics to those that matter and can easily be interpreted

Avoid choosing an excessive number of metrics, or metrics that produce a voluminous amount of data. At the outset of drafting the SLA, an organization may be tempted to include too many metrics, reasoning that the more measurement points it has, the more control it will have over service provider performance. In practice, this rarely works. Instead choose a select group of metrics that will produce information that can be simply analyzed, digested and used to manage the project. If the metrics generate an inordinate amount of data, the temptation will be to ignore the metrics, or subjectively interpret the results, negating their value in the SLA.

Establish a baseline benchmark

Defining the right metrics is only a first step. To be useful, the metrics must be set to reasonable, attainable performance levels. It may be difficult to select an initial, appropriate setting for a metric, especially when a customer does not have any readily available performance metrics or a historical record of meeting those metrics. Companies with active metrics programs will have the data needed to set a valid baseline benchmark. Others will have to perform an initial assessment to establish that baseline. Metrics should include:

• Volume of work
• Quality of work
• Responsiveness
• Efficiency
  

Cloud computing capacity planning is complex

The cloud computing model reduces the need for capacity planning at an application level. An application can simply request resources from the cloud and obtain them in less than an hour in accordance with dynamic demand. Thus, it is far less important to correctly predict the capacity requirements for an application than it is in traditional data centers, for which as many as six months might be needed to order and install hardware dedicated to the application.

The Cloud Outsourcing HandiGuide is delivered electronically in WORD and/or PDF format.  Included is a detailed sample contract, a sample service level agreement with metrics, two (2) Job Descriptions, and much more.