Monday, November 30, 2009

Widget Analytics in Omniture – Part 1 of 3

Widget metrics are fairly straight forward. Getting the metrics, however, can be a bit more complex, depending on your tools. This is the first part of a three part article that will discuss both widget metrics and how to implement them in Omniture.

What’s a Widget?

Widgets are primarily used to present content or ads. Many widgets do both by providing content with embedded ads. Sometimes widgets are used for direct response to start a conversion process.

Ad widgets can consist entirely of an ad. For example, movie studios create a trailer widget that then spreads to social network sites, blogs, or home pages by individual users. This engagement builds brand affinity and extends the audience reach beyond the initial impression.

National Geographic Photo of the Day Widget
Web based widget

Content widgets can consist of copy, images, links, video, games, tickers, or just about anything else one can think of. Again, these can spread to social network sites, blogs, or home pages. For the content owner, these widgets can be a traffic generator in addition to building brand awareness.

Individuals can “grab” the widget and place it on sites or devices over which they have control. They can share it with friends who can share it with their friends. As you can see, widgets have some commonality with magazines in that there is a “pass along” effect that can greatly magnify your campaign. (These days we call it “viral” rather than “pass along”.) For companies, this is probably the most important value-add for widgets. The distribution by individual users is an implicit endorsement of the widget and, by extension, your product, site, and company (brand building) and those users have the potential to reach people in a way that your marketing budget never could.

Yahoo Clock Widget
Yahoo desktop widget

In addition to building audience, brand, and driving additional traffic, widgets have also been used to start a conversion process. One can sign up for a newsletter, more information, or even make a direct purchase. An example of a “purchase” is a “donate” button for a not-for-profit organization.

A widget can be placed on a Desktop, a Mobile device, or a Web Site. A website widget lives on a Web page. For example, this can be a stock ticker on a Google home page or on a social site such as Facebook. Mobile widgets live on those types of devices as little applications. Think iPhone apps. Desktop widgets live on an individual’s computer desktop. For example, I have the Yahoo Clock set to the time in Hamamatsu Japan because my daughter is currently living there. It sits next to all my other desktop icons.

The technology that powers a widget can vary. Some are html based, some are built in rich applications such as flash, and some can be full-blown compiled application code. Anything that can be made available on the Web can probably be made into a widget.

Widget Metrics

Generally widgets are intended to support branding, reach, and acquisition. So widget metrics are for the most part campaign metrics, not much different than those used for emails, banner ads, or your direct mail promotion.

Once you know what your widget is supposed to do for you, there are three types of metrics that you will want to look at. One type speaks to the contribution to the business. The second set is concerned with whether users are interested in your widget offering. The third set asks how usable the widget is (see my earlier article 5 Types of Success).

Broadly, here is what you will want to know:

  • Branding
  • How many people saw the widget and where? What host sites or type of sites work best?
  • Reach
  • How many domains have placed the widget and how many people have seen or used it? What host sites or type of sites work best?
  • Acquisition
  • How many people interacted with it, how often, click-through rates, what is the post-click behavior, etc? What host sites or type of sites work best?
  • Usability
  • What is the best configuration to produce the best Reach and Acquisition response?

Here are some metrics that can be obtained to inform these questions:

Metric Business Use
1. Reach by widget (count of domains)This counts the number of different domains a widget lives on and measures the level of viral uptake. You can plot this over time to determine a velocity (rate) of distribution. The greater the velocity, the more interest your widget is generating.
2. Which domains have placed the widget? Used to know which sites have placed the widget. This will help inform your branding impact. Are you reaching the right audience?
3. Widget placements per domainUsed to know the level of penetration in social media sites such as Facebook.
4. Which domains have distributed the most widgets?Used to know which sites are most involved in spreading the widget. The vendor Clearspring calls these distributors “hubs”.
5. Specific places the widget was placed.Used to know which specific pages or devices are hosting the widget. This is useful to understand the context in which the widget is being presented and informs both your Branding evaluation and how that location may be affecting the level of interaction with the widget. You may not be able to affect the relevance of the placement, but it looking at that will help understand what is and is not working.
6. Acquisition by site (UU per domain where user clicked or interacted with the widget)This measures the ability of a given site to engage. This is both a reach and acquisition measure. Reach because it is based on UU and acquisition because it only counts users who have initiated an interaction.
7. Repeat use for a Widget (user must have clicked or interacted) across all domainsUsed to gauge widget interest and engagement over time.
8. Impressions for a widget across all domainsUsed to understand branding exposure and to calculate widget CTR.
9. Impressions for a widget per domainUsed to understand which domains provide a greater Branding exposure and to calculate widget CTR per domain.
10. Unique user presentations for a widget across all domainsUsed to understand the branding reach.
11. Unique user presentations for a widget per domainUsed to understand which domains provide a greater branding reach.
12. Count of clicks (or whatever you deem an interaction) to a widget across all domainsThis counts every interaction with the widget to build an engagement measure. It is used to gauge widget interest and engagement and to calculate widget CTR.
13. Count of clicks to a widget per domainsThis is the same measure as above, but by domain. It is used to know which domains perform better for a given widget.
14. Count of clicks to a given link in a widget.Used to gauge link effectiveness. As you modify the widget configuration, this will help you understand the usability of the widget.
15. CTR of a widget across all domainsMeasures ability of the widget to generate engagement. This is a typical campaign metric that is also used to gauge the level of usability.
16. CTR of a widget per domainMeasures the ability of a given domain to generate engagement.
17. Site activity driven by a given widgetMeasures the ability of the campaign to drive monetized results. For media sites this might be Page Consumption. For commerce sites it might be conversions such as Purchases or Order Value.

If you find some domains are particularly productive, you might contact them to see if there is any way to enhance the relationship between you and the other company. Often times you will have no ability to influence placement beyond creating an interesting or useful widget. Even so, by understanding where your widget is placed and the level of engagement in those places, you will better understand your Branding context and the fit of the audience you are reaching.

In part two of this article, Wiget Anlytics in Omniture, part 2 of 3, I will go through how Omniture can be configured to provide these metrics.

Labels: , , , , ,

Wednesday, September 16, 2009

About PVC: A Key Metric for Media Sites

This article provides an overview of a key conversion metric for a media sites: PVC. It provides a measure of the pages most involved in driving page consumption, whether that is overall consumption or the consumption of specific high value pages. If you are a CMO or report to one, you will care about PVC.

  • Here is what we will cover:
  • What is PVC?
  • How is it related to other metrics?
  • How is it used?
  • Extending the metric.

What is PVC?

PVC stands for Page Velocity Consumption.

Page indicates that this is a page based measure. Velocity is a term taken from Economics and is a measure of how much money is circulating in the monetary system. In this case it’s a user, not money, which is circulating1. Consumption is about how much that user is circulating. As we will see, for media sites the metric is correlated to revenue.

When a page, page set, or link is seen or clicked by a user we begin counting all subsequent pages in the visit. It is the average count of pages consumed in a visit after touching a particular asset (page, page set, or link). So the PVC calculation is “pages seen after / visits”.

A simple PVC example:

A simple PVC example

In this case the PVC for page 1 is 4. Note that we count from the first “view” until the end of the visit. Even though Page 1 is seen twice, we only count from the first view. The PVC for the second page is 3.

Here is an example of counting from a link click:

A link click PVC example

In this case, the PVC for the link on page 1 is 3. We don’t count the page the link was on because that view was before the click action to be tracked.

PVC can be thought of in a number of ways

For many media sites driving additional PV is the purpose of every page or application. Over many visits, PVC is a measure of the likelihood to drive additional page views. PVC can be a measure of “Persuasion Architecture”2 (from Brian and Jeffry Eisenberg) when applied to media sites.

PVC measures the downstream impact of changes made to the site. On most media sites the paths people take through the site are just too numerous to be useful. However, PVC lets you see the impact of changes to a given page or page set over many paths. If the goal is to drive additional page views, does your new page lead to a more or less involved experience?

PVC is an engagement metric. It is different than regular Page Consumption (PV/Visit) in that it measures consumption over the whole visit, not just for the page or within the page set.

PVC is a conversion metric for media sites that sell ad or sponsor impressions. It is thus correlated to Ad revenue or Sponsor revenue. It is a conversion metric similar to Average Order Value (AOV) on commerce sites. (We will discuss Sponsor PVC later in this article).

Because it reflects revenue, PVC is used to measure business optimization. That makes it a key metric concerned with the economic success of the company and its strategic goals. Any CMO should care about that!

How is PVC related to other metrics?

  • PVC is a metric used to understand the ability to convert. Examples of other standard media metrics are:
  • Repeat use (Visits/UU).
  • Page Set Contribution. (PV/Site Visits).
  • Page Consumption (PV/Visit).
  • Click Through Rate (Clicks/Impressions).

These metrics and PVC are used together to understand site activity and contribution to the site. Sometimes these metrics are used to understand usability, sometimes product viability, sometimes campaign success. Usually PVC is used to understand business optimization.

  • Like the other metrics in the list, PVC is also available to track site performance at various levels of granularity. This makes it excellent for inclusion in monitoring reports. Levels of tracking can be:
  • Site
  • Page Set (Page Type, Tool, Product, Subject, etc.)
  • Page
  • Link
  • Tracking PVC at various levels allows you to monitor the larger trends and then drill down to identify what component of the site is driving that trend.

PVC is most closely related to the Page Consumption metric (PV/Visit). In fact, at the site level they are the same number. PVC can be used as a segmentation of site wide Page Consumption.
The difference between PVC and Page Consumption for a given asset measures the ability to drive additional page views (PVC - Page Consumption) for that page or page set.

PVC, like most of the other metrics in this list, is based on a visit. This provides some consistency between these metrics. However, if your technology supports it, PVC can also be based on visitors and longer time periods such as pages per month or per quarter.

How is it used?

Page Velocity Consumption can be used to identify trends in consumption over time. For a given asset (page set, page, link), is the trend showing improvement? By segmenting at various levels of the site, you can identify what components are driving that change. For example, look at individual pages for a given page set or at individual links on a given page.

It can provide a measure of the overall business outcome for changes you intentionally made to an asset. For example, the CTR of a page may have doubled, but the overall engagement may have gone down. In this case, you may have lost money overall. Compare the PVC values before and after the change to determine the level of impact.

PVC can provide a measure of relative value for different assets. Do some assets provide a better monetized visit than others? Can a high performing asset be better utilized by driving more traffic to it? For example, if you track tools or applications, compare the PVC values for the various tools. If you track content by subject, compare the PVC value for the different subjects. You can go further and compare the values of the pages within a given subject.

Here is an example of a PVC report in Omniture (the real values have been changed):

An example of a PVC report in Omniture

Extending the metric

A variant of PVC is Sponsored PVC. Some media sites allow the sponsorship of specific pages. In this case, the Sponsor PVC metric is constructed by counting only the sponsor pages seen after touching a given asset. It is analyzed in the same manner as PVC. One can then go further and look at which sponsorships the asset drove to.

Ads and sponsored pages are not likely to all have the same value. So the monetized value of pages can be different. PVC can be extended by incrementing the value of the page instead of simply incrementing the count of the page view. Pass the summed value of the ads on the page or its sponsorship value. This will provide a measure of the monetary contribution of the asset to the business. As Avanash Kaushik might say, “How cool is that?”

The PVC concept can be used to track value by referring sources or campaigns. In this case, you are tracking the PVC value from a source rather than an asset on the site. (The number is the same as if you calculated Page Consumption where the entry source was X). The source could be a search engine, direct traffic, newsletters, etc. Simply pass the value for the referrer on the landing page and start counting. You could then answer questions like which keywords are driving the most consumption and should I buy more?

1This explanation of “velocity” was provided by the Jared Cook at Omniture.
2Brian Eisneberg & Jeffry Einenberg, Call to Action: Secret Formulas to Improve Online Success (Wizard Academy Press, 2005).

Labels: , , , , , ,

Wednesday, May 13, 2009

5 Types of Success

Not all success is the same. In deciding on the success measures for a project, one must be aware of what type of success is being considered. There can be a success by one measure while at the same time producing an absolute failure in another dimension.

One can have a product of significant interest to users but has poor usability. One can have a product that generates a great deal of activity that contributes little value to the business. Your application’s interface could be perfect where no user ever makes an error. However, the application only calculates yesterday's date and no one will ever care.

There are generally 5 purposes for which various metrics are used. These are concerned with different types of success:

  • Business optimization
  • Feature optimization
  • Campaign optimization
  • Process optimization (usability)
  • Quality assurance

The different types of success measures answer different kinds of questions and the same metrics are often used to measure the different types of success. (Unique Users, Page Consumption, Repeat Use, etc.) Those metrics will have a different meaning depending on which success question is asked. This can lead to confusion both for you and others in your organization so care must be taken to carefully identify what type of success you are discussing.

The type of success measure also tends to map to the responsibility of the “metric consumer”. A Producer responsible for links might tend to look at process and campaign metrics while a product Vice President might tend to look more at business and feature metrics.

  • Business optimization
  • Business Optimization is concerned with the economic success of the company and the metrics are highly correlated to business revenue or strategic goals. In other words, it’s about the money. If you have a media site, this may be ad revenue or Page Velocity. If you have a commerce site, it’s about sales. This type of success measure gets reported to senior management.

  • Feature optimization
  • Feature optimization is concerned with the success of a specific product or application. Metrics are focused on customer interest in the product and its parts. Is the product something your customers want? Is the feature set complete or needlessly complex? A Product Manager will follow these metrics closely.

  • Campaign optimization
  • Campaign optimization is concerned with maximizing the effectiveness of a discrete effort over some period of time. These are the marketing measures used to track reach and acquisition. SEO fits into this category. This success is, of course, what your Marketing Department lives for.

  • Process optimization (usability)
  • Process optimization focuses on user interaction with the product. The metrics are used to improve the ability to use the tool and the effectiveness of the user interface. Can your users operate the tool? Your usability engineers and designers may tend to use this one more than others.

  • Quality assurance
  • Used to monitor the operational conditions of the site. For example, was there a sudden drop in traffic due to server failure or an increase in 404 errors? This type of metric is usually of concern to Operations and Development managers.

When you plan a project and define how you will measure the success of that project keep in mind what type of success you are considering. Often you will be looking at more than one measure of success. Often you will get different outcomes for the different types of success. You may have to decide as a business to prioritize them.

Labels: , , ,