More on measuring

10 February, 2009

I’m in the process of writing plans, speaking to stakeholders and setting my performance targets. Every year this is a difficult process, I’m trying to balance between setting myself challenging targets for the year and setting myself impossible targets for the year.

My primary objective for the year is to implement and grow online communities that provide value to the user and our organisation, these communities will focus on facilitating knowledge sharing and networking .

On the surface this  is a simple enough objective, the tricky part is how to measure the value.

Now we head in the murky waters of perception, how on earth do you define the value of an interaction? Here are a couple of the considerations floating around in my head.

  • If a user interacts with a community are they: A. providing value to the community? and B. obtaining value from the exchange?
  • If a user regularly visits the community but does not participate, do they obtain value?
  • If users come to the community to ask questions what proportion of questions must be answered and what time frame must they be answered in for the community to be valuable?

As you can see none of these questions can be answered unless you are speaking to the user. I am very keen to speak to users to hear stories and anecdotes, but the the first target is to scope and define the community projects, that means investigating needs, speaking with stakeholders and defining objectives, until that work is done it’s hard to define what  success will look like my performance targets.

The commonly held wisdom is that the only way you will measure the success or otherwise of a social project is to have crystal clear objectives. This is something I will have for every project this year but I’ve not managed to define anything yet.

I need some sort of measure of how a community is going that we can track throughout its life. To achieve this I propose a ‘Community Health Check’, this will by no means be an exhaustive measure of success or value obtained from a community but it could provide an insight into how the community is travelling. It should also give a high-level overview that can help to feed further research into the community.

I suggest ten (or so) questions, that are on a rating scale of 1- 10, the questions would be divided into three groups: usability, type of use and value obtained. Here are the first few questions I’ve come up with:


How easy / difficult it was to access the community site?


How easy / difficult was it to find what you were interested in on the community site?


Type of use

How many times have you accessed the community in the last month?


How many questions / discussions have you started in the community in the last month?


How many questions / discussions have you responded to in the community in the last month?



In general terms rate how valuable you believe the community has been to you or your business


How many questions have you found answers to in the community in the last month?


How useful has the information you have found in the community been to your work?


That’s a start, please keep in mind that this is my first pass at this and it will evolve over the coming months.

So now, what do you think? What is missing? Do you think this will this actually be useful?


2 Responses to “More on measuring”

  1. mikestuchbery Says:

    It might sound ungodly primitive, but what about face-to-face, a beer over a chat about how easy the community is to use?

  2. Mick Leyden Says:

    @mickstuchbery – that will be part of the review process but I want something we can easily use to track how people are feeling about it. I guess I want a snapshot of what is happening that can help to feed further research (and chats over drinks).

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: