A couple of weeks ago, I left the following comment on my post about "Do social media and online community metrics really matter?" in response to someone else's comment, asking "How are the metrics actionable?"
A concrete example are the SharePoint forums at http://MSSharePointForums.com that I used to be responsible for. After trying without much success to convince more product people to help answer questions in the forums, it took just one "Answer Rate" report (that showed an abysmal ~20% compared w/ 60+% for SQL Server and Exchange) to convince the right people that we need to put dedicated support engineers to monitor the forums. Within 3 months, the 5 most trafficked SharePoint forums achieved an avg Answer Rate of ~65%. While I didn't have enough time to validate the correlation between the increased Answer Rate and higher Customer Satisfaction scores, one very important metric that we did confirm was that "Cost Per Incident" in the support forums was only ~10% of the CPI for commercial phone support. Now *that* is ROI that any organization would get excited about these days.
Since then, I've received several direct inquiries about whether or not just having the ability to analyze and report relevant metrics would be sufficient to attain self-sustainability in community forums. By self-sustainability, I really mean minimizing the people cost of maintaining successful forums rather than eliminating all costs, which I don't believe is ever possible. Anyway, my answer is, "No, metrics alone aren't enough." -- even with useful reports like the following "Forums Answers" report that our Harvest 2.0 product provides:
To achieve self-sustainability, you need subject matter experts, who are willing to essentially work for free -- at least from your (budget's) perspective -- in your forums. How can you motivate someone to work for free in a world where there is simply no such thing as a free lunch? Well, the only viable way that I'm aware of is to recognize and compensate someone in other ways aside from cold hard cash. At Microsoft, I leveraged the MVP (or Most Valuable Professional) program, which is funded by our Customer Support Services organization and provides MVPs with free stuff (worth US$thousands) and invaluable opportunities to speak at Microsoft hosted conferences and to engage with Microsoft product groups during all phases of the product cycle. I recruited a subset (around 10) of the 130+ SharePoint MVPs worldwide to do the following tasks in the SharePoint community forums:
- Answer as many questions as they can that are less than 48 hours old. If they have more free time, answer as many questions as they can that are more than 48 hours old.
- Moderate by moving incorrectly posted questions or discussions, pinning answers for FAQs, marking relevant replies as answers, and notifying me of any inappropriate behavior.
The MVPs were very willing to do this because their contributions in the forums were a significant factor in their annual MVP award renewal. Yes, that was indeed the carrot that I had to dangle in front of the MVPs because like I said earlier, there's just no such thing as a free lunch. And I wish that at the time, I had the following "User Activity" report that Harvest provides to easily assess how MVPs (and perhaps more importantly, MVPs in the making) had been contributing to the forums:
With the help of the MVPs, I was able to boost the answer rate in the forums by an average of 8-10% over a period of 3-4 months. But as SharePoint continued to gain momentum in the market, even with the MVPs' help, the number of unanswered questions in the forums far exceeded answered questions. So, with a single report that showed this growing trend over a few months, I was able to convince my management that the problem was getting worse and that we needed to dedicate some support engineers to monitor the forums and answer questions. And the rest, as they say, was history. :-)
P.S. It was critical that I had leveraged the MVPs prior to asking my management for support resources because I was able to answer, "Yes!" to the question, "Are you sure that the real answer rate isn't higher because many replies of acceptable answer quality to questions simply aren't marked as answers?"
[Update 11/15: Here's a relevant post by Martin Reed on what not to do when trying to attain self-sustainability in forums: Online community competitions attract mercenaries.]
Posted
Nov 14 2008, 09:53 PM
by
Lawrence Liu