Social media monitoring – listening is The Future
So, yesterday’s Monitoring Social Media conference is over, and all I have to show for it is a heightened case of RSI (ok, ok, I jest). My live notes from the talk are here – Alice, you were an inspiration, I just had to call up mental images of your GDC typing-at-the-speed-of-light – how could I not publish the notes I was already taking? All that training at videogame events has certainly paid off.
But now it’s time to reflect and put together some marginally more coherent thoughts on social media and the lessons of the day.
Lesson One. Social media is people.
We’re finally starting to get it. Social media isn’t about numbers, or spreadsheets, or models, or calculating ROI to the last tenth of a decimal point. It’s about people, and you can’t (always) chain people down in tidy little tickyboxes and assign numbers to them.
We are not numbers.
This causes conflict in organisations that are used to the ‘old’ ways of doing things and don’t really understand the ‘new’. The case for the new was presented again and again and again yesterday. Look. We get it. Social media matters. People matter. It’s just difficult convincing higher-ups that it’ll impact the bottom line.
There were a few attempts to get some slightly more detailed answers on this subject. What exactly is the investment, when we talk ROI? Is it the cost of a tool? The cost of an agency? The cost of people? What will make the higher-ups listen? In the case of STA Travel, it was pointing out the properties of existing customers (that the STA relationship stopped once customers had booked a trip) and making a clear, coherent case for engagement to extend that relationship. But this brings me on to…
Lesson Two. Everyone is different.
We’re all human, and so naturally we want easy answers. But there are none. It seems that currently the range of social media monitoring tools (in terms of software offerings) is very much an off-the-shelf jobbie – obviously customisable to some extent within that, but still, off-the-shelf. Indeed, some companies with freemium/SaaS products seem to be encouraging this approach.
But if I learned nothing yesterday, it’s that everyone’s totally different, and that works for one client won’t work at all for another. Enter agencies, and humans (see point 3), and customisation, and tailoring. Hell, the agency behind Skype built a dashboard because nothing out there fit their needs! Weren’t all the SMM providers in the audience cringing at that? Speakers repeatedly said that today’s tools aren’t really that great – but some speakers praised them! What a load of mixed messages.
There is method to this madness, though, and it’s all about the human. People praising the tools probably used them well for their specific needs – people dissin’ them probably found that they were looking for something that the tools didn’t do. One thing seems sure though, the tools should work for the clients, rather than the 37signals-etc approach of ‘fit your thinking into the way the tool does it’.
Lesson 3. Automatic isn’t good enough.
This is obviously something I’m interested in, but it was almost disheartening to hear it repeated so much.
Basically, we need humans. We’ll always need humans. Tools help us cut down the humans’ time involvement, but there seems this fundamental mistrust – sentiment is wrong too much and too often, and even humans disagree 15% of the time (bang in line with the kappas I’ve seen in academia).
So even if there were a brilliant, perfect, 100% reliable sentiment detection system, it would be wrong 15% of the time, and so humans would want to check every message just in case. And if all you want is a ‘temperature’ type analysis, well, free tools already do that, and even allowing for error they’re just about good enough.
Lovely.
Lesson 4. We’re too close to the curve to see what’s around the corner.
The whole social media landscape is changing, and the monitoring stuff is just starting to catch up. Two years ago it was rubbish, nowadays it’s OK, and in two years it’ll be great. But the future’s not about technology, it’s about business intelligence, business process, and getting companies to embrace social media and its feedback loops at every level.
Because this is going to become such a fundamental part of how we do business, major players are already getting in the act. Search engines are integrating realtime search, so ’social’ SEO – building social capital – will become as important as keyword-based SEO. But you can’t just add in ’social keywords’ – that concept simply does not transfer.
As well as that, Google and Twitter could well be (hell, let’s just say it, they are already) developing their own social media monitoring systems. Google Analytics is powerful, but not in a social way – but it could be. Twitter could launch their own monitoring product and charge us for API use, creating an.. interesting, albeit unlikely, situation. Sure, cross-platform will still be a need, but we’ve already seen that that need varies so much even by department within a company!
One of the more interesting concepts to come up yesterday was that of an open source framework for monitoring social media, a plug and play approach that everyone could be using in two years – with a company making money where the hard stuff is, consulting and the human factor. I do wonder if this is perhaps viable, especially adding in outsourced human validation (MTurk) and cross-classification to reduce error.
Anyway, this is certainly all food for thought, and <shameless plug>should give me plenty to talk about at the RealTime ChristmasCrunch, at least!</plug>
Agree with your points.
I also think it’s worth mentioning that no amount of social media activity will help you if you have a crap product or service.
social media can help you find out where you suck and then communicate to your customers how and when you are going to stop sucking.
social media can also tell you and your prospective customers where you shine !
the point is that pretty soon *all* your prospective customers are going to be using social media to find out whether, on balance you suck or shine *before* becoming actual customers.
most of us suck at something, but using social media to communicate effectively how and when you’re going improve can actually be good for your reputation.
Hi Jennie,
I enjoyed your well written article. In particular, I agree with your point that technology cannot be 100% accurate.
“So even if there were a brilliant, perfect, 100% reliable sentiment detection system, it would be wrong 15% of the time, and so humans would want to check every message just in case. And if all you want is a ‘temperature’ type analysis, well, free tools already do that, and even allowing for error they’re just about good enough.”
For effective social media monitoring, someone still needs to look at the data to weed out spam, sarcasm, decipher acronyms and evolving languages. Thats why my company Brandtology weaves the human intelligence component into our processes.
Will be following your updates. :)
Ashley
Social Media Consultant
http://www.brandtology.com
Nice post. The validity and utility of info extraction tech built into commercial SMM tools was a topic I was to cover in my aborted case study pres at the end of the day (called away by a family emergency, and immensely grateful to Marshall Sponder for stepping into my spot).
For me, automated sentiment analysis in general is compromised but by no means useless. As you say, the focus on individual user needs is key – and while any margin of error is a problem when used as a tool to aid relationship management, it’s much less of a problem when used as a large scale research tool.
Investment banks are happy to use data of uncertain reliability for eye-wateringly large trades via high frequency trading software after all. Understanding and modelling margin of error is the key to usability.
For me the main problem with sentiment automation that is built into SMM tools isn’t accuracy, it’s granularity (I’ve worked in environments where humans code sentiment, and results achieved in that way have their own set of problems).
Essentially, sentiment at document level is pretty hopeless. Sentiment in commercial SMM tools is no better than an interesting toy to us – we’ve experimented a lot with them, but just can’t find a way to make them useful.
And the open source approach is right on the money. there is much more potential in building out custom applications using open source data mining / information extraction tech, than in waiting for licensed SMM tools to meet research needs.
Great summary. Lots of great insights crammed into one post – thanks for advancing these topics.
As a paid SMM service provider, we learned about lesson 3’s wisdom quite some time ago, and have always offered human review with all our tiers of service. The idea of tools only being able to do so much and the need for people to take over really resonates with us. We listen to a lot of the feedback from our clients and the crowds, and when we hear the ‘automagic’ speak overshadowing the tremendous work vendors are doing in this space, one wonders whether its become more of a problem with overselling the tools.
With regard to your comment on tailoring for the tool, I come to this argument with the bias that the paid SMM vendors need to spend more time understanding the requirements and strategies of each opportunity that converts to a client assignment. The other bit is that vendors need to assume some responsibility when it comes to explaining any/all limitations clearly. The basis for this argument is that clients are coming to paid vendors with the expectation that we are ahead of the SMM curve, and vendors need to live up to that role in so far as equipping the client with a tool that fits their specific and unique monitoring/listening requirements.
On your points in lesson 4, I’m just not hearing enough “involve” and “evolve” happening in terms of ridding the problems of “spam” and “malware” from monitoring dashboards and subscribers inboxes. It kind of reminds me of the way MS Outlook kept going through different iterations without tackling the spam problem. The trend most vendors seem to be taking is to continue to inundate machines with the tasks of analysis and “smart” filtering, and giving Google and Microsoft – two companies with no track record on “personalized” service – full license to enter the SMM fray certainly isn’t going to change this trend.
IMHO, the precision and qualitative analysis portion of SMM tools is what needs to be happening around the corner. What’s kept the tools and this space going for this long is the “service” element and that still needs fine-tuning. It’s where the reputation of this space rests and to some extent its worked in keeping the big players with no service experience on their resume at bay for this long.
Joseph
@RepuTrack
I have little to share and none of it is new.
My world is narrower than an ant’s.
From this, my lowly station, it appears that ever so often the wisdom de jour served as the summary of some fancy convention is the excited ground shaking rediscovery of the most fundamental precepts of human interaction.
Social evolution is, and will increasingly be, a slower process than technological innovation. Tools and platforms will come and go but the way people relate to them will remain relatively static.
Conversations once reserved for water-coolers and barber shop gossip now take place online and at the scale this allows, but the underlying need and governing principles are pretty much the same:
Be a mensch,
Don’t kvetch and moan too often,
Do unto others as you would have them do unto you,
Wear sunscreen and never forget your towel.
: )
@pop_art
You should rename this post to Social Media Truths! ;)
You are correct, no matter what tools are out there, we cannot forget the human element. It pays to listen, but such returns come only after you have established a relationship. Humand forge relationships, not statistics or automated features or a faceless brand/organization.
Looking forward to reviewing your live notes.
Lauren Vargas
Community Manager at Radian6
@VargasL
Dear Reid
Thanks so much for this post. You wrote:
“So even if there were a brilliant, perfect, 100% reliable sentiment detection system, it would be wrong 15% of the time, and so humans would want to check every message just in case. ”
But our tests found out that it is worse than what you describe:
http://commetrics.com/articles/fails-validity-test/
Regards
Urs
@ComMetrics
Urs,
Afraid your comment got flagged as spam for some reason so I’ve only just picked up on it. (I’m not Reid, by the way!)
Not sure you quite got my point. Humans disagree 15% of the time. Your post doesn’t really relate to that, and it certainly doesn’t provide solid figures – just a quantitative “oh no everything is terrible” kind of outlook!
J