<image No Trumpets Please! by Mr Jaded http://www.flickr.com/photos/jaded/1907907/>
Warning! This post may contain own trumpet blowing!
Laura Dewis sent me a report the lovely OU Online Services people had prepared in collaboration with consultants (MarketSentinel). They were interested in examining the broader influence of various web sites and looking at sentiment mining. The idea from an official communications perspective being you can see how well regarded your institution is in different sectors, and maybe influence that perception.
But from my perspective the analysis they performed could be tweaked to provide a measure of an individual's influence or prominence in the online community of their particular topic. As you will know, I am interested in the concept of 'digital scholarship' and in trying to get the type of online activity many of us engage in recognised as scholarly work.
The problem has been a mismatch between what has traditionally been measured to indicate academic standing in a particular field and the type of activity that takes place online. In short, the formal systems such as the RAE or the REF in the UK, are focused on outputs. They make a nod towards impact, or reputation, but it's really outputs that they want – and we largely know what these look like in this world: Refereed papers in respected journals; Keynote speeches; Gaining large research bids; etc. These don't map well onto the digital world – is 1000 blog posts good? What about 1000 subscribers? Is having a large twitter network a sign or standing? All too easily such measures fall down. So I have been looking for something more robust which might act as a metric for measuring an individual's reputation in their subject area.
The report the Comms team provided me gets some way towards this. They chose the subject area of 'distance learning' really as a test to see how it worked – one would expect the Open University to come out well in this. Here is the blurb from the company on how they determine influence:
When performing a MarketInfluence study our computer systems initially collect any documents on the Internet (web pages, Word, pdf or PowerPoint documents) which match a defined search phrase.
These documents (often hundreds of thousands) are then analysed. The analysis is especially focused on identifying who references whom.
Based on these references it is possible to calculate influence."
They then apply a number of filters to prevent self-citation. From this they drew up the following list of top 100 influencers in 'distance learning':
I come out 4th, Brian Kelly 6th and Tony 10th. I'm sure that Tony and I both gain some credit benefit from being associated with the OU, which drags up our distance learning status, but even so the mix of individuals in with large professional output such as the Guardian, JISC and BBC is interesting – the space has become democratised to an extent.
They then have a measure of friends in common, what they term 'betweenness':
"Stakeholders with high Betweenness are “stations” where information (on the issue in focus) is passed via in order to reach the constituency of said stakeholder. "
Here is the table for those with high betweenness:
They reference Gladwell's Tipping Point and his notion of 'connectors', to suggest some sites/people have a high level of 'hubness': "Hubness is the characteristic of disproportionately linking to those who are authoritative on a given topic."
It is still problematic, and could be gamed. I don't know enough about the algorithms used to assess this. One would also need to be careful about the search term used – 'distance learning' is quite a niche, UK term I think (had it been 'educational technology' we would have had a very different list). But, all this isn't (just) about ego massage – it strikes me that if we could develop such an algorithm so that we could easily enter any subject domain, this would provide a useful tool for measuring an individual's online influence/reputation/status etc in their field. This would then provide evidence for justifying this type of work and in seeking promotion. It could offer us the Alt-REF I was after. At the moment this work belongs to the consultants, and we would want to tweak it for academic use, but it does suggest that such an analysis is possible. A JISC project to develop the service for all academics?
What this gets at is that online activity is different – it is less focused around outputs and more around overall activity and reputation. And it does begin to back up what I've always felt – that this stuff isn't just peripheral, playing around, but increasingly is significant to individuals and organisations. I wouldn't want to try, but one could think about it in monetary terms – how much is this influence worth if you were to try and buy it (through advertising or other means?).