Wednesday 16 September 2009

Tech addiction 'harms learning' .....really??? $24.99 and I am no wiser




EDIT 11/12/09 This post has been nominated for an Edublog Award for "Most Influential Blog Post" You can vote here. Thank you to Sarah Stewart for her nomination.




Last night, I started noticing tweets about this BBC News Education story in my twitter stream. Researchers at Cranfield University had published a report "Techno Addicts: Young Person Addiction to Technology" about a study they had conducted where 267 secondary school pupils completed a written questionnaire about their mobile phone and internet use. Included in the BBC story is the statistic that 63% of respondents 'felt addicted' to the internet and 53% 'felt addicted' to their mobile phoneThe BBC headline ("Tech addiction 'harms learning'") suggests that the researchers have established a relationship between this feeling of addiction and poor learning. In fact, the headline suggests a causal relationship which a cross-sectional study could not establish, but the body of the text doesn't really support any relationship between addiction and learning.


I wanted to know more so I set out to find and read the report. Googling the full title pulled up a link to the Sigel Press site where the report could be purchased for $24.99. And a press release from Cranfield university confirmed that this was the only way to get my hands on it. It also was clear that none of the authors had an education background. The 2 main authors, Nadia and Andrew Kakabadse, have a blog showcasing their many interests but education doesn't feature amongst them. They descibe themselves as "experts in top team and board consulting, training and development". I bought the report.











I expected the report by university academics to follow a standard format but it doesn't. It is 24 pages long and contains no references and no appendices. The survey instrument is not included.


Mainly it consists of charts illustrating question responses. Unfortunately it contains some typos and poor grammar.


No response rate is given, although we are told that the single school contained 1277 students and that there were 267 respondents, so it may have been as low as 21%.

With regards to 'tech addiction' this seems to have been a self-assessment based on response to the question: How addicted are you to the internet or your mobile phone? The proportions given in the BBC report are those who stated they were 'quite' or 'very' addicted. Of course, we don't know what the students meant by 'addicted'.
With regards to this addiction harming learning, there is no analysis relating the perception of being addicted to outcomes in learning. In fact very few of the questions are related in any way to learning.
It is hard to understand several sections of the report because of lack of access to the questionnaire. For example, with regards to plagiarism the authors state that "A high proportion of students (84.3%) openly admitted that they inserted information from the Internet into their homework or projects on a number of occasions." The tone of this sentence reflects some of the bias which is found throughout the work. The authors don't seem to be aware that if referenced it is acceptable to insert information from the internet into work, so the students would have no reason to be ashamed and fear 'openly admitting' this. The finding that 59.2% of students have inserted information into work without reading it is more concerning. It is also reported that 28.5% of students "feel it acceptable to insert information from the Internet straight into schoolwork without editing or making adjustment, recognising that such behaviour is considered plagiarism." It would help a lot to see how that question was actually worded in the survey, as in the figure it is simply represented as "Ok to “insert” information from the Internet straight
into schoolwork- Yes/no". That's not quite the same!

But there is no analysis relating amount of time spent online (or perception of addiction) and likelihood to insert internet contents into work without reading it. It may be that those who spend less time online, have less skills in information literacy and are more likely to plagiarise.

In summary this report tells us very little about internet addiction or learning. Do you think that someone writing for the BBC website actually read the report? Many of those who tweeted about the BBC article thought there were no suprises in the findings, and that perhaps it suggested that teaching methods needed to change.

This evening Ben Goldacre and Lord Drayson were debating the state of science journalism in the UK. I wonder why do the BBC give space to research which is so poor? How did they manage to concoct such an alarming headline? And why do people believe it? Is it because as one person responded to me last night, there is the perception that "U may fault methodology, results true".
And the quotes from the authors are not even results, just their thoughts which may chime with readers. But it's definitely not science.



Image: "Playing with the new baby cell phone" http://www.flickr.com/photos/cwinters/2150107228/



EDIT: You can read the BBC response to this blog post here.


26 comments:

  1. When I Tweeted about this I thought it was comical that they had conducted the survey using a written questionnaire. But your analysis is a wonderful satire of their "research methods." A clever critical analysis. Thanks for turning your misfortune into a great read!

    In fact, I think the real issue is that digital tech gives students control and functionality over information - something they rarely get at school.

    Our readers might like my related blog post "Engage Student Discussion: Use the Social Network in Your Classroom" http://bit.ly/TlzrN

    ReplyDelete
  2. Thanks for sharing your thoughts on this report - your post confirms my suspicions. Sorry you had to be the one to fork out the $24 to discover that. I agree with Peter and thank you for entertaining us with this whimsical response.

    Readers might be interested in my post on this report - http://tinyurl.com/pylfa7

    ReplyDelete
  3. Generally Learning Technology research is disappointing. I have no intention of making things worse which is why I'm trying to do my bit 'properly'. The report's authors are taking their cash without regard to any discipline's need to carefully build theory. Fair enough: I feel the big publishers are no better, it's just that you're protected from paying every time through your organisational subscription. Even the better conferences (like Networked Learning) have sessions which do not seem to add much to what we already know and if you get unlucky you can really wish you were elsewhere (even in the keynotes). The reasons for this are many but I've mapped some at:
    http://bit.ly/4gcCVX

    ReplyDelete
  4. I was fairly annoyed when I saw it as well. The premise is just old fashioned, technology is here and it's reshaping how everything works including education. It makes me very happy that you sacrificed the money to show the fallacy plainly. Thanks for taking one for the rest of us.

    ReplyDelete
  5. The BBC in headline not reflecting the actual 'study' shocker? Well-I-never.

    And the good old British public largely not reading past the headline?

    Well, I am shocked to my very core ;-)

    Excellent blog post, glad I was directed here.

    ReplyDelete
  6. Great post. Have to say I can be guilty of just scanning the BBC headlines, without reading further. It's interesting to see someone go to the trouble of looking even deeper.

    ReplyDelete
  7. Thanks for the this post. I have to say my response after reading the article earlier this week was simply "bol****s"! Your analysis of the report thankfully supports my somewhat knee-jerk reaction! I really wish journalists would be a little more rigorous when it comes to selecting stories. It is articles like these that makes life difficult for people in education (like myself) who are trying to use technology in innovative ways for learning and teaching.

    Thanks

    Simon

    ReplyDelete
  8. Thanks for digging into this - if there was a tipjar that I could donate to to repay your $24 I'd pay into it.

    I've contacted someone at BBC Online to ask who wrote it, to point them at your blog post, and also asking why the article doesn't have comments enabled.

    ReplyDelete
  9. does anyone seriously believe ANY of the so-called research results published in the press these days? Even supposedly respected organisations are often guilt of slanting their survey questions to get the 'right' answer.

    Press are only interested in controversial headlines -so no wonder that when any genuinely, properly thought out research actually is carried out - and comes up with slightly equivocal 'maybe it's a problem, maybe it isn't results' that gets ignored in favour of spurious surveys

    ReplyDelete
  10. This comment has been removed by the author.

    ReplyDelete
  11. Great post and well done for probing more deeply. Too many headlines misrepresent the underlying research as it is, without giving credence to studies that are simply poorly executed by unqualified individuals and therefore deliver meaningless results.

    The BBC ought to know better but this is not the first time they've parroted this sort of thing. And it won't be the last.

    ReplyDelete
  12. Thans for this - like others who have commented, I had skimmed past this headline. Personally, I had assumed that it was probably based on rather dodgy research and the headline was sensationalised, but I have to confess I hadn't got as far as reading the article, let alone the report, which makes me as bad as those who had read it but without a critical approach.

    I am fascinated by the idea of 'addiction' to the internet and other digital media. I know that I find I feel a bit 'twitchy' when deprived of my usual channels, and can feel really quite low if the deprivation continues for any period of time. I certainly find that when my access is removed, I don't feel very inspired to find other ways to work. I have a theory about why this is - relating to the brain mapping the communication channels and when they are not available it finding a separate 'sense' is missing.

    It is unfortunate that so many of these "studies" are, apparently, not well designed, poorly interpreted and even more poorly reported. Quite often I find that even the sponsors of a report can produce a headline which seems to me to contradict the actual findings in the report itself - which I find worrying in the extreme.

    ReplyDelete
  13. A key issue here is that this report comes out of a university, so an obvious assumption is that this is grounded in proper research.

    Problem two - this assumption is done by busy people who haven't bothered to look properly or maybe lack the skills.

    Yes the Beeb got it massively wrong, and it is important we know why - but surely the university should get a massive smack for publishing something that doesn't appear to be up to par and charge $24 for the prospect.

    I'd chip in to repay the $24, but I think there should be a refund here.

    Deftly done Anne Marie - a good piece of work and one that could be described as good journalism, unlike the one you comment on.

    ReplyDelete
  14. Good critique.

    There is a difference in emphasis between the use of the word 'harm' (BBC) and 'disrupt' (in the press release). Journalists have a tendency to do this ... it gets readers' attention.

    I think the comment about the (mis)use of 'text message' spelling in particular says more about the bias of the researchers than the learning ecology of the students in this school.

    I feel disappointed that management consultants feel they have the skills to do good educational research.

    ReplyDelete
  15. How about this for a headline: "Relying on hack researchers and sloppy editorial procedures harm journalism".

    ReplyDelete
  16. Excellent piece of work. Has this been brought to the BBC's attention. It is clear that you have made more effort to analyse the document than they did. It is very tabloid/low grade of them to spit out the sound bite they think will bite hardest rather than be anywhere near a truth. Big headlines like these are what stick in the minds of those starting out with tech and can cripple their edventures with doubt. Well done for taking the time (&cash). TY. Dai

    ReplyDelete
  17. The conversation is being carried on here too:

    http://blog.core-ed.net/derek/2009/09/need-we-be-alarmed.html

    ReplyDelete
  18. It has been brought to the BBC's attention - I've emailed the editor of the Education section - and have still to receive either a reply or see a correction. In addition a number of BBC journalists have tweeted about it, but again, nothing seems to have filtered across/down/up. Rather disappointing.

    ReplyDelete
  19. Very precise analysis!

    I nearly always have similar experiences when I go to the original "paper" after I read some sensationalistic article on the major newspapers.

    Very disappointing. I'm not sure if journalists who report such poor "social research" know what science, scientific method and collection of data, statistics and evidence are.

    My conclusion? I stopped giing importance to any kind of scientific news on the mainstreem non scientific newspapers / magazines.

    ReplyDelete
  20. Has anyone had any response from the BBC about this yet? The article remains unchanged and I've had no response from my email to the Education Editor Gary Eason. Am chasing it up elsewhere with a view to a blog post.

    ReplyDelete
  21. Yes, the BBC did get back eventually. You can read all about it here:
    http://wishfulthinkinginmedicaleducation.blogspot.com/2009/09/bbc-responds.html

    ReplyDelete
  22. I work as a dental assistant training and find it hilarious the means at which they went about conducting their survey.

    ReplyDelete
  23. BBC responds link seems broken. I am sure it is getting lots of hits. Great post and good research - disappointed in BBC.

    ReplyDelete
  24. The BBC responds link is fixed now

    ReplyDelete
  25. I feel that this blog does an excellent job of pointing out the need for the media to be critical consumers of research. So often the media reports on articles which, though published, do a very poor job of ethically and effectively reporting findings. I feel it is important that we watch out for claims based upon unsubstantiated and poorly defined research.

    ReplyDelete
  26. Marvelous work.Just wanted to drop a comment and say I am new to your blog and really like what I am reading.

    ReplyDelete

I am reintroducing word verification to cut back on spam posts. I'm sorry if you find it frustrating,