Where do junior doctors look things up?

A short time after my post on where medical students look things up, @drcolinmitchell tweeted about a paper on where junior doctors look things up.
Hughes, B., Joshi, I., Lemonde, H., & Wareham, J. (2009). Junior physician’s use of Web 2.0 for information seeking and medical education: A qualitative study International Journal of Medical Informatics DOI: 10.1016/j.ijmedinf.2009.04.008
I have to admit that when I first glanced at this paper I thought the methodology was good. There is talk of triangulation and inter-coder reliability etc. But when it is read more deeply much of it simply does not make sense because key concepts are so loosely defined. In the past few days I have seen this paper mentioned several times on twitter and in blogs, but there has been little or no mention of the poor quality of this study. Therefore I thought I should add my thoughts to the debate.

Method

The study took place in the NHS in England. The subjects were junior doctors. 55 were identified through a stratified sample (of 10 different specialties) from 300 graduating from a London medical school. 50 of these agreed to participate but only 35 completed all three stages. More demographic data on the participants would have been useful.

Next, they were given a questionnaire, used in previous research on this topic, and asked to keep a diary over at least 5 days of every website they accessed for work. Finally each participant was interviewed although themes were saturated after 20 interviews.

Results

From the survey data, 32 of 35 said they used web2.0 sites and of these 28 used wikis (read the content, only one doctor contributed to wikis). Next, looking at the diary data, confusingly, google.com is now referred to as web2.0 content, whilst in the survey it was not. 80% (28/35) of physicians used google during the five days. 25/35 reported using wikipedia. Smaller percentages used yahoo.com, doctors.net.uk and Facebook. This data is presented in chart form with percentages of physicians accessing each site (eg google, wikipedia, NICE). Presentation in tabular form with absolute numbers of accesses would have given more information.

The participants were asked to state for each of the 444 events where they accessed information online whether this was on a wed2.0 or user-generated content site, or hybrid, or traditional content site. The doctors said that on 235 occasions they were accessing web 2.0 content. However, the authors have classed the 142 uses of google and 115 uses of wikipedia (total 257) as web 2.0 content. No absolute numbers for the access of yahoo, facebook and doctors.net are given so the agreement between the authors and participants over what constitutes web 2.0 is not clear.

The authors then present themes from the interview data. Here "using the internet" is confusingly equated with web2.0 content. There is mention that doctors look things up online because it is easily accessible, and up to date, but at times they are uncertain about the quality or usefulness of the information found. The authors introduce a taxonomy of information needs from the interviews which they then use to analyse the information needs addressed inthe diaries, categorising 237 out of 444 internet accesses/information needs:

1. "to solve an immediate defined problem" "to advance an immediate task in the clinical context and forms a closed question with a specific answer" "closed questions" 107 of total information needs, of which 90 addressed through use of "hybrid or best evidence tools" (these tools are not specified)
2. "background reading on a subject" 130 information needs, of which 107 addressed through the use of "web 2.0"
   

It is not clear why 207 web accesses were not classified, or which sites were accessed in those diary entries.

Futher information is given on the way that doctors used google. 21 out of 35 mentioned using google as a way of navigating between trusted sites. It is not stated if these trusted sites were named in the diary.

There is then some discussion of how these (web 2.0) sites could be better used in clinical contexts. Doctors mentioned :

• patient education- comment is made of patient use of wikipedia and need to educate patients on different sites
• physician education- awareness of "web2.0 sites" as difficulty is in finding out about sites(wikipedia and google? or were they referring to some other web 2.0 sites? or to trusted web 1.0 sites?) , not much training necessary as sites so easy to use.
  
• remove blocks to web2.0 sites - it is reported that google is blocked in

I am not commenting on the discussion of the paper because I found the method and results section quite perplexing. No clear definition of web 2.0 content is given. It is not clear why the use of google is considered use of web 2.0. As Mark Hawker has pointed out google is a web 1.0 application. (Data is indexed by computers and pulled by humans. The content is not in any way user-generated or social. ) Previous researchers such as Sandars and Schroter, who this paper cite, did not consider google to be a web 2.0 application.

Because the authors did not use a clear definition of web 2.0 content this work can tell us very little about doctors use of web 2.0 content. It is possible that most doctors are using the same trusted websites that they have always uesd. They use wikipedia because it is easily accessible (free and no passwords needed) and is equivalent to an online textbook. The user-generated content of Wikipedia is not a factor for most doctors. Credibility of user-generated content for physicians did not emerge as a theme in the qualitative work. Instead they were concerned about how patients might use the same websites that they use.

Overall, I found the study very disappointing. We need debate and discussion on how best to address the informational needs of clinical staff. To me, the best description of these needs still seems to be Richard Smith's BMJ review in 1996. Now we should be asking, have the information needs of doctors changed in the last 13 years? How are these needs best addressed by current technologies and what tools should we be trying to develop.

What do you think? Am I being too harsh? Why did you like this paper?

Metaphors of Medicine... and implications for medical education

Just a short post. Last week I saw Annemarie Mol speak at COMET09 about the 'messiness' of clinical practice. Today I received an email from a colleague where we were discussiing the fact that many aspects of medicine do not "offer a script". And then through Rakesh Biswas I came across this blog of Shashikiran Umakanth, a physician and associate professor in India. His former student comments that "no patient comes in a neatly packed diagnostic disease".

So these metaphors of medicine are about its complexity and unpredictability. But do we acknowlege this in our medical courses?

What are your metaphors for medicine and how do they inform the way we prepare students to become doctors?

Tweeting from a conference.

If you are following me on twitter you will have noticed many #comet09 tweets in the last two days. There will be more tomorrow as the conference finishes in the afternoon. So what was it about? COMET is an interdisciplinary conference in communication, medicine and ethics. It moves around the world and this year was on my doorstep in Cardiff so too good an opportunity to miss.

I decided to tweet because I thought the content would be interesting to quite a few of the people I know on Twitter. And it was. Some of the things that happened:

• @jrbtrip found out what an old colleague was up to
• @hadleybeeman and I had a discussion about whether patients had the right to not engage in partnership with their GP in their care
• @mrhyde found out about some research on how people who are obese access information on the internet.
• @vmontori who was a co-author of some work that my colleague presented was kept up to date although he was still on the other side of the Atlantic.
• @casesnetwork kept up to date with Brian Huritz's lecture.

And after seeing a presentation about using multiple modalities in drug patient information leaflets I remembered that I had seen a lightning talk about this last week at Health Camp but couldn't remember who. I tweeted and had the answer before the speaker had finished her presentation.

I do seem to have been the only person tweeting from COMET 09, but I predict that come COMET 10 in Boston there will be a few more. This was not about providing a backchannel to the conference, but simply about bringing the contents of interesting dialogues to a wider audience. So if you are listening to someone stimulating, think about tweeting. The chances are that someone you know will be glad that you have made the effort to share.

Where do first year medical students look things up?

In the last two days I have spoken to 31 first year medical students about their early clinical attachments in primary and seconday care. I asked them where they looked up unfamiliar clinical topics. These are some of the responses:

• Wikipedia
• Google
  
• Kumar and Clark
• Medical dictionary
  
• YouTube (especially to find out more about operations)
• NHS Direct/Choices
• Oxford Handbook of Clinical Medicine
• I didn't look anything up.

Wikipedia was definitely the most common choice. Many students said 'I know I shouldn't but....' and then qualified that they used Wikipedia first because it was easy to understand, they felt it was reasonably reliable, and accessible. One student used it to search directly from her phone when on placement.

I was intrigued by one student who was very keen to distinguish 'learning' which was what he did for exams... spotting questions on past papers and reviewing lecture notes... from 'experience', when he would access YouTube or Wikipedia to find out more about something that really interested him. His reluctance to call this learning reminded me of a third year student I spoke to earlier in the year. We were talking about how she would continue learning for the rest of her life. "That's so depressing", she said. In her mind learning was bound up with exams and assessment.

Should we worry about students turning to Wikipedia so often? Which other resources are just as user-friendly and comprehensive?

I think that NHS Choices is a good place to start.

EDIT: Just to make clear, the first year students I am referring to here are in an undergraduate 5 year course. The first few years of the course are pre-clinical but these early clinical attachements are to give them some initial insights into the world of clinical medicine. Some medical schools in the UK have no distinction between the pre-clinical and clinical parts of the course.

Facilitating a network

I am hoping to establish a network between those leading programmes in which medical students have the opportunity for contact early in the course with patients in their own homes or in the community. I think we will benefit from sharing motivations, materials and ideas on assessment. So I am wondering how I could support this. We could simply have an email list. Or a google group. Or a NING.

What would you use?

EDIT: This is to be a network with my peers- fellow medical educators- not with students.

Are you a digitially competent doctor? Do you need to be?

I was talking to a colleague in the last few days about digital competencies. He wondered if we should be exploring developing a set of competencies for medical students.

I started this blog more than 6 months ago as part of my learning journey on the use of web 2.0 (or whatever term you prefer) technologies and to meet other people who were on the same journey in medical education. I know that I could not yet say what the competencies needed for medicine are. I work with and know many good doctors providing good quality care and they do not blog, or use social networks, or collaborate online in wikis, or use rss feeds, or save or share links in social bookmarking tools. Would they be better doctors if they did?

And if I am not competent yet myself how could I decide that these, or others, are areas which students need to be competent in. How could I assess if they are competent?

Yesterday I was co-ordinating 3rd year exams assessing students skills in clinical examinations. Competency in clinical examination has been regarded as essential for doctors for many years. Maybe in the future it will be irrelevant. But for the moment we, as a profession, hold that it is important.

Will we ever have the same agreement about digital competency?

In praise of the walled garden (VLE)....

I have to start this by saying I am not a techy. I struggled a few nights ago to install MS Office on a netbook. But I am interested in how new technology can improve the way that we do things.

Back in 2004 I was invited to go on a Blackboard training session as there were plans that the medical school would use the VLE " increasingly to deliver course information and material". But when I went to the training session it wasn't this that got me excited but the discussion boards. I immediately thought that this would be a good way for me to communicate with and facilitate communication between 300 2nd year students undertaking a course I co-ordinated over 9 months. They were not even based in the same building as me. I've posted more about this here.

This year I used discussion boards, wikis and a course blog. Participation is voluntary. I don't assess contributions to the boards but students seem to find them a good way of accessing me and sharing with each other. The connections that they make through the discussion boards should help them to do better in the assessed written work.

So in my experience VLEs can work.

But many people do not like VLEs, or the way they are used or what they stand for (large, monollithic companies which I don't like either).

Martin Weller said the VLE is dead or dying back in 2007.Instead we will using "Loosely Coupled Teaching"... lots of different, freely available websites pulled together. Yes, that could mean lots of different log-ins and getting to grips with different websites but learning how to use wikis and discussion boards and blogs takes time no matter where they are, and tools such as openID, and facebook connect, might get past the log-in problems.

In 2009, Mike Bogle wrote about Distributed Online Learning Frameworks, now possibly including twitter, and was inspired by the experience of David M Silver.

But talk about moving away from VLEs is not just that they are big and cumbersome and slow, there is also a sense among many that it is the walled garden that is the problem. Access is restricted to those within the course within the institution. It is anti-edupunk and anti-connectivism. Mike Johnston thinks the VLE might be 'killing connections' for the institution's benefit.

But might there not be advantages to a walled garden? Can't students benefit from being able to talk and share in a private place where they can make a mistake and ask or say something stupid. We know the Cisco Fatty story. We're learning about digital identities. Is education in public really better? If institutions have any role in education might it not be the provision of a walled garden or safe space?