Academic Exchange Quarterly Spring 2003 ISSN 1096-1453 Volume 7, Issue 1
Daniel Kipnis, Thomas Jefferson University.*
Dr. Anthony Frisby, Thomas Jefferson University.
Daniel Kipnis is Education Services Librarian at Scott Memorial Library. He was awarded a Master of Science in Information from the School of Information at the University of Michigan, Ann Arbor.
Dr. Anthony Frisby is Director of Education Services at Scott Memorial Library. He was awarded a Doctorate in Instructional Design and Technology from the Ohio State University.
At Thomas Jefferson University, Academic Information Research and Services (AISR), has designed a required online Medical Informatics course for 230 first year medical students. The course is designed to demonstrate the need for lifelong-learning skills, to train students in how to ask the appropriate questions to find an answer to their information needs, and to instill an awareness of the various types of information sources available to them and the skills to use these resources.
The entire medical informatics course is completed online. Each student must complete a computing survey, a pre-test, two case studies, and a post-test. A common misconception among both students and administration is that because students are now more familiar with searching due to the World Wide Web they are automatically able to also search the scientific literature for answers to their questions. Based on data from the computing surveys, post-course evaluations and the answers from one question from the case studies, this paper will demonstrate the continued need for teaching Medical Informatics to medical students, and their reactions to learning medical informatics online.
During the second semester of their first year, all Thomas Jefferson University medical students participate in a three-week course called Medical Informatics. Medical Informatics is part of ‘Jan (January) Plan', which also includes courses in Genetics, Health Care Policy, Ethics and Biostatistics. Medical Informatics is the only self-taught computer course.
The concept for ‘Jan Plan' was developed in the late 1980's. Faculty and the curriculum committee felt there were increasing medical related concepts that students needed to learn about, but they did not fit into existing course structures. These five three-week courses were designed to provide a break period from the intensive workload required during the basic science courses and to introduce new medical aspects into the traditional first year medical school curriculum.
All students are required to attend an introductory lecture and are expected to complete the online course within a three-week period. In order to successfully pass the course, the student must complete the two assigned case studies and score 80 points or higher on the post-test. The course objectives and metrics are outlined below.
"An understanding of the need to engage in lifelong learning to stay abreast of relevant scientific advances.” 
Life-Long Learning is more than simple computer literacy. Life-Long Learning requires: the ability to recognize when you need additional information, the knowledge of where to go to obtain information, the ability to ask the proper question, and the skills to evaluate and act on the new information.
"The ability to retrieve (from electronic databases and other resources), manage, and utilize biomedical information for solving problems and making decisions that are relevant to the care of individuals and populations.” 
"The ability to critically evaluate the medical literature and to seek opportunities to expand understanding and appreciation of scientific discoveries and their applications.” 
Before beginning the online course each student is asked to complete a computing survey. Of the 230 students in the class, 211 responded, which produced some helpful data in teaching medical informatics. This survey instrument helps AISR analyze the class's general familiarity with different resources, potential skill level, and assist with planning the number of hands-on workshops that might be required. Oddly enough there are still significant differences in student skill levels year to year. Some students demonstrate effective searching skills and demonstrate extensive usage of medical databases, while other students lack experience. Certainly they have improved as a whole since the early 90's, but there are year-to-year fluctuations as well.
One question asks, "During the past year, how often did you search MEDLINE or CINAHL?"
211 students responded with the following results:
· Daily 2%
· Weekly 9%
· Monthly 19%
· Seldom 42%
· Never 28%
Students also lacked exposure to other medical databases. Another question asks, "Please indicate each of the information resources below which you have used:"
· Current Contents 8%
· PsycINFO 17%
· HealthSTAR 23%
· Science Citation 13%
· Micromedex 32%
· MD Consult 46%
· Cochrane 3%
· Google /Other search engine 74%
· Metacrawler 15%
· Best Evidence 2%
The majority of students had used search engines such as Google, but the majority had no experience with the types of medical databases that they will be expected to use as medical students and practicing physicians. Moreover, only 52% understood MeSH.  “MeSH is the National Library of Medicine's controlled vocabulary thesaurus. It consists of sets of terms naming descriptors in a hierarchical structure that permits searching at various levels of specificity.”  This data indicates that teaching Medical Informatics to medical students is still an important and necessary element of medical school education.
Based on the data collected from the computing survey, Jefferson medical students expect to have access to electronic resources and welcome the opportunity to learn in an online environment.
· Over 90% of the students planned to have their own computers.
· Ninety-two percent used the Internet daily (we did not specify for what uses).
· The two most preferred search engines were Google (41%) and Yahoo (38%).
· Sixty-eight percent had used computer-based learning programs in undergraduate courses. (In 1991, only 35% of students had exposure to computer-based programs in their undergraduate courses). 
· Sixty-one percent of the 68% who had used computer-based learning programs found it an effective way to learn. (In 1991, 70% of the 35% who had used computer-based programs in their undergraduate curriculum found it an effective method for learning course material). 
· Ninety-seven percent recognized the importance of learning medical informatics skills. (In 1991, 90% of the students recognized the importance of learning medical informatics skills). 
· Ninety-nine percent recognized that medical informatics skills would be important for their skill set as physicians. 
Surprisingly, the majority of students had confidence in rating their medical informatics skills even though they had not had experience searching medical databases. Forty-two percent had seldom searched MEDLINE or CINAHL and 28% had never searched either. 
Even though the students had not searched MEDLINE or CINAHL, they were still confident in using online technologies. Fifty-four percent expressed confidence in being able to locate an authoritative discussion of medical pharmaceutical or therapeutic topic on the Internet, while 48% were confident that they could locate authoritative patient education material on the Internet. 
Based on these figures, it is clear that these medical students are a confident group with extensive experience using the Internet to help find information. Although the majority have no experience searching medical databases, they recognize the importance of learning effective medical informatics skills, and the majority found computer-based learning programs an effective method to learning medical informatics skills.
In one of the case studies, we present a patient who has insulin-dependent type II diabetes who requires medication and is requesting permission from her doctor to go scuba diving. Based on information gathered from the Internet and medical databases, we asked the student if he or she would recommend that the doctor sign the release form.
The answers surprised us. Students made decisions based on the first web site they found acknowledging that scuba diving was safe. The students were expected to search MEDLINE and the web to find articles and URLs that would help them come to a well-informed conclusion regarding the safety of insulin-dependent type II diabetics who want to scuba dive.
Several students said that it was safe for insulin-dependent type II diabetics to scuba dive, and they offered the following web sites for evidence:
1. http://www.diabetesmonitor.com/diving.htm (Research into the Reliability of Current Guidelines for Recreational Scuba Divers with Insulin-Dependent Diabetes).
3. http://www.ymcascuba.org/ymcascub/diabetic.html (Scuba protocol for divers with Diabetes written by YMCA).
4. http://www.ukdiving.co.uk/ukdiving/info/medicine/diabetes.htm (Article written by Chair of the UK Sports Diving Medical Committee outlining standards for divers with Diabetes).
Many of these web sites post articles written by doctors, and for many students the M.D. designation was authoritative enough for them to allow diabetic patients to scuba dive.
First year medical students did not know what questions to ask in doing research. For example, only a handful of students thought of searching for practice guidelines. How can one proceed when there are no practice guidelines established? Is there a better source of information? Yet, many students did not question the web sites and proceeded to say that it was ok for a doctor to authorize a patient with insulin-dependent type II diabetes to scuba dive, regardless of their sources' credentials.
None of the students thought of a government agency that might offer neutral information about this situation. The National Oceanic and Atmospheric Administration diving program (http://www.ndc.noaa.gov) has a medical form online that is designed for an examining physician in their evaluation of an applicant's fitness for scuba diving.  This form was last revised November 2001 and it clearly states:
"Endocrine- Endocrine disturbances including diabetes, required medication shall disqualify the applicant [from scuba diving].” 
In addition, in the following MD Consult patient education handout "Travel to Developing Countries. What general Health Precautions should the traveler make?" states:
"Travelers planning to descend rather than ascend must also take precautions. Individuals with the following conditions should not scuba dive: … insulin-dependent diabetes.” 
Also, among the absolute listed contraindications for fitness for diving is insulin-dependent diabetes mellitus. 
One study did find it safe for divers with type-1 diabetes to scuba dive.  The divers who participated in the study remained free of symptoms and signs of hypoglycemia throughout the course of the trial, but the researchers concluded that the sample size was small and larger studies including subjects with type II diabetes would be necessary to extend these results to the diabetic diving population at large. 
The variety of responses on this one question demonstrates the continued need to educate medical students in the essential skill of evaluating and questioning resources. In an effort to educate medical students to properly evaluate web sites we have presented them with questions that they should ask themselves as they conduct research:
· Is the journal title part of the Abridged Index Medicus, the 120 core clinical medical journals?
· Who is the author and what are his/her credentials?
· Are complete and accurate references listed? In some instances, references are erroneously cited and this is an indicator of faulty research.
· How current is the information? Obviously, this depends on the type of research that your are conducting, but the publication year of an article can have relevance.
· Is a bias expressed in the abstract? This may be difficult to determine, but look for political, ethical, commercial or religious arguments.
· Is the information peer-reviewed? Who is reviewing? Are the authors authorities in the field? Conduct an author search to determine what other articles they have published.
· Is a study sample size large enough for effective conclusions?
Many students believe that just because it is on the Internet that it is a valid source of information. This case study question, and the responses garnered indicate, that students need continued medical informatics training to effectively conduct sound clinical advice to their patients.
Student responses indicate that online medical informatics can be successful. The students had very positive comments about the online course.
· For 79% this was their first extensive training in how to use the medical literature databases.
· 74% of the participants were satisfied with the medical informatics course.
· 81% found the online resources valuable for learning.
· Before taking the course 25% rated their searching skills at good or excellent.
· After taking the course, 88% rated their searching skills at good or excellent.
· 85% thought the two case studies were good to excellent.
· 92% felt the computer-based course was an effective way to learn how to search the medical literature. 
This data clearly indicates that first year students at TJU welcomed the online medical informatics course and found the benefits of the format an effective method of learning a valuable life-long learning skill.
Several comments from the students regarding their experience with the online courses included:
· "I like the computer based aspect. It meant I could do the work at my own pace and actually finish it before many others even started"
· "I felt that it was a good course, with the appropriate level of work"
· "Everything was great"
· "I liked it and thought that it was interesting.” 
The Medical Informatics course at Thomas Jefferson University is an alternative method to teaching medical informatics to medical students. Overall, the students are eager to learn in an electronic environment that is self-paced. Based on their answers to the one question in our case study there is still a need to help students learn how to distinguish between resources and what constitutes the best resources available in helping to answer real life clinical situations. Based on our data, online research and education continue to increase in popularity. In 1991, 50% of JMC students had "not much" research experience with computers, while only 25% had moderate or "very much.”  The emergence of the Internet in the mid 90's and the increase in online databases are key reasons for the popularity of computer based research, but based on survey results from 2002, online education has also grown in popularity among medical students. The survey results show that Jefferson medical students are willing and enjoy the use of computer-based instruction. Their answers also indicate the need to continue to teach them effective use of online technologies and how to discriminate among sources.
Our findings suggest that other healthcare providers would also benefit from the skills necessary to answer their clinical questions. This is much more than being able to "search the Internet." It requires the ability to recognize the need for more information, the knowledge of the different resources available, the different search tools/languages the particular resource uses and an awareness of the credibility of resource and findings. We are preparing a freestanding self-evaluation tool that will help users identify which skills they would benefit from learning and to help them develop those skills. Our target audiences are practicing physicians and nurses at the Thomas Jefferson University Hospital.
* I would like to thank Tricia Kipnis, my wife, and Liz Mikita, Education Services Librarian at Thomas Jefferson University, for their comments and guidance in writing this paper.
 Thomas Jefferson University "Leaning Objectives for Medical Education" (Revised from the Medical School Objectives Project Guidelines of the AAMC.) 2000.
 Jefferson Medical College computing survey. January 2002.
 National Library of Medicine “Fact Sheet Medical Subject Heading” Available: <http://www.nlm.nih.gov/pubs/factsheets/mesh.html> Accessed: December 10, 2002.
 Office of Academic Computing. Annual Report. Thomas Jefferson University (1990-1991): D-1.
 The NOAA Diving Program "NOAA Diving Medical Evaluation Criteria Form"
Available: < http://www.ndc.noaa.gov/pdfs/SFKeys2.pdf >. Accessed: February 7, 2002.
 MD Consult patient education handout "Traveling to Developing Countries What General Health Precautions Should the Traveler Make?" Available: http://www.mdconsult.com. Accessed: May 21, 2002.
 Dillard, Thomas A MD and Ewald, Frank W. Jr. MD, "The Use of Pulmonary Function Testing in Piloting, Air Travel, Mountain Climbing, and Driving. Clinics in Chest Medicine. 22(4) (December 2001), 812.
 Edge CJ, Grieve AP, Gibbons N, O'Sullivan F, Bryson P. "Control of blood glucose in a group of diabetic scuba divers." Undersea & Hyperbaric Medicine 24(3): 201-7, Sept 1997.
 Post course Medical Informatics evaluation. January 2002.
 Office of Academic Computing. Annual Report. Thomas Jefferson University (1990-1991): D-5