Usage of Content in
Web-Supported Academic Courses
Tel-Aviv, 69978
Dr. Rafi Nachmias
is a Faculty member at the
Limor Segev
is a graduate student at the
The use of the
Internet as an instructional tool in higher education is rapidly increasing.
Most universities, including leading academic institutes, are implementing
advanced technologies as a part of existing teaching frameworks (Mioduser & Nachmias, 2002).
Some models were developed for characterizing online courses (for example, Harasim et al., 1995; Bonk et al., 1999). Harasim (2000) claims that the invention of CMC on one hand
and the Internet (specifically the WWW) on the other, led to the development of
two main models for online learning: one based on cooperative learning and
interaction, while the other relies on publication of information on the
Internet. Indeed, we are currently witnessing the development of huge amounts
of web-based learning materials (Bork, 2001) and
web-based contents has become a major component in many academic courses (Nachmias, 2002). Cummings, Bonk & Jacobs (2002) believe
that presentation of educational contents on the Internet is highly valuable
for students, who enjoy visual presentation of information and supplements to
materials taught in lectures. However, the authors worry that presentation of
learning materials does not necessarily lead to their utilization by students
in order to enhance their learning or course understanding. Another source of
concern derives from the fact that the development of online learning materials
requires investment of considerable amounts of resources, financially and in
time and effort (Nachmias, 2002). Therefore, the
extent to which these resources have added value must be examined (Soong, Chan & Chua, 2001).
The dominance
of contents in course websites, the investment in preparing this material, and
the lack of certainty regarding their utilization, emphasize the need to
examine whether and to what extent students make use of learning materials
presented in course websites. However, there are only few studies dealing with
this issue (for example, Rafaeli & Ravid, 1997). Moreover, these studies have examined
consumption of contents in specific courses and do not deal with fundamental
questions relating to content usage in course websites. One of the few studies
attempting to identify patterns of content usage in course websites was conducted
by Sasson and Nachmias
(1999). They analyzed the connection between the structure of information pages
in online courses and the way in which these pages were viewed by students.
Results showed that viewing of information pages in the course website
decreases as a function of the location of the online chapter in the learning
process and the location of the lesson within the chapter. In addition, it was
found that many information pages were not being viewed at all. In general,
then, the observation is that content at deeper levels is less likely to be
viewed and used in the learning process.
One of the
unique tools that can be used for evaluating content use in course websites is
the analysis of the computer log derived while students access the contents.
There are number of reasons why analysis of computer log files may be
worthwhile (Peled & Rashty,
1999). First, such an analysis gives us insight into students navigate in a
course website. Secondly, one can trace the usage of the course website, by
looking at usage of its different components. Evaluation of online courses by
means of analysis of computer logs also has methodological advantages: it does
not suffer from biases due to self-report methods (McLaughlin, Goldberg,
Ellison, & Lucas, 1999). In addition, the information regarding students’
learning is accumulated automatically and the data collection does not
interfere with everyday learning activities.
The main goal
of the current study is to evaluate content usage in websites of academic
courses, applying the potential of computer log analysis for this purpose. The
evaluation of content utilization involves three main questions.
1. To what extent are contents presented in
course websites?
2. What is the usage rate of content items in course websites?
3. What are the individual differences among students regarding content consumption?
METHOD
“Content
item”: Content item is
a general name for all types of information uploaded into a database of course
websites, i.e. presentations, lesson summaries, exercises, etc. Content items
vary also in their presentation format (Word, Acrobat, PowerPoint etc.) and in
their size. Each student could access and see only the learning materials of
the courses he or she were signed to.
“Viewing
a content item”: By the word "viewing" we refer to
each time a student clicked on a link to a content item and opened it. We shall
have to assume she or he looked at that item and read it.
The study was
conducted on 117 course websites which are part of the Virtual TAU project, during 2001. Over 5,000 undergraduate,
graduate and Ph.D. students from all faculties took part in these courses.
Course
websites that were included in the study were developed as part of the Virtual TAU project -
Data for this
study were gathered from the automatic documentation of students’ viewings of
content items in course websites. The following steps were taken with relation
to raw data: The number of content items was counted for each course; the
number of content items viewed by each student was documented, and the number
of students viewing each content item was examined. The data were analyzed by
quantitative means, using descriptive statistics.
The results are
presented according to the research questions.
Amount of
content items presented in course websites:
In 117 course
websites examined over 3,300 content items were presented. The average number
of items presented in each site was 28 (SD 24.9), ranging between
2 to 118 items for each website. For most courses (88 courses, comprising 75%
of the courses in the study) up to 35 items were in existence. Only in one
course over 100 items were presented. There were no differences in the number
of content items presented in websites of Exact Sciences faculties in
comparison to Social Sciences and Humanities faculties, nor between websites
for undergraduate vs. graduate courses. A positive correlation was found
between the number of students in a course and the number of content items
presented in the website (R=0.21; p<0.05).
Characteristics
of content item usage:
Examination of
contents presented on the website reveals that students viewed a large
proportion of this content. Of the 3,301 items available online, students
viewed 2,926 items, 89% of all items presented. An average of 25 items in each
course website was viewed (SD 21.9). Over 35,000 content items
were accessed by the students in all courses together. Items that were not
viewed were found in 59 of the 117 course websites (about half of the studied
courses), i.e. in about half of the courses all content items were viewed by at
least one student.
In addition,
results show that each item was viewed by an average of 28% of the students
allowed to access it (SD 21%). It was found that 55% of the items
were visited by less than a quarter of the students listed in the course, and
only 16% of the items gained the attention of over half of the students.
Moreover, only 2 items were used by more than 90% of the students, and no
content item was found to be used by all students listed in a course.
Individual
differences among students:
One of the
central findings of this study is that 62% of the students viewed the content
items, while 38% of the students listed in the courses did not do so even once.
The study shows that the rate of students that did not use the items at all was
higher among undergraduate than graduate students (c2=0.09; p<0.01), and again among Social Sciences and
Humanities students, as opposed to Exact Sciences students (c2=-0.08; p<0.01).
According to
findings, among students using content items there is variance with regard to
the number of items viewed. Most students viewed only a small percentage of the
available items, whilst few students viewed all of them. About 20% of
the students have viewed no more than one tenth of the items available, using
only a small part of the information presented in their course website. Other
students, however, viewed a greater percentage of the items presented, with 5%
consuming all content items presented on the course website. Each student
viewed an average of 38% of the items available (SD 30%).
Examination of
individual differences among students has shown that students in the Social
Sciences and Humanities faculties viewed a larger number of items compared to
students in Exact Sciences faculties (t (3165)=-13.5; p<0.01). In addition, graduate
students viewed a larger number of items compared to undergraduates (t
(3165)=-16.8; p<0.01).
Universities nowadays are offering a lot of distance learning courses via the Internet to potential new students. The more courses are offered this way, the greater the need to understand how students learn from course web-sites. Hence, it is important to outline and study learning patterns of students in course websites.
The present study shows that lecturers in their first year of Internet integration in instruction rely on the course website in presenting large amounts of learning materials. One may conclude from this finding that the lecturers view course web-sites as a useful means for improving teaching. A similar usage pattern of course websites by lecturers was described by Bonk (2001), who found that most lecturers upload files to websites and use means of presenting questions, queries and case studies. Harasim (2000) also recognizes the fact that the Internet has created an unprecedented emphasis on the publication of information. She claims that this derives from the preference of lecturers for traditional teaching models, mainly frontal lectures. Peffers and Bloom (1999) ascribe this emphasis to the fact that these activities are easy to implement and do not oblige a great investment in resources. For many of the lecturers in the current study it was a first and innovative experience in implementing the web in academic teaching. It is possible that they not ready yet to change their instruction philosophy and habits.
One of the
remarkable findings in the current study is that most of the content items were
viewed by the students. In this respect it seems that the references and
resources were useful to the students. This finding regarding the relevance of
contents contradicts the descriptions in Sassson and Nachmias (1999), who found great variance in viewing rates
of web pages in course websites, with some pages not being entered at all. This
contradiction in findings can be explained, among other things, by the fact
that the web pages examined by Sassson and Nachmias (1999) were linear in nature. The researchers
found that this linearity bore a clear positive relation to the viewing of
content within the website. In the current study the materials presented in the
course websites were arranged in a variety of ways. Possibly, the fact that the
content were not arranged linearly caused students to expose themselves to
greater amounts of content items during the search for specific contents.
The study found individual differences among students
in viewing the content items in course web-sites. Nearly 40% of the students did not view content items at all.
Furthermore, there was a variance among students who did view the contents,
with many of them viewing only a few items, and some viewing a lot of them. A
similar finding regarding interpersonal differences among students learning
with courses web-sites is presented by Nachmias and Shany (2002),
who found a large variance in students' tendency to succeed in a virtual
course. The researchers found that one third of the students were left outside
the learning circle, while the online course suited about 40% of the students.
Findings of the two studies match the AFT report (2001), according to which a
certain type of student tends to succeed more than others in distance learning
courses on the Internet.
Findings related to variance among students with regard to usage of contents and characteristics of item viewing demonstrate the potential of computer log as a tool for evaluating ICT implementation in academic teaching. Furthermore, analysis of a computer log can be a valuable tool for developing a set of measures that will be part of online evaluation tools, which are easy to apply and understand. This kind of tool can be of importance on three different levels: the lecturer level, the project administration level and the decision-maker level:
1. For the lecturer, measures regarding the
degree and manner in which students view content items can be a powerful tool
for feedback relating to the implementation of the course website. The lecturer
can learn about the rate of content item usage by students and make decision on
whether certain actions should be taken in order to increase students' viewing
of the items.
2. A report including measures regarding students' behavior and content utilization for all courses in a certain project of ICT implementation can be a useful tool for project managers. The measures can be used for instructing the lecturers in developing and operating course websites, as well as for directing implementation efficiently.
3. Measures based on an analysis of computer log can be a unique and powerful tool for comparing the goals of developing course websites with their realization de facto. The measures can be used for comparison between faculties within a university, as well as for follow-up of a project in the course of its operation.
“We are in a period of history when new tools based on information and communications technologies have the potential to advance learning dramatically and make it more accessible” (The Advisory Committee for Online Learning, 2001, p. 28). There is no doubt that all those involved in learning processes are on the verge of an exciting era, that can dramatically change the manner in which students learn, lecturers teach and academic systems are operated. This is an era full of promise also for researchers focusing on different aspects of teaching and learning in academic institutes. For these researchers many questions regarding the implementation of ICT in academia are still a waiting answers.
1.
AFT
(2001). A Virtual revolution: Trends in the expansion of distance education.
American Federation of Teachers, May 2001 [On-line]. Available at:
http://www.aft.org/higher_ed/downloadable/VirtualRevolution.pdf
2.
Bonk,
C. J., Cummings, J. A., Hara, N., Fischler, R. B.,
& Lee, S. M. (1999). A ten level web integration vontinuum
for higher education: New rsources, partners,
courses, and markets [On-line]. Available at:
http://www.idea-group.com/Instructional_and_Cognitive_Impacts_of_web_Based_Education.htm
3.
Bonk,
C. J (2001). Online teacing in an online world
[On-line]. Available at:
http://www.courseshare.com/reports.php.
4.
Bork, A. (2001).
What is needed for effective learning on the Internet? Educational
Technology and Society, 4 (3) [On-line]. Available at:
http://ifets.gmd.de/periodical/vol_3_2001/bork.html
5.
Cummings,
J. A., Bonk, C. J., & Jacobs, F. R. (2002). Twenty-first century college
syllabi options for online communication and interactivity. Internet and
Higher Education, 4, 1-19.
6.
Harasim, L., Hiltz, S. R., Teles, L., & Turoff, L. (1995). Learning networks.
7.
Harasim, L.
(2000). Shift happens: Online education as a new paradigm in learning. The
Internet and Higher Education, 3, 41-61.
8.
McLaughlin,
M, Goldberg, S. B., Ellison, N., & Lucas, J. (1999). Measuring Internet
audiences. In: S. Jones (Ed.): Doing Internet research (pp.
163-178).
9.
Mioduser, D.,
& Nachmias, R. (2002). WWW in education. In H. Adelsberger, B. Collis, & M. Pawlowski (Eds.), Handbook on Information Technologies
for education & Training,
10. Nachmias, R. (2002). Research
framework for the study of a campus-wide web-based academic instruction
project. Internet and Higher Education, 5 (3), 213-229.
11. Nachmias, R., & Shany, N. (2002). Learning in virtual courses and its
relationship to thinking styles. Journal of Educational Computing Research,
27 (3). 313-327.
12. Peffers, K., & Bloom, S. (1999). Internet-based innovation for teaching IS courses: The state of adoption, 1998-2000. Journal of Information Technology Theory & Application, 1 (1) [On-line]. Available at:
http://ihome.ust.hk/~jitta/journal/Issue1/Issue1pg2.htm
13. Peled, A., & Rashty,
D. (1999). Logging for success: Advancing the use of WWW logs to improve
computer mediated distance learning. Journal of Educational Computing
Research, 21(3) [On-line]. Available at: http://www.rashty.com/art2.html.
14. Rafaeli, S., & Ravid,
G. (1997). Online, web-based learning environment for an information systems
course: Access logs, linearity and performance [On-line]. Available at: http://mis.huji.ac.il/papers/mmm.paper.htm
15. Sasson, E., & Nachmias, R.
(1999). What makes them click? Paper presented in the ED-MEDIA 99. World
conference on educational multimedia, hypermedia & telecommunications,
16. Soong, M. H. B., Chan, H. C., & Chua, B. C. (2001). Critical success factors for on-line
course resources. Computers & Education, 36, 101-120.
17. The Advisory committee for Online Learning
(2001). The E-learning E-volution in colleges and
universities. A pan-Canadian challenge [On-line]. Available at:
http://schoolnet.ca/mlg/sites/acol-ccael