Academic Exchange Quarterly
Spring 2004 ISSN 1096-1453
Volume 8, Issue 1
To
cite, use print source rather than this on-line version which may
not reflect print copy format requirements or text lay-out
and pagination.
Students Perception On E-Learning: A
Case-Study
Vito Pipitone, Italian National Research Council, Palermo
Giovanni Fulantelli, Italian National Research Council, Palermo
Mario Allegra, Italian National Research Council, Palermo
The authors’ scientific interests
concern the pedagogical, economic and social impact of the use of ICT in
training and educational contexts, with special focus on higher education and small
and medium sized enterprises.
Abstract
The reflections presented in
this work lead us to emphasize the existence of a problem in the new market of
university online learning. The costly evaluation of the quality of new courses
can in fact increase the risk for students in making the best choice. In the
absence of enrollment fees, which can act as a signal of quality, market growth
would seem to be severely hampered. On the basis of the answers to a question
administered to 1,790 students at Italian universities, we are interested in
analyzing the perception of quality of university courses available on line, in
order to distinguish signals, à la
Spence, which can reduce the problem of
adverse selection.
Studies concerning the way the
markets work often assume that individuals are able to make a correct
assessment of the quality and price of the goods which are exchanged. Unfortunately, these hypotheses are rarely
borne out in the real world.
In a famous article published
in 1970, “The market for Lemons: quality uncertainty and the market mechanism”,
George Akerlof introduces an idea, which is
simple but also of deep and universal significance. Using the second hand car
market as his example, Akerlof shows that when the
buyers and sellers have different information about the quality of the goods
(information asymmetry) this results in a problem of adverse selection on the
market. In other words, the agents who
have less information find themselves operating with those they would have
preferred to avoid.
In fact, if the quality is not
apparent, goods of both good and poor quality converge in a single market and
have a single price tag. The latter,
determined on the basis of an expected average quality, provides ample profit
for the worst sellers but not necessarily for the best who find themselves
driven out of the market. Consequently the buyers are left to operate only with
the worst sellers. So the term “lemons” is used to indicate poor quality goods
which are found in the market in the absence of correct information.
This conclusion leads Akerlof to explain the set up of many third-party
institutions which offer guarantees for the real quality of the goods, so
contributing to a reduction in the inefficiencies of the market.
The work we present here
attempts to analyse the problem of adverse selection in an unexplored context,
namely that of online university learning.
In this market there is great information asymmetry which penalises the
students (buyers of instruction) in their evaluation of the quality of the
courses offered by the universities (producers of learning). However, unlike Akerlof’s
hypothesis, the mechanism of adverse selection in this case is not due to a
reduction of the market price (strict, by definition) but rather to a reduction
in the number of buyers.
However, following the
contribution of Michael Spence, the market
itself can offer a way out of the problem. In 1973 in “Job Market
Signaling”, Spence showed that the best sellers are not necessarily forced out
of the market if they ‘signal’ the true quality of their goods. But in order for the signals to reach their
objective of contrasting the adverse selection, they need to have a production
cost which is inversely correlated to the quality of the seller. In other words, the signal must be more
expensive for the worst sellers.
Following the paths traced by Akerlof and Spence, there are numerous applications and
empirical tests to be found in the latest literature. In the financial economy,
for example, Myers and Majluf (1984) have shown how shareholders can become
victims of adverse selection among firms, while John and Williams (1985) have
highlighted how dividends can act as a credible signal of the profitability of
the companies quoted. In the field of industrial organization, Nelson (1974), Milgrom and Roberts (1986) have shown how the highest prices
and the use of advertising are generally considered by consumers to be signs of
the high quality products.
From a methodological point of
view, Rothschild and Stiglitz (1976) among others, show how, under certain
conditions, agents who are less well informed can indirectly obtain information
from those who are better informed, while Cho and Kreps
(1987) use the game theory to discriminate between the many equilibria
which can be resolved on the market in the presence of signals.
On the basis of the answers to
a question administered to 1,790 students at Italian universities, we are
interested in analyzing the perception of quality of university courses
available on line, in order to distinguish signals, à la Spence, which can
reduce the problem of adverse selection.
Our approach differs from some previous work. Hsing and Chang
(1996) and Bezmen and Depken
(1998) analysed the market demand for distance
learning and concluded that university fees are an efficient signal of the
quality of courses. In contexts where
the fees are fixed administratively, however, they lose their intrinsic value
as a signal. Alternative signals are
thus required.
As Harasim
(2000) emphasised , online learning must
not be confused with distance learning.
They share many features, such as the context not limited by space or
time, but they are completely different as regards group communication. This is
a phenomenon which, as Dean (1994) pointed out, is at the centre of the learning
process. It is in fact through group
participation that students have the chance to elaborate concepts, share
experiences, acquire knowledge and socialize. Moreover, interaction between
teachers and learners helps to increase student motivation and satisfaction,
and consequently to produce greater benefits.
From this perspective online learning appears to be more similar to
traditional learning, although differing in its pedagogical approach.
Recent studies of online
learning have tried to compare the results of online learning with traditional
‘face to face’ learning. Analyses of
students’ marks at the end of a course of studies by Smeaton
and Keogh (1999), Wade (1999), Navarro and Shoemaker (1999), Sener and Stover (2000), Fallah
and Ubell (2000) show that no substantial difference
exists between the results of traditional learning and online learning. This
would also confirm Russell’s position (1999) in his well known book “The No
Significant Difference Phenomenon” in which he underlines, however, the
importance of the quality of online learning.
While there are numerous
contributions concerning the effectiveness of online learning, according to Fresen (2002) there is little research regarding
quality. Phipps and Merisotis
(1999), in their report which generated wide debate amongst US academics about
what constitutes quality in learning, focused their attention on the
specificity of online learning. In fact
the authors noted that the way in which technology conditions the learning and
teaching processes is still far from clear, so any question regarding
‘specific’ traits of quality becomes superfluous. In agreement with this position, Pond (2002)
considers it useful to distinguish “universal” traits of quality, independent
of teaching methodologies. However, while it is true that the final
result of learning is important, notwithstanding the pedagogical approach, an evaluation of
the quality of the different specific teaching tools makes it possible to
anticipate, as well as to improve, the final assessment of the learning
process. We consider this to be the
correct line of approach and we are encouraged by the contribution of McGorry (2003), who identifies six ‘specific’ traits of
quality in online learning: flexibility, responsiveness and student support,
student learning, interaction, technology and technical support, and student
satisfaction.
Aside from the difficulty in
identifying ‘specific’ traits, the evaluation of quality from an empirical
point of view, appears to be particularly costly for potential buyers,
generating a classic problem of information asymmetry. Actually a solution to
this could be found in the workings of the markets, in sending signals. The
fees for online courses, for example, could be interpreted as signals of quality
if they were fixed for each individual course on the base of production costs
and expected demand. The problem becomes more complex in contexts in which the
enrollment fees are decided by the public
administration on the basis of welfare state policy. In these circumstances, which are, moreover,
common practice in most European countries, such a signaling function is
invalid.
If the market is unable to
send signals and if the evaluation of quality is particularly costly, the less
well informed agents are faced with the problem of adverse selection. Students who have no way of distinguishing
good online courses from mediocre ones, perceive a greater risk in choosing
online courses compared to traditional educational contexts. A “virtual
examination” of a course which is only accessible online, is, in fact, much
less conclusive than physical contact with a traditional educational
institution which can easily be assessed for the number of facilities, the
atmosphere of the campus and the level of satisfaction of the students.
While the risk of choosing
online courses is greater than making alternative choices, it is likely that
students who are averse to risk taking will opt for the latter, so causing a
drop in the numbers of online enrollment. This fact is particularly relevant
for the equilibrium of the online learning market. As Rumble (2001) stated, the creation of an
online course imposes costs which are strictly related to the use of
technology. Therefore, if there is a drop in student numbers, the faculties
which intend to make the best use of technology by offering high quality online
courses, may no longer find it profitable to stay on the market. Increased production costs could, in fact,
exceed the benefits. The best would find themselves excluded from the market
and the risk in choosing online courses of quality would increase. This leads a
majority of students to opt for alternative teaching methodologies, so setting
up a vicious cycle which would tend to wipe out the whole online learning
market.
The classic solutions to the
problem have already been indicated in the introduction to this paper, but our
attention is focused on the study of signals à la Spence.
In carrying out our survey we
collected 1,790 answers from students in the Faculties of Science, Political
Science and Communication Sciences at the
In the following list,
students’ answers and related scores:
career prospects 842
quality of teachers 725
interaction with
teachers 631
adequacy and clarity of
materials 461
degree of flexibility 387
ease of finding
materials 358
efficiency of auxiliary
services 351
technology used 281
limited number of
participants 230
personalization of
study programs 209
use of tests for self
evaluation 86
availability of tutors 79
prestige of the faculty 21
Career relevance appears to
receive most attention from the students. Over 46% of the students mentioned
it, considering it to be an important signal in evaluating online courses. It should be noticed that results of Dey, Astin and Korn’s studies (1991) on what quality means for students in
traditional university courses apply also to on line courses. The closeness of
the learning path to the world of work in fact presupposes great attention to
the curriculum and the effectiveness of group communication, factors which are
crucial to the process of acquiring skills and professional competence. In this
direction, the university could produce statistics, certified by external
institutions, about the percentage of students who have got a job (within six
months, one year and three years), their average salary and whether their
expectations have been fulfilled. For the
purpose of our analysis, it is interesting to observe that the production cost
of the highest values of the above indicators is
inversely correlated to the quality of the courses: in fact, increasing career
relevance proves to be, ceteris paribus, more
expensive for online courses of mediocre quality. This allows us to deduce that
the information regarding the career relevance of online courses could constitute
real signals à la Spence.
The quality of teachers,
indicated by about 40% of those interviewed, would, on the other hand, be an
ambiguous indicator. How can you measure the quality of teachers? According to their teaching experience, their
academic achievements or other complex parameters? And once you have assessed the quality of
each individual teacher, how do you relate this to the quality of online
courses? In fact, online courses require
different teaching methods from traditional courses; they involve the use of multimedia teaching
materials, as well as the construction and implementation of a
computer-mediated communication space.
These skills are not part of the usual teaching experience.
The following three
indications provided by the students appear to be of a more concrete
nature: the level of interaction with
teachers (35.2% of those interviewed), the adequacy and clarity of the
materials (25.7%) and the degree of flexibility of the course (21.2%). As Arbaugh (2000) points out, these are factors which can have
a direct influence on online learning and the satisfaction level of the
students.
Concerning the
“interaction with teachers” issue, it should be said that there is an open
debate in literature regarding the different ways for measuring interaction in
on-line educational systems, and many solutions have been proposed [Anderson et al. (2001), Rourke et al. (2001)].
Nowadays, the technological
solutions for managing on-line education processes, such as LMS and LCMS,
include tools that can measure statistical data on specific elements of
interaction. Of the different proposals, the average response time of the
teachers to students’ questions and the number of discussion threads proposed
by the teacher for a course can be considered a measurement of the interaction
between teachers and students that could be collected automatically. It should
be noticed that the answer to the questionnaire provided by the students
concerns only interaction with teachers, and shows that students are not aware
of the importance of interaction with other students.
Regarding the “adequacy
and clarity of materials”, some well known measurement criteria are: the full
availability of on-line material, the necessity to buy printed material in
addition to that available on line, the organization of learning paths, the
presence of references to external sites in order to deepen the explanations,
and so on.
Besides, the students interviewed
highlighted the importance of flexibility, considered as the opportunity of
accessing the educational material and interacting with teachers without time
constraints. One of the measurement criteria for flexibility is the frequency
of synchronous sessions. Actually, this parameter is not an absolute value of
quality, but it is important in the personal evaluation of the adequacy of the
organization of the course with respect to the single student needs. The same
number of weekly interactions could be, for example, too many for some students
and too few for others.
We think that a very
useful tool to measure the level of the courses respect to indicators discussed
above, is an anonymous
questionnaire administered to students from previous years, because their
estimation is related to the effectiveness of learning. In this way, the obtained data could enable the best online universities to send signals of
quality. It is interesting to observe, also in this case, how the cost of
producing quality signals concerning student satisfaction is inversely
correlated to the quality of online courses.
The reflections presented in
this work lead us to emphasize the existence of an adverse selection problem in
the new market of university online learning. In fact, since the evaluation of
new courses is costly and the enrollment
fees cannot be used as a signal of quality, students perceive a greater risk in
choosing on line courses; consequently they could be encouraged to enroll on traditional
courses.
In response to these concerns, the
Italian Ministry for Universities and Research issued a decree on
The considerable limits of this
decree lie, however, in the fact that it is merely an administrative act. In
fact, it does not provide for a continuing evaluation procedure nor establish
methods of comparison or a system for publicizing the evaluation results. This regulatory instrument seems, therefore, to be unable
to send out signals regarding quality, in order to prevent problems of adverse
selection.
In
order to avoid this risk, we propose the use of signals à
la Spence.
Our survey has enabled us to
underline certain types of information which students in
Further research is necessary,
however, in order to test the validity of these signals empirically within the
market.
Akerlof G. (1970), “The Market for Lemons: Quality
Uncertainty and the Market Mechanism”, Quarterly
Journal of Economics, 84, 485-500.
Arbaugh J. B. (2000), “Virtual classroom characteristics and student
satisfaction in Internet-based MBA courses”, Journal of Management Education, 24, 32-54.
Anderson, T., Rourke, L., Garrison, D.R. &
Archer, W. (2001). Assessing
teaching presence in a computer conferencing context. Journal of Asynchronous Learning Networks,
5,2.
Bedard K. (2001), “Human Capital versus Signaling Models: University Access and High School
Dropouts”, Journal of Political Economy,109, 749-775.
Bezmen T. and Depken
Dean
L. (1994), “Telecomputer communication: The model for
effective distance learning”, ED Journal,
8(12), J-1-J-9.
Dey E.L., Astin A.W. and
Korn W.S. (1991), “The American freshman: Twenty-five years trends, 1966-90”, Higher
Education Research Institute, Graduate School of Educatione,
Fallah M. H. and Ubell R.
(2000), “Blind scores in a graduate test: conventional compared with web-based
outcomes”, ALN Magazine, 4(2).
Farber
H. and Gibbons R. (1996), “Learning and Wage Dynamics”, Quarterly Journal of Economics, 111, 1007-1047.
Fresen J. (2002), “Quality in Web-supported
learning”, Educational Technology,
42, 28-32.
Harasim L. (1993), Global
networks: computers and communication, MIT Press,
Harasim L. (2000), “Shift happens Online education as
a new paradigm in learning”, The internet and
Higher Education, 3, 41-61.
Hsing Y. and Chang H.S. (1996), “Testing increasing sensitivity of enrolment
at private institutions to tuition and other costs”, American Economist, 40, 40-45.
John
K. and Williams J. (1985), “Dividends, Dilution and Taxes: A Signalling
Equilibrium”, Journal of Finance, 40,
1053-69.
Lang
K. and Kropp D. (1986), “Human Capital versus
Sorting: The Effects of Compulsory Attendance Laws”, Quarterly Journal of Economics,101,
609-624.
McGorry S. Y. (2003), “Measuring quality in on line programs”, The Internet and High Education, 6,
159-177.
Milgrom P. and Roberts J. (1986), “Price and Advertising Signals of Product
Quality”, Journal of Political Economy,
94, 795-81.
Myers S. and Majluf N. (1984), “Corporate Financing
and Investment Decisions when Firms Have Information that Investors Do Not
Have”, Journal of Financial Economics,
13, 187-221.
Navarro
P. and Shoemaker J. (1999), “The power of cyberlearning:
An empirical test”, Journal of Computing
in Higher Education, 11, 33.
Nelson
P. (1974), “Advertising as Information”, Journal
of Political Economy, 82, 729-54.
Phipps
R. A. and Merisotis J. P. (1999), What’s the Difference? A Review of Contemporary Research on the Effectiveness
of Distance Learning in Higher Education, DC: American Federation of
Teachers and National Education Association, Washington.
Pond W. K. (2002), “Twenty-first century education and training
Implications for quality assurance”, The
Internet and High Education, 4, 185-192.
Riley
J. (1975), “Competitive Signalling”, Journal
of Economic Theory, 10, 174-186.
Riley
J. (1979), “Testing the Educational Screening Hypothesis”, Journal of Political Economy, 87, 227-52.
Roberts S. K. (1999), “A survey of Accrediting Agency Standards and
Guidelines for Distance Education”, Theological
Education, volume 36.
Rothschild
M. and J. Stiglitz (1976), “Equilibrium in
Competitive Insurance Markets: An Essay on the Economics of Imperfect
Information”, Quarterly Journal of
Economics, 95, 629-649.
Rovai A. P. (2003), “A practical framework for evaluating on line distance
education programs”, The Internet and
Higher Education, 6, 109-124.
Rumble
G. (2001), “The costs and costing of networked learning”, Journal of Asynchronous Learning Networks, 5.
Rourke, L.,Anderson,
T.,., Garrison, D.R. & Archer, W. (2001). Assessing
social presence in asynchronous text-based computer conferencing. Journal of Distance Education, 14, 21,
50-71.
Sener J. and Stover M. (2000), “Integrating ALN into
an independent study distance education program: NVCC case studies”, Journal of Asynchronous Learning Networks,
4.
Smeaton A., Keogh G. (1999), “An analysis of the use of virtual delivery of
undergraduate lectures”, Computers and
Education, 32, 83-94.
Spence
M. (1973), “Job Market Signalling”, Quarterly
Journal of Economics, 87, 355-374.
Spence M. (1974), Market Signaling,
Wade W. (1999),
“Assessment in distance learning”, T.H.E.
Journal, 27, 94-100.
Join Academic Exchange Quarterly editorial staff