News | about | Search Language: DE | EN | FR
 
 Forum on Quality - Week II - Archive

European Treasury Browser
About
ETB Handbook
Interoperability
Multilinguality and Thesaurus
News
Outputs and Documents
Quality
•  Controlled Vocabulary: Quality Selection Policy
•  Forum on Quality - Week I
•  Forum on Quality - Week I - Archive
•  Forum on Quality - Week II
•  Forum on Quality - Week II - Archive
•  Forum on Quality - Week III -Archive
•  Forum on Quality - Week III -Summary
•  Forum on Quality - Week III: Recommendation on Quality Assurance and Selection of resources for the ETB-network
•  Rationale behind the quality management
•  Recommendations on quality criteria
Search this area

Advanced Search

Print this page!
Tell a friend!
To get a reminder when this page is updated, please enter your email address here:



Find here all the comments that were given during the second week of the Virtual Forum on Quality.

ETB Virtual Workshop on Quality of Web-based Learning Resources
WEEK II: Forum on Quality Assurance Approaches
Moderator: Jan Pawlowski



Title:  Welcome to Week2: Main issues
Name:   Jan Pawlowski

Message: Dear Members of the quality workshop, I would like to welcome you to the second week of the ETB quality workshop. Many thanks to EUN to provide this opportunity to discuss this controversial topic. I am looking forward to having a fruitful discussion in the next few days.

The goal of this workshop is to identify trends in quality assurance related to e-learning and to determine appropriate standards. The discussion should foucs on different perspectives on quality approaches, including learners, managers, teachers, authors, and content providers in order to find a common understanding of quality needs and requirements.

In a study from 1998 for the Council for Higher Education Accreditation (CHEA) the following questions were identified as the main issues for research in quality assurance for distance education, particularly for e-learning:

What are the existing or emerging strategies for quality assurance of technology-mediated distance learning programs?
How can the educational outcomes and experiences offered by these programs and institutions be measured?
What specific policies and procedures are required for providing quality assurance of technology-mediated distance learning?

This virtual forum should try to answer some of these questions and give the opportunity to share experiences in this field.

But first of all, some background on quality assurance approaches.
Quality standards in general can be divided into process and product oriented standards.
Process oriented approaches focus on the transparency and consistency of processes leading to a product or service. An example is ISO 9000:2000, ensuring the consisitency of generic processes. Another example is the use of Lifecycle models and procedural models.

Product oriented standards ensure the quality of the result of these processes. Usually a the quality of a product is measured using certain criteria, checklists, or similar mechanisms.
Additionaly, some approaches define competencies for actors. These approaches assume that approapriately skilled actors lead to results of appropriate quality.

The following questions should be a guideline for the discussion:
Can there be a common quality standard for learning environments in the future?
This topic is highly related to the first part of the discussion. Is it possible to define a generic e-learning standard, independent of the domain or a certain community? Will there be specific standards set by communities? What are the requirements for such a standard?

Secondly,which are the emerging strategies and related procedures for quality assurance in the field of e-learning?
Should organizations focus on product-oriented, process-oriented, or competency-oriented standards? Which are best-practice cases of quality assurance for e-learning?

Thirdly, which areas should be covered by an E-Learning quality standard?
Should an e-learning quality standard cover all areas of learning, including adminstration, delivery, security, ...? Should a standard focus on the appropriate use of technologies? Should didactic concepts be considered?

Background information on the issues will be given in a separate document.


Title:  Re:which areas should be covered by an E-Learning quality standard?
Name:   abeckers

Message: I do think there is less need for an over-all (TQM) quality assurance. For me it is more practical that there are guidelines or strong suggestions that an organization wanting to 'produce' material for e-learning works according to international renown standards.

For example ISO 9001, EFQM, Malcolm Baldridge, Deming etc.

By using guidelines instead of insisting on using standards creates a low threshold for creators. On the other hand this leads towards an extra step: control. Somebody has to be in charge to check wether the entries are of quality.

This approach may have some serious implications on the internationalization of learning material.

But even with standards, we have to create a universe where the audience, teachers and students conform to the use of those standards. Who will check this? I mean opening a door without a doorkeeper is useless.


Title:  Re:Re:which areas should be covered by an E-Learning quality standard?
Name:   Jan Pawlowski

Message: Guidelines might be a pracmatic approach to quality assurance - since there are thousands of proproetary guidelines out there, these will have to be generalized and put into some form of "meta-guidelines" leading towards a standard - this also means that a standardization body would be the doorkeeper for "accrediting" guidelines...
But would this be an effective process? Could it influence innovation and creativity?


Title:  Re:Re:Re:which areas should be covered by an E-Learning quality standard?
Name:   abeckers

Message: Thinking about it, my suggestion is just a shift of responsibility.

My problem is, how to paproach this: Using a standard? It seems to me like Deming says in his book "out of the crisis": the combined effort does not lead to quality. If everybody does his/her best without knowing the objective/goal the road we are heading too much energy is wasted. Dispersed in all directions.

From this point of view a better approach would be to describe the framework of our intent and what the objectives are. I visted the Danish Poseidon project (one of the partners of EUN and on the link page).They say: "Generation of practical experience often has a tendency, through the power of example, to gain greater influence and affect developments more effectively than long analyses and theoretical reports".

Does this ring a bel? CEN/ISS-LT liasons like prometeus are following more or less the same route.

I do think that we should work in a 'protected' environment where the members/attendees conform to rules. Thus to create e-learning standards,we might start a poll asking what is the way of working. Specific on what grounds do you decide/measure the quality of a contribution of for instance a class on getting a drivers license?



Message: Hi,
Name:   Carmen

I also agree that "Generation of practical experience...gain greater influence".

Of course quality standards are needed in eLearning, but I think that sometimes too much theorizing induces a kind of fear. What I want to say is that quality can be obtained only by practicing.

How? Having:

-quality training programs for instructors, instructional designers, web designers, etc.;
-mentoring programs at the beginning of one's activity;
-peer-to-peer collaborations;
-online visitors in online courses;
-permanent evaluations, feedback from participants, but also from instructors and experts;
-sites with best practices resources;
-online learning communities.

eLearning means continuos learning not only for the participants/students, but also for instructors themselves.

It is known that in an online course, even if the instructor is not the best, the gain in knowledge for everyone is important, because of the extraordinary interactions, feedback.

And... I don't want to be bad, but, in this workshop about Quality, the virtual space, indeed with a lot of nifty facilities and appealing, is a little too slow and weak in the most important part, the Forum ( no HTML, no possibility of reedit a message, nopossibility of reading a whole thread... ). Sorry if I am too sharp...

Carmen
 




Title:  Towards a quality standard?
Name:   Jan Pawlowski

Message: Ever since buzzwords like Total Quality Management and ISO 9000 became popular in the business community, this topic has been discussed from controversely. Particularly in the field of education, quality assurance mechanism are still being developed.

Currently, there is no accepted quality standard in the field of e-learning. Different organizations are working on different approaches and it seems, as if the chances for an agreed standard are quite low.

So, will a standard help to improve e-learning applications? This question can be answered from various perspectives:

The business perspective would appreciate a standard as a marketing tool.

Authors might be affected that a standard could limit their creativity.

How will affect a standard the main actor, the learner? Will a standard help to improve transparency? Can a standards improve the sucess of a learning process? Or will a standard be "abused" for marketing purposes and limit creativity?





Title:  Goals and methodology
Name:   Jan Pawlowski

Message: As an output of the first few points I would like to open this new thread in order to increase the readibility.

First of all, the objectives of a quality standards have to be well defined. What could the objectives of a quality standard be?

* transparency: the learner will be informed what to expect from a learning environment. This would require a detailed description of a learning environment (product standard) or of the development process (process standard)

* certification and accreditation: the learner is ensured that he will receive some form of certificate after a course.

* consistency: the learner knows that learning environments are developed in a controlled way.

* interoperability: learning environments can be used on different platforms and systems.

* .... (open for discussion)





Title:  Characteristics of 'quality learning environments' (reposted, by Carmen)
Name:   Carmen

Message: I also agree that "Generation of practical experience...gain greater influence".

Of course quality standards are needed in eLearning, but I think that sometimes too much theorizing induces a kind of fear. What I want to say is that quality can be obtained only by practicing.

How? Having:

-quality training programs for instructors, instructional designers, web designers, etc.;
-mentoring programs at the beginning of one's activity;
-peer-to-peer collaborations;
-online visitors in online courses;
-permanent evaluations, feedback from participants, but also from instructors and experts;
-sites with best practices resources;
-online learning communities.

eLearning means continuos learning not only for the participants/students, but also for instructors themselves.

It is known that in an online course, even if the instructor is not the best, the gain in knowledge for everyone is important, because of the extraordinary interactions, feedback.


Title:  Re:Characteristics of 'quality learning environments' (reposted, by Carmen)
Name:   abeckers

Message: Re: a point of view.

Carmen I totally with your point of view except that quality only can be obtained by practicing. I agree that quality is merely a study of processes.

Ø True that you need to be started in order to be able to speak of processes.
Ø True that one needs proper and extensive training in the beginning during and continuously.
Ø True you need feedback from all parties involved.
Ø True you need ‘best practices’ for goal setting

All of this needs proper planning. (Theoretical but foremost by applied knowledge of related processes) ‘Design parameters’ are needed. If you do not know where you are ultimate heading is you will spend a lot of time and effort in various directions without coming near where you intended to come. It is more practical to start thinking about the rules of the game, about your public, about the creators.
Maybe the whole thing seems to be so difficult and complex because we do not know who is working on it. We have not met face to face like in a normal project. Our effort and enthusiasm has therefore to be stupendous. But I do think we all have the same intentions: delivering high quality class for pupils. Content in a form that interests and attracts students and gives them a knowledge boost that has never been experienced before. Do we assume that by using a fast developing medium that content and knowledge transfer will also be at the speed of light?

From your point of view as a teacher how do you include and define quality in your lessons? What are the characteristics, what to exclude but certainly to include to make that hour unforgettable? In my 30 years as pupil I can only recall a handful teachers that were outstanding. The memories I keep are personal ones meaning the way they intrigued me as a person. Few classes I can recall where ‘I have seen the light’.

Those ‘knowledge moments’ were the moments that I discovered through much pain, failing and relentless effort I was able to express and describe my problem in a way that the teachers support made the final connection. In this observation you can see some typical elements:
Ø Understand the issue/question
Ø Receive the basics to get started
Ø Try to solve the issue with the content received
Ø Research other sources to be of help
Ø Timeless support at the crucial moment

Are there any other things that you know of or recognize?
Do you think if we can address all elements they can be used as a guideline or even a standard of those who want to be part of this process?

Secondly, the methodology to develop such a standards should be defined:

* theoretical analysis & synthesis: This approach would be an analysis of existing standards, guidelines, and quality assurance approaches. The synthesis would be "meta-guidelines". As an example, the output of this approach could be which fiels should be covered by "approved" guidelines. Using this approach, different existing approaches could be included.
 

* practical approach: this approach has been discussed in the first week. As an example, communities of practice can determine a quality standards for a specific domain.
 

* mixed approaches: Mixture of theoretical & empirical approaches.

Still, there are a lot more parameters for the methodology to be discussed in the next few days...



 
Title:  Nothing more can be said
Name:   Carmen
Message: Hi,

Your postings are so thoughtful, that it is very difficult to find something more to say.

When I wrote that quality can be obtained only by practicing, I presume that the practice comes after a quality training, study, and, as it happens often, after a serious experience in traditional teaching.

I had the extraordinary opportunity to obtain the certification for online teaching after a training course at Maryland University USA, half of it being in the position of student, half online instructor, preparing an online course; all the time had the support of the facilitator, ideas from the other participants, access at their library, resources of best practice, and the encouragement that you become to be part of the faculty staff of a successful university with 70000 students and 5000 teachers. The online workshops organized monthly also continue to make you to learn.

Carmen


Title:  Re:Re:Characteristics of 'quality learning environments' (reposted, by Carmen)
Name:   Jan Pawlowski

Message: Bram, thank you for verbalizing these rare moments, I totally agree with you...I would like to add several points which made learning experiences valuable for me...

let me divide these parameters in a few categories and add some personal experiences:

* presentation:
- right media at the right moment: content is presented using text & graphics appropriately. Although several studies say that learning performance does not depend on the media used, I personally perfer just text based material with graphical & interactive supoort for complex concepts (I know this is far too unspecific, just expressing a general feeling)

* learner support
- adaptability: for me it is important to be in control of the media used. Secondly, the influence on the learning process was important for me. Speaking in a more abstract way, I enjoyed it when my experiences and preferences were valued by a teacher...
-adaptivity: systems should have the ability the adapt to the users needs. Personally, I do not like to be controlled by a system, although some features are quite practical.

*didactics:
- an appropriate mix of methods (e.g. self-controlled learning, collaborative learning, role-plays), depending on the content, other learners and available time.
- well-prepared teacher: having exciting front-presentations can be a good method, although it is not learner-centered at all...the crucial point is that a teacher is skilled for a certain method.
-motivation: this is the crucial point, how to make people excited...for me it ios important that a variety of methods is used, but motivation is still very personal parameter..
 

*communication:
-learner-tutor: tutors need to be reachable 24/7 (or at least at specific regular appointments).
- leaner-learner: I have only had fun with like-minded learners...
 

These are just some unstructured thoughts...





Message: Is this Babylon?Can Learning and Technology be inegrated into one new tool? Or are they like water and oil?
Name: by abeckers

Since some time I have been thinking what stops us from being productive in the field of quality assurance and quality management? Are we trying to match two contradictions: technology and learning? A method to simplify the issue is to separate them and identify the essentials linked to the subject being standardization of learning technology.

Technology is abstract, bits and bytes machine made, modular, relations/connections are easy to understand and measure. All basic principles can be applied without in-depth knowledge. Effectiveness, Flexibility, Quality and Time form the foundations. Technique can incorporate these elements without problem. True all processes and means have to be of the same level to run the technology in respectable order. Technology consists of parts that can be exchanged from one till the other if both parts are matching and of the same specifications. In short technology is kind of static.

Learning is far more tacit than technology. What do we know about learning? To keep it simple: learning is subjective, the same information is interpreted differently by different people, every person is different some how, a learning element can be used in a myriad of situations not easy to predict, cultural differentiations make it vary from place to place, learning processes are not fully understood and vary from person to person, learning is time dependable, recipients have to be conditioned to have the highest effectiveness rate etc.etc.

Is it therefore justified to search for the differences? Which practice/method is best? Is it not more appropriate to look for commonalities? In other words which elements of learning can be included in a framework of learning technologies and which cannot? The elements defined and excluded can they be of use in a standard if described in guidelines or examples of best practice?


Title:  Re:Can Learning and Technology be inegrated into one new tool? Or are they like water and oil?
Name: Dippe
Message: We all have different notions of what learning really is. Whether something is a piece of technology or not is much easier to define.

Basically learning is a lasting change of behaviour. Learning is not a tool, it's a continuous process which cannot be stopped, at least not until we as human beings cease to exist.

So at least from my point of view the question in the title is not especially well formulated.

Technology may be thought of as a set of different tools that we mainly use as a complement and enhancer of what we're less able to do without them (and sometimes not able at all to do without them).

The idea of efficient learning is confused with efficient training which one usually means. The latter is something else than learning.

So what's needed?
A common language would be a good starting point. Defining what we want to achieve and what problems we want to solve is probably good in the beginning too. We're then somewhat ready to discuss the means and how they might be used to help us.

Starting with tehnology and asking "which elements of learning can be included in a framework of learning technologies and which cannot?" is justifying technology for technology's sake. And that discussion doesn't have anything with learning to do.

I hope I haven't offended anybody reading this message. I only wanted to add my point of view.
Cheers
Gunther


Title:  Re:Can Learning and Technology be inegrated into one new tool? Or are they like water and oil? (reposted; originally by abeckers)
Name:   Jan Pawlowski

Message: "Designing learning environments depends on an adequate, wholistic combination of pedagogical, technological, and domain-specific know-how" (cited from a definition of learning processes).

In my opinion, it is very hard to distinguish between learning and technology and limit the scope to one of these areas. As an example, the IEEE Learning Technology Standards Committee limits its scope to the speicifcation of standards for the technology perspective. Many approaches also define technology-oriented architectures for learning environments. These approaches might lead to interoperable, portable systems. However, these factors are not the only factors, determining the "quality" of a learning environment.

A quality measure must definetely be a multi-dimensional (technology, pedagogy, content), multi-perspective (authors, teachers, learners, ...) measure.

Anyhow, looking at different components and identifying best-practice approaches for each of these components might be a feasible approach.

Which areas of "learning processes" could be included in such an approach. Which are the components we have to look at? Are there architectures helping us to identify relevant components?


Title:  Re:Re:Can Learning and Technology be inegrated into one new tool? Or are they like water and oil? (reposted; originally by abeckers)
Name:   avsoeiro
Message: Although at present we can view educational technology as static, it is unlikely
that we will be able to do so for long. Complex learning environments
involving sophisticated simulations and complex adaptive systems will
become much more common.  It is likely that at least some of our focus
will be related to the idea of augmentation first proposed by Doug Engelbart.
What the learner knows will be augmented by what the system knows producing
a more "knowledgable" learner-system entity.  This process spirals up
to inform the environment and simultaneously make it more personal. Any
evaluation of such a system will need to be highly context sensitive.  Moreover,
analyzing and standardizing evaluation of current technologies may be
premature, since we certainly haven´t exhausted our inventive possibilities. If our
evaluation procedures do not account for clever pedagogical designs, for example, they
may have a limiting effect on creativity.

Title:  Re:Re:Re:Can Learning and Technology be inegrated into one new tool? Or are they like water and oil? (reposted; originally by abeckers)
Name:   Jan Pawlowski

Message: I do not think that guidelines for pedagogy would effect the creativity - as long as they are not prescriptive. It will not be feasible to even prescribe e.g. a certain level of interactivity (there are examples of "good" learning environments without interaction tools), so this is as you mentioned, highly context sensitive.

So what would be the options for a non-prescriptive standards? Would it help to just describe the pedagogical method, so that the learner is informed (e.g. describing the interaction strategy, adativity, evaluation methods, presentation formats...)? Or should just meta-guidelines be described, such as: "In an educational organization, a guideline for pedagogical design has to be in place, supporting authors in their design process."?




 

Information:
Author: Riina Vuorikari
Web Editor: Riina Vuorikari
Published: Tuesday, 9 Oct 2001
Last changed: Tuesday, 9 Oct 2001
Keywords: etb, quality-issues, workshop, archives