ETB-network will have participants from local, national and European wide repositories of educational material in different subject areas. ETB is offering users the possibility to search multilingual, classified, high quality educational material within all the repositories in the network.
I. ETB Documentation System
as a Part of Quality
ETB aims to represent
a European documentation system, where the actors use documentary standards (see
the whole ETB
datamodel here), selection criteria and quality assurance procedures which
are common to the system and established in advance.
Submission of web-based learning resources is based on the policy that all the records have to pass a local quality
assessment before being submitted to the network. This is in the responsibility
of each member repository and highly emphasised by the ETB when member is joining
the network. This allows ETB to state that a repository becoming a member of
the ETB-network already indicates that resources are of a good quality.
repositories shall make their quality assurance policy and quality assessment
procedures available for everybody on the Web. This allows users to evaluate
weather the quality assessment policy of learning resources meets their demands whilst
assure them about the material found through the ETB. These two requirements
create the base principal for 'Trusted sources' in the ETB-network.
five questions will be asked from each repository when joining the ETB-network.
For the reasons of unified responses, a controlled list has been prepared. This
information will be provided in the metadata (namely in Collection
Level Descriptors) describing the learning resources (collection) the repository
makes available for the network. The lable is named "Quality
Selection Policy", namespace ETB CLD, defining the collection policy associated with the collection.
Do you have a Selection Policy for the material included in your collections?
Does your selection policy follow a...:
1. formal procedure
2. informal procedure
3. random procedure
If you answered “No”
and you have an informal and random procedure,
it is not necessary to answer
the following questions.
You shall only describe the
procedure you follow for the selection of the material to be included in your
Does your Selection Policy include a Quality Statement?
- national criteria (please specify)
-other (please specify)
Who is responsible for the Quality policy:
- teachers' association
- governmental agency
- regional or local school
- pedagogical organisation
- national association
- content editors/ subject
- ETB recommendations
- ISO 9000
- other (please specify)
4. Who is doing the actual
selection of resources to be included in the collection:
- teacher/ teachers' association
- content editors/subject specialists
- national association
- governmental agency
- robot harvester
- other (please specify)
How often is the material reviewed?
(automatic) link checker
b. intellectual review
Quality Selection Policy: "No, random procedure"
Quality Selection Policy: "Yes, formal procedure, URL:http://www.quality.org,
pedagogical network, teachers' association, b-monthly"
II. TWO SETS OF RECOMMENDATIONS
Quality of resources
is a key factor in ETB. There are recommendations on quality assurance procedures
provided by the workpackage dealing with the Quality Assurance of the project.
These recommendations are aiming at two different groups: there are guidelines
on quality processes and recommendations for (1) teachers and (2) repository
The First Set
of Recommendations aims at teachers working in the educational settings
using web-based material. On one hand the recommendations deal with the aspects
educators must keep in mind while creating the content of their own material.
In other words these recommendations are focused on the process of developing
the resources. The recommendations are also to consider when a teacher or a
student wants to submit material to the ETB-network through the EUN native repository.
On the other
hand these recommendations are good to keep in mind when searching and
selecting already existing material or products from the Web. (Note the
differentiation between quality of development process and product itself,
this is what CEN/ISSS Learning Technology Workshop - Quality Assessment
Group is looking into. )
on the developing process of material are much in agreement with GEM's
indications and with DESIRE.
This list will be further developed in the future.
Accuracy: the resource must be reliable, valid and produced
by a Trusted source; information should be impartially presented; resource
must not contain biases, mistakes or omissions.
· Appropriateness: the resource should contain information
for the intended learners' level; the resource should use an appropriate
and suitable vocabulary, language or concepts, avoid mistakes or stereotyping.
Clarity: information should provide a clear tie between the
purpose (goals, objectives) and the content and procedures suggested.
Correlation should be comprehensive and obvious. Redundancy is usually
unwelcome and isolated activities without a relationship are superfluous.
Completeness: the resource should be complete, i.e., offer
all essential information and elements, as well as inclusion of such
components as self-contained activities, materials required, prerequisites,
information for obtaining related resources, assessment criteria,
links to quality indicators and standards. The resource should offer
wide and in-depth information related to the topic.
Motivation: the resource should achieve the active engagement
of the learner and be interesting, innovative and appealing, build
on prior knowledge and skills, and promote relevant action on the
part of the learner.
the resource should be easy to use and logically sequenced, with each segment
of the resource related to other segments. It should flow in an orderly
manner, using organising tools (i.e. headings, a map, etc.) and avoiding
use of unrelated elements that are potentially ineffective or overpowering;
it should provide for references, bibliographies and other supporting materials
available for the users.
Set of Recommendations is for repository owners, i.e. managers, curators
and/or administrators of an educational server . This group is strictly responsible
for submitting and administrating the metadata records circulating on the network.
The recommendations will deal with issues that matter for the network; i.e.
what kind of material is wanted for the ETB-network.
repository carries out documentation processing (source selection, documentary
processing, information processing and diffusion) in relation to its target
users whose needs have been analysed. However, it is important to emphasise
that repositories are not supposed to submit all of the records of resources
and collections to the network; quality, European level and good practice of
resources is a priority issue of ETB rather than sheer numbers.
check list have been established to guide the selection of material to the ETB-network
highlighting the ETB-quality factors.
Quality Selection; ETB recommendations as regards
the lifecycle and organisation of the resources
boxes: | Yes | No | Not considered
Information on Collection/ Repository
there stated criteria for inclusion of a new learning resource to the collection?
Has the learning resources been filtered, e.g. peer-reviewed?
Is the subject matter of sites linked to the resources important?
Does the resource have an added
Information provider/ Source
Is the resource attributed to a reputable author or organization, and
is that information stated?
Are commercial resources acceptable?
Are private resources acceptable?
the information appear to be well researched, e.g. are references and contact
Is the content of the resource verifiable, e.g. can you cross check the
Is there information that has a limited period of use?
Is information is current and up-to-date?
Are all the pages dated with the last revision date?
you able to check the accuracy of the information?
Are resources that contain advertising acceptable?
Is biased information e.g.
are opinions and ideologies acceptable?
Are there subjects which could be censored, e.g. for ethical
reasons (please specify)?
Does the resource contain substantive
Could the resource be used in another European language/cultural/learning
Composition and Organisation
the information clearly organised, i.e.
arranged logically and consistently?
Is the resource
organised by the needs of intended users' level?
Are key words
given to indicate the information content?
Is it important that the grammar
and spelling is accurate?
Evaluation of the Medium
Are you considering the resources' ease of navigation/browsing/searching?
Are you considering the resources'
provision of user support, e.g. online help, documentation?
Are you considering the resources
appearance for different users, e.g. valid HTML, older browsers?
III. Factors of Quality
ETB judges four
(4) factors valuable for quality assurance of the learning resources that are
submitted to the network. These factors are Trusted Source (as explained
above), Usefulness, Attractiveness and Satisfaction. These
four factors of quality of material are top level terms containing several sub-factors,
which are, in their turn, again linked back to the top-level factors (see Figure
1). These factors don't pretend to be exhaustive standards of quality, but more
acting as methodological guidelines and recommendations for achieving better
for example, is linked to the 'Information quality' and 'Learning capability'
(see Figure 1), what kind of skills learner gets from using the material. Since
each learning situation is different it is impossible to define one set of quality
standards that will fulfill all the needs of end users. This is why the metadata
records of learning material, i.e. documentation system of resources (see ETB
datamodel) , attempt to describe the frame and setting as well as possible.
This is where
the factor Usefulness is linked to the metadata issue; how well and detailed
the metadata records of learning material are described, structured and if they
are consistent. Naturally, if all the elements of the ETB datamodel are used
datamodel), the information of the material is more complete and more informative,
hence of higher quality.
This emphasises the importance of use of factor Satisfaction too. ETB
believes that if the end user is provided with enough information of the resource
s/he will be able to decide weather a piece of material in concern meets the
standards s/he has in mind. Thus, the overall satisfaction to the material,
and also to the documentation system, will be enhanced.
The factor for
Attractiveness of resources might sound pretentious and outplace in the
list of factors judged valuable. But the playfulness and attractiveness is not
to neglect, after all it is an important asset of using ICT in the educational
setting. ETB promotes explorative, innovative and enjoyable learning material
that can make educational setting appealing and attractive to its audience.
The two sets of
recommendations introduced in this paper are described in a very general way.
They will be further developed and completed in the document D 3.2 Recommendation
on Quality Assurance and Selection of resources for the ETB network, which
will come out in the beginning of 2002.
More on the ETB quality
issue by Maria Daniela Nenci, INDIRE (exBDP) here
||Vincenza Benigno, Riina Vuorikari + WP 3 partners
||Thursday, 27 Sep 2001
||Tuesday, 9 Oct 2001
||etb, quality, standards, recommendations