News | about | Search Language: DE | EN | FR
 Forum on Quality - Week III: Recommendation on Quality Assurance and Selection of resources for the ETB-network

European Treasury Browser
ETB Handbook
Multilinguality and Thesaurus
Outputs and Documents
•  Controlled Vocabulary: Quality Selection Policy
•  Forum on Quality - Week I
•  Forum on Quality - Week I - Archive
•  Forum on Quality - Week II
•  Forum on Quality - Week II - Archive
•  Forum on Quality - Week III -Archive
•  Forum on Quality - Week III -Summary
•  Forum on Quality - Week III: Recommendation on Quality Assurance and Selection of resources for the ETB-network
•  Rationale behind the quality management
•  Recommendations on quality criteria
Search this area

Advanced Search

Print this page!
Tell a friend!
To get a reminder when this page is updated, please enter your email address here:

ETB-network will have participants from local, national and European wide repositories of educational material in different subject areas. ETB is offering users the possibility to search multilingual, classified, high quality educational material within all the repositories in the network.

I. ETB Documentation System as a Part of Quality

ETB aims to represent a European documentation system, where the actors use documentary standards (see the whole ETB datamodel here), selection criteria and quality assurance procedures which are common to the system and established in advance.

Submission of web-based learning resources is based on the policy that all the records have to pass a local quality assessment before being submitted to the network. This is in the responsibility of each member repository and highly emphasised by the ETB when member is joining the network. This allows ETB to state that a repository becoming a member of the ETB-network already indicates that resources are of a good quality.

Secondly, the repositories shall make their quality assurance policy and quality assessment procedures available for everybody on the Web. This allows users to evaluate weather the quality assessment policy of learning resources meets their demands whilst assure them about the material found through the ETB. These two requirements create the base principal for 'Trusted sources' in the ETB-network.

The following five questions will be asked from each repository when joining the ETB-network. For the reasons of unified responses, a controlled list has been prepared. This information will be provided in the metadata (namely in Collection Level Descriptors) describing the learning resources (collection) the repository makes available for the network. The lable is named "Quality Selection Policy", namespace ETB CLD, defining the collection policy associated with the collection.

1. Do you have a Selection Policy for the material included in your collections?

Yes              No

2. Does your selection policy follow a...:

1. formal  procedure
2. informal  procedure
3. random  procedure

If  you answered “No” and you have an informal and random procedure,
it is not necessary to answer the following questions.
You shall only describe the procedure you follow for the selection of the material to be included in your collection/s.

3.1 Does your Selection Policy include a Quality Statement?

- national criteria (please specify)
-ETB recommendations
-other (please specify)

3.2  Who is responsible for the Quality policy:

- teachers' association
- governmental agency
- regional or local school
- pedagogical organisation /network
- national association
- content editors/ subject specialists
- ETB recommendations
- ISO 9000
- other (please specify)

4. Who is doing the actual selection of resources to be included in the collection:

- teacher/ teachers' association
- content editors/subject specialists
- librarian/-s
- webmaster
- national association
- governmental agency
- robot harvester
- other (please specify)

5. How often is the material reviewed?

a. (automatic) link checker


b. intellectual review


Example: Quality Selection Policy: "No, random procedure"
Quality Selection Policy: "Yes, formal procedure, URL:, pedagogical network, teachers' association, b-monthly"


Quality of resources is a key factor in ETB. There are recommendations on quality assurance procedures provided by the workpackage dealing with the Quality Assurance of the project. These recommendations are aiming at two different groups: there are guidelines on quality processes and recommendations for (1) teachers and (2) repository managers.

The First Set of Recommendations aims at teachers working in the educational settings using web-based material. On one hand the recommendations deal with the aspects educators must keep in mind while creating the content of their own material. In other words these recommendations are focused on the process of developing the resources. The recommendations are also to consider when a teacher or a student wants to submit material to the ETB-network through the EUN native repository.

On the other hand these recommendations are good to keep in mind when searching and selecting already existing material or products from the Web. (Note the differentiation between quality of development process and product itself, this is what CEN/ISSS Learning Technology Workshop - Quality Assessment Group is looking into. )

The recommendations on the developing process of material are much in agreement with GEM's indications and with DESIRE. This list will be further developed in the future.

· Accuracy: the resource must be reliable, valid and produced by a Trusted source; information should be impartially presented; resource must not contain biases, mistakes or omissions.

· Appropriateness: the resource should contain information for the intended learners' level; the resource should use an appropriate and suitable vocabulary, language or concepts, avoid mistakes or stereotyping.

· Clarity: information should provide a clear tie between the purpose (goals, objectives) and the content and procedures suggested. Correlation should be comprehensive and obvious. Redundancy is usually unwelcome and isolated activities without a relationship are superfluous.

· Completeness: the resource should be complete, i.e., offer all essential information and elements, as well as inclusion of such components as self-contained activities, materials required, prerequisites, information for obtaining related resources, assessment criteria, links to quality indicators and standards. The resource should offer wide and in-depth information related to the topic.

· Motivation: the resource should achieve the active engagement of the learner and be interesting, innovative and appealing, build on prior knowledge and skills, and promote relevant action on the part of the learner.

· Organisation: the resource should be easy to use and logically sequenced, with each segment of the resource related to other segments. It should flow in an orderly manner, using organising tools (i.e. headings, a map, etc.) and avoiding use of unrelated elements that are potentially ineffective or overpowering; it should provide for references, bibliographies and other supporting materials available for the users.

The Second Set of Recommendations is for repository owners, i.e. managers, curators and/or administrators of an educational server . This group is strictly responsible for submitting and administrating the metadata records circulating on the network. The recommendations will deal with issues that matter for the network; i.e. what kind of material is wanted for the ETB-network.

Every joining repository carries out documentation processing (source selection, documentary processing, information processing and diffusion) in relation to its target users whose needs have been analysed. However, it is important to emphasise that repositories are not supposed to submit all of the records of resources and collections to the network; quality, European level and good practice of resources is a priority issue of ETB rather than sheer numbers.

The following check list have been established to guide the selection of material to the ETB-network highlighting the ETB-quality factors.

Quality Selection; ETB recommendations as regards the lifecycle and organisation of the resources

Tick boxes:       | Yes |  No | Not considered |

a) Information on Collection/ Repository

Are there stated criteria for inclusion of a new learning resource to the collection?
Has the learning resources been filtered, e.g. peer-reviewed?
Is the subject matter of sites linked to the resources important?
Does the resource have an added European value?

b) Information provider/ Source

Is the resource attributed to a reputable author or organization, and is that information stated?
Are commercial resources acceptable?
Are private resources acceptable?

c) Validity

Does the information appear to be well researched, e.g. are references and contact data given?
Is the content of the resource verifiable, e.g. can you cross check the information?
Is there information that has a limited period of use?
Is information is current and up-to-date?
Are all the pages dated with the last revision date?

Are you able to check the accuracy of the information?

d) Information Coverage

Are resources that contain advertising acceptable?
Is biased information e.g. are opinions and ideologies acceptable?
Are there subjects which could be censored, e.g. for ethical reasons (please specify)?
Does the resource contain substantive information?
Could the resource be used in another European language/cultural/learning environment?

e) Composition and Organisation

Is the information clearly organised, i.e. arranged logically and consistently?
Is the resource organised by the needs of intended users' level?
Are key words given to indicate the information content?
Is it important that the grammar and spelling is accurate?

f) Evaluation of the Medium

Are you considering the resources' ease of navigation/browsing/searching?
Are you considering the resources' provision of user support, e.g. online help, documentation?
Are you considering the resources appearance for different users, e.g. valid HTML, older browsers?

III. Factors of Quality

ETB judges four (4) factors valuable for quality assurance of the learning resources that are submitted to the network. These factors are Trusted Source (as explained above), Usefulness, Attractiveness and Satisfaction. These four factors of quality of material are top level terms containing several sub-factors, which are, in their turn, again linked back to the top-level factors (see Figure 1). These factors don't pretend to be exhaustive standards of quality, but more acting as methodological guidelines and recommendations for achieving better quality.

Usefulness, for example, is linked to the 'Information quality' and 'Learning capability' (see Figure 1), what kind of skills learner gets from using the material. Since each learning situation is different it is impossible to define one set of quality standards that will fulfill all the needs of end users. This is why the metadata records of learning material, i.e. documentation system of resources (see ETB datamodel) , attempt to describe the frame and setting as well as possible.

This is where the factor Usefulness is linked to the metadata issue; how well and detailed the metadata records of learning material are described, structured and if they are consistent. Naturally, if all the elements of the ETB datamodel are used (see ETB datamodel), the information of the material is more complete and more informative, hence of higher quality.

This emphasises the importance of use of factor Satisfaction too. ETB believes that if the end user is provided with enough information of the resource s/he will be able to decide weather a piece of material in concern meets the standards s/he has in mind. Thus, the overall satisfaction to the material, and also to the documentation system, will be enhanced.

The factor for Attractiveness of resources might sound pretentious and outplace in the list of factors judged valuable. But the playfulness and attractiveness is not to neglect, after all it is an important asset of using ICT in the educational setting. ETB promotes explorative, innovative and enjoyable learning material that can make educational setting appealing and attractive to its audience.

The two sets of recommendations introduced in this paper are described in a very general way. They will be further developed and completed in the document D 3.2 Recommendation on Quality Assurance and Selection of resources for the ETB network, which will come out in the beginning of 2002.

More on the ETB quality issue by Maria Daniela Nenci, INDIRE (exBDP) here
Author: Vincenza Benigno, Riina Vuorikari + WP 3 partners
Web Editor: Riina Vuorikari
Published: Thursday, 27 Sep 2001
Last changed: Tuesday, 9 Oct 2001
Keywords: etb, quality, standards, recommendations

•  Virtual Workshop on Quality: enter here with password