Going back to the past and recalling the specific atmosphere of those years, what were the challenges that you wanted to address through the eLearning programme?
The overall objective of the eLearning programme was to "support and develop further the effective use of ICT in European education and training systems", which should constitute a crucial factor of "their adaptation to the needs of the knowledge society in a lifelong learning context". Under this framework, the programme addressed essentially the contribution of ICT in lifelong learning through the promotion of digital literacy, in particular for those with poor or no access to the technologies, the promotion of virtual mobility in higher education, the support to a Europe wide school networking scheme, including the training of teachers, and the dissemination, promotion and transfer of good and innovative practices of ICT use in education and training.
As underlying dimensions we can mention the contribution of ICT to strengthen social cohesion, to enhance the European dimension in education and to support the development of innovative teaching methods.
What do you think was particularly successful about the programme?
Although the programme was in general quite successful and cost-effective, I would put a very special emphasis on eTwinning, which is still a major ongoing European initiative in the field of school networking, supporting the exchange of innovative teaching methods and contributing to students' virtual mobility and teachers' professional development.The programme also successfully enhanced the European dimension in education and promoted co-operation between stakeholders in e-learning, for example in the fields of quality and open resources.
What are the most important lessons learnt?
The design and successful implementation of eTwinning, including the effectiveness of its funding model, demonstrates the potential of ICT to support innovative learning experiences, which would not be possible otherwise. It also shows the importance of integrating ICT in a coherent pedagogical project addressing clear needs and with the allocation of appropriate content and technical resources.
What are in your view the main challenges for the future of ICT in learning?
In my opinion the main challenge now is the full integration of ICT in education and training, both as a learning aid and a subject, implying the rethinking of organisation, curricula, teacher training and school infrastructure.
A particular effort should also be put in ensuring access to all, which would contribute to create the conditions favourable to increase participation in lifelong learning.Access the DOSSIER: eLearning programme 2000 - 2006: The Legacy to learn more about the eLearning programme
The educational value of the tool lies in a more effective and objective evaluation of school web Sites with a view to establishing self-confidence and improving quality. “HETSI Sites” was tried on a pilot basis to assess specific school websites while being simultaneously evaluated by 5 teams that checked criteria validity and reliability counting on a qualitative, hermeneutical approach.
The overall aim of OPAL is to improve the effectiveness of teaching and learning by enhancing the quantity and quality of Open Educational Resources that can be incorporated into higher education and further education provision.
The “European Open Educational Quality Initiative” will help to boost the use and acceptance of OER in Europe and beyond, by establishing a European Quality environment for individuals, organisations and policy makers, helping them to define, identify and develop quality for OERs. In particular the project will work towards building a quality consensus between various stakeholders and networks through dialogue and involving them into interaction about how OER should be supporting innovation and quality. The project supports the following objectives:
- Promote a new vision for quality of OER as levers for innovation of learning experiences embedded into long-term educational objectives and integrated in lifelong learning strategies.
- Foster the uptake of OER in education and training systems through provision of a quality infrastructure (a quality handbook, an award and a quality clearinghouse).
- Support educational actors to use OERs by identifying high quality resources and methodologies for defining, identifying and developing quality.
- Link and connecting learning communities in the field of OER in order to form a European consultative group for Quality of OER and building a partnerships for quality consensus
- Analyse and consolidate evidence on the added-value and impact of OERs, with a particular attention to institutional as well as pedagogical innovation and change.
- Develop easy to use tools and make them available as innovative practices or services, and will have a multiplier effect and result in greater knowledge about the use of OER. * Identifying and study innovative uses of OER in order to derive quality practices from the actual successful use and innovative integration of OER in educational scenarios. It will not limit its scope to but will pay special attention to OER for groups at risk of exclusion.
Promote OER as a catalyst for innovation and creativity in lifelong learning with the aim of forming a consensus on tools and mechanisms to evaluate and identify high quality OER for fostering innovation skills such as creative problem solving, discovery, learning by doing, experiential learning, critical thinking and creativity.
The project will support the mainstreaming of OEP, and will
- study and map quality approaches and methods for quality and link them to OEP in order to provide validated European Guidelines for Quality and Innovation through OEP in HE and AE,
- launch the European Consultative Group for Quality and Innovation through OEP in order to concert existing initiatives, form a multistakeholder validation environment, and liaise with existing international networks in order to contribute a chapter on quality and innovation to them,
- set up the EU Open Educational Quality Clearinghouse in which peer-reviewed OER and OEP will be linked and which will serve as a Register for organisations which want to join the European Charter for Quality and Innovation through OEP, and
- develop a European Award for Innovation and Quality through OER in HE and AE.
Extracted from OPAL
Évaluation de la qualité de l’e-learning grâce à une communauté internationale de révision par des pairs
L’analyse détaillée réalisée montre qu’en général, les organisations spécialisées dans le renforcement des capacités sont très soucieuses d’obtenir un label de qualité. Elles sont particulièrement intéressées par un outil qui permettrait d’évaluer et d’améliorer l’efficacité de leurs actions, afin de les aider à garantir le succès des programmes d’e-learning et de faciliter les comparaisons avec d’autres organisations.
L’analyse présentée dans cet article montre également que les organisations envisagent de façon différente le recours à la formation en ligne pour leurs activités de renforcement des capacités. Pour tenir compte de ces différences, un label avec deux options a été prévu : l’une pour les programmes individuels et une autre pour les institutions.
La production par des pairs n’est pas seulement une méthode innovante d’élaboration de contenus e-learning, c’est également une approche qui permet de faire participer une plus grande variété de professionnels dans la production de contenus d’apprentissage. Cependant, les problèmes que pose la gestion de la qualité de ce type de contenus peuvent remettre en cause les mérites de la méthode.
Un certain nombre d’approches et d’outils sont actuellement mis en place pour garantir et améliorer la qualité des contenus e-learning produits par des pairs. Cet article présente QualityScape, une méthode développée dans le cadre du projet européen QMPP avec l’objectif de veiller à la qualité des contenus e-learning produits par des pairs. La principale conclusion tirée de ce travail est que la qualité est le résultat de l’interaction entre le processus de production par des pairs et celui de la validation par des pairs. Mais, surtout, le point clé de ce projet est la mise en place d’une approche holistique de la production par des pairs, ce qui permet une utilisation efficace de cette méthode unique de création de contenus.
Nous orientons notre travail sur les processus d’apprentissage et d’enseignement, et considérons l’évaluation avec une approche pédagogique et une théorie didactique des médias. Ainsi, les ressources Internet facilitant un apprentissage actif par l’utilisation d’éléments multimédias sont considérées comme du contenu de qualité. Nous concentrons notre travail sur les processus d’enseignement et d’apprentissage et abordons la qualité en tant que valeur éducative potentielle du contenu.
L’une des principales conclusions tirées de notre étude est que, plutôt que chercher à adapter un modèle d’évaluation de la qualité « universel », de toute façon inapplicable à l’ensemble des domaines d’intérêt en raison de la grande quantité et de l’importante hétérogénéité de l’information disponible sur Internet, il convient d’utiliser des modèles d’évaluation propres à chaque domaine d’intérêt (éducation, économie, culture, etc.) durant toute la durée de vie du contenu.
Cet article présente également des conclusions intéressantes sur l’attitude et la compétence multimédia des enseignants, à partir des résultats obtenus auprès de groupes de discussion.
La nécessité de la certification des compétences numériques – les certificats Informatique et Internet (C2i)
Au niveau de la licence, le C2i niveau 1 certifie des compétences opérationnelles, mais surtout des aptitudes à communiquer et à travailler en collaboration en utilisant les TIC. Au niveau du master, les C2i niveau 2 regroupent les compétences identifiées par les branches professionnelles du domaine visé par le cursus. La méthodologie d’élaboration et la mise en place de ces certifications sont exposées dans cet article, avec en illustration une présentation complète des compétences qui ont été identifiées pour le C2i niveau 1 et le C2i niveau 2 « métiers du droit ».
L’introduction d’une certification des compétences numériques dans l’enseignement supérieur s’inscrit dans la volonté que l’ensemble des étudiants ait une maîtrise certifiée de ces compétences transversales, tant pour une réussite dans leurs études que pour leur insertion professionnelle future. La révolution culturelle que constitue la certification au niveau universitaire et sa généralisation dans les cursus « traditionnels » seront continuellement en « fil rouge » de l’exposé.La version intégrale de cet article est disponible en français et en anglais.
Andy Lane: "The benefits of sharing should be to improve the range of resources available so as to allow teachers more time to interact with their students"
Yes there is a tension that I think is largely created by Institutions mostly offering tightly defined programmes of study that have clear titles e.g. Biology or Architecture. While these programmes provide very useful learning experiences that can often be targeted at certain areas of work, the degree of choice in what students can study within the programme is limited as it does make it more manageable for the Institution and is also what is often expected by Professional Bodies. However many students may want to learn about things not covered in that programme, either because it has some relevance to the programme of study but is not included in it or because it offers other learning relevant to their wider aspirations. Open Educational Resources (OERs) can meet some of this need as they offer an alternative view of a topic covered in a programme (many users of MIT’s Open CourseWare are undergraduate students at other Universities who like to compare similar courses at MIT to ones they are taking) or act as enrichment study (we have found that some Open University students are studying units on OpenLearn from very different subject areas to their degree programme as they provide some other personal or professional development e.g. creative writing, learning a language).
Hello Andy, I was wondering if there exists or will exist any quality system for the open learning materials? How will the teachers or independent learners know about the quality and reliability of the material? I ask this because besides MIT and Open University there are surely other entities that offer open content.
Currently the quality of OERs is most often defined by the provider, and Institutions like MIT and the Open University are the guarantors through their normal quality assurance processes. In other cases it is for the users to judge for themselves whether the quality is good. While this may be difficult for an individual to do, the views of large numbers of people using rating schemes like that on the Amazon website for books could provide such a service.
Many people involved in the Open Educational Resources movement are looking at the different ways in which quality could be determined for users, especially resources developed by individuals or groups of people who are not part of an Institutional initiative. Two examples of this are the non-institution-based MERLOT and Connexions collections of OERs. In the former case they are using a traditional peer review mechanism often before publication of the resource (http://taste.merlot.org/peerreviewprocess.html) supplemented by user comments and ratings. In the latter case they have set up different ‘lenses’ (http://cnx.org/news/LensesIntroduced) for the resources to be judged after publication on the site. They have endorsement lenses for material reviewed by an authoritative body, affiliation lenses where content has been created by someone from an institution but not necessarily had it reviewed and members list lenses where registered users can give their views. We are looking at doing similar things for material contributed by others in the LabSpace of OpenLearn.
Is the disinclination of academics to use other people's materials an underestimated factor inhibiting the success of sharing initiatives, particularly expensively developed and maintained repositories?
Our research confirms Brian's view that academics are reluctant to re-use other people's materials. What do you see as the key challenges to changing this point of view?
I don’t underestimate this factor but see it as one of changing practices slowly. I believe that academics are not reluctant to use bits of other people’s material in the privacy of the lecture room but are reluctant to use substantive amounts in a more public way, either on an Intranet/Virtual Learning Environment or on the Internet. I say this because I don’t know a fellow academic who does not include a figure or graphic or small quote in slide presentations or handouts which has been published elsewhere. This is standard practice in teaching and writing but has been largely governed by the fair use or fair dealing rules around the use of copyrighted material. However most academics have little idea about the copyright laws in general and so are unsure what they can do and cannot do, even if someone has applied an open licence such as a Creative Commons one (http://creativecommons.org/) to the material. This is even more so if they are putting the material on a website and they have spent many hours developing it and do not want to give it away lightly in the mistaken belief they may make money out of selling such content (few can – even the Open University).
So the legal side does inhibit sharing, but I think that two other cultural practices are even more important. First, most academics teach as individuals. They are not used to working with others in either developing or sharing materials as they may be the only person teaching what they do in their institution and even where there are more of them there are no mechanisms for peer review of teaching in the way there is for research, and few Institutions recognise teaching in the way they do research through promotion and reward schemes. Second, most academics are not used to producing resources to publish on the web and they can worry about how best to do it and then worry whether their work is good enough to expose it to others. New technologies are easing the technical barriers for individuals to produce resources (although it can still be a big issue) but it is still the case that having more than one person developing materials with differing expertise is likely to be more effective.
Finally, there is also the question that many lecturers feel that the materials are not quite what they need for their situation – a point I take up in the next answer. These barriers may sound daunting but then we all had to learn how to use computers and that did not happen overnight.
Dear Andy. I have read about an issue that might be a problem to use and produce OER. It's about the concept that the production of educational material shouldn't (?) be separated from its usage. That is, the material is usually created for a certain context and situation of teaching and learning. However, when there is no connection - the one who makes the material, doesn't know when or how the material will be used / the one who uses the material can't evaluate the relevance and quality of the material without knowing the background - it can cause uncertainty both for production and usage. Do you think this is a relevant problem and is there any solution to it?
In my opinion OER are available to everybody but I do not think they are accessible and exploitable by everybody. The diversity of learning contexts is not taken into account: for example different curricula (or, in the case of higher education, students that come from pre-university education with different curricula) and different learning objectives (matching the learner's needs taking into account his/her knowledge capital). The diversity of the learning cultures is not taken into account: the teaching and learning environment in different countries/school/university systems do not have the same overall objectives and methods. What is your opinion on the current situation of OER concerning these aspects and on possible solutions in the future?
This issue of resources being localised or contextualised to the needs of the user is a common discussion point because societies and cultures do differ in what is expected in their formal education systems. I believe it is necessary to look at the needs of teachers and learners separately.
For teachers, an OER may not be exactly what they want, but with an open licence and appropriate formats they do have the potential to take and adapt that OER for their own situation. The more (good quality) OERs there are, the more choice a teacher will have to be able to put together resources for their class just as they do now in a smaller way when taking images and figures and so on for slide presentations at the moment. Why reinvent the wheel or put together a neat Flash animation or YouTube video clip explaining something if it has already been done? Even better, if someone in India has created a new resource specifically for use in India and you teach in India as well. The benefits of sharing should be to improve the range of resources available so as to allow teachers more time to interact with their students, providing even more of the localisation that way, rather than just lecture to them. It is not about creating a single version of a course, but some building blocks with which to construct multiple versions, including versions in other languages (check out this study unit on OpenLearn that has been translated into Catalan http://labspace.open.ac.uk/course/view.php?id=3173).
Learners who directly study an OER, on the other hand, may have to accept that it has been developed for a different setting to the one they are in, and get what they can from it as supplementary study for their formal studies or as a non-formal learning opportunity as I explained earlier. OERs are there for self-study, so there is no teacher to help contextualise it, but in OpenLearn you can always post questions to other learners who may respond and help you out with any difficulties you have. It is also the case that the degree of contextualisation varies greatly with the subject being studied. For many science topics there is a more stable body of knowledge while for other subjects, such as law, there are more limited similarities between jurisdictions.
Hello, Andy, in my opinion exploitation of OERs offered at The Open University in the United Kingdom is really not connected with the problem of "the sophisticated learner with very good access to the internet". However, it is another kind of sophistication -- learners should be experienced in English. Because many of my students at Poznan University of Technology (http://www.put.poznan.pl/en) can read English texts, I'm going to encourage them to use the your OERs during the e-course “Metody i techniki kształcenia na odległość” (http://www.geocities.com/eklezjastka/ester1.html). Luckily, some of those PUT students and especially my future students at The European Career College (http://www.kde.edu.pl/page.php/2/0/show/1/) will write on Open University forums. Do you believe in the value of such international cooperation in the exploitation of your OERs?
Yes I do believe that there is value from international cooperation and exploitation of our OERs and of other peoples’ OERs. We do want people to use them whether that is done formally or very informally. We already have several international projects that we have given space to in the Collaboration Zone of LabSpace (http://labspace.open.ac.uk/course/filter.php?grouping=topic&detail=20&order=date) and we know of many others using our materials, all of whom we are in contact with to know what they are doing and what value it has to them. Since the ethos of OERs is about sharing we also belong to the Open CourseWare Consortium (http://ocwconsortium.org/) and are participating in a funded project called MORIL (Multiligual Open Resources for Independent Learning) that involves several members of the European Association of Distance Teaching Universities (http://www.eadtu.nl/). Through these and other ventures we are researching and evaluating OER use in as many areas as possible so as to understand better all the issues that have been raised in the questions here. Some initial findings were reported at the OpenLearn2007 Conference we recently hosted (http://kn.open.ac.uk/workspace.cfm?wpid=7979)
The open contents initiative is a very good movement and I hope that people benefit from it. My concern is if the developing countries can gain from it too. Do you think the open material can help the developing countries in improving their conditions for teaching and learning? Can you name any specific projects about this? Are there any results so far?
Many developing countries have realised the potential of OERs (on top of Open Source Software) for reforming their education systems. Just before writing this I learned about the official launch of the Vietnam OpenCourseWare website at http://www.vocw.edu.vn with the aspiration to make OCW/OER features rich, accessible, and reusable at no cost for Vietnamese faculty members, students and self-learners. There is also plenty of OER work happening through the Commonwealth of Learning such as with WikiEducator (http://www.col.org/colweb/site/pid/4051) while here at the Open University we contribute to some specific projects in Africa such as the TESSA programme (http://www.tessaprogramme.org/). Many of the same issues I have discussed above apply to developing countries but the challenges are often greater because the physical and virtual infrastructure for education is not very good. It has been calculated that to have as great a proportion of the population go to University in developing countries as do in developed countries would require a new university being built every day for many years to come. OERs are not a complete solution by any means but meeting this scale of challenge is unlikely to succeed without them. The early results from these various initiatives are encouraging but reforming educational systems takes many years and substantive investment of time and effort as will be evident if you read up about the ones I have mentioned above.
Ulf-Daniel Ehlers: "E-learning seems to create a situation whereby quality considerations become unavoidable"
Do you think e-learning quality measures are equivalent to face-to-face learning measures; are they both competing on an equal basis? Or we are asking e-learning to go through a process in which conventional learning would never have succeeded?
Thursday, 9 November 2006
I totally agree with your point that there should be quality commitment in both face-to-face education and e-education. I also think that it makes no sense to develop quality standards/guidelines only for e-learning.
However, I have noted that e-learning is often leading to educational scenarios that reach new target groups, which are often remote, even crossing borders into other countries. Such an acceleration in terms of access might require special quality consideration; what do you think?
In addition, we often quote the “burning glass” metaphor, meaning that, where e-learning is introduced, it functions like a “burning glass”. It immediately highlights where educational processes are not thoroughly planned and implemented, and where quality has not been ensured. Where face-to-face processes can still be carried out – even in bad quality conditions – e-learning is just not working any more because of the immediate reliance on student participation. E-learning seems to create a situation whereby quality considerations become unavoidable.
I agree with you: Quality should not be mode-specific! I think we need specific quality models for all modes – and I would like to know if you agree with me when I say that, where e-learning comes into play, quality becomes particularly unavoidable (this is, of course, not to say that it is not important otherwise).
Dr Ehlers, In the case of adult learning and lifelong learning, the motive is a system of sharing information and ideas with a view to promoting action. How would you define quality in this field?
Dr Susan Sharma
Friday, 1 December 2006
SusanSharma (rest of the world)
This field seems to be a prototypical case of what we call “quality as a negotiation process”. Quality development in education requires the participation of all stakeholders; they have to come together to set out the priorities and work together to organise learning opportunities that address these.
For the design of high quality learning environments, this view has some consequences. Learning environments – a term used here in the broader sense, referring to the sum of all processes constituting the learning opportunity, including all resources and persons forming part of it – have to be designed in a way that enables learners to express their demands and preferences as part of the construction process. Only then can learners bring in their experience, backgrounds and demands, thus enabling providers to design learning environments in such a way to enable active learning, problem solving and competence development orientated towards the learners’ individual needs. The assurance of quality, reached exclusively through predefined, static frameworks (e.g. standard evaluation questionnaires) often does not pay sufficient attention to this particular need for co-production in educational settings (cf. Baijnath/Singh 2001, Freesen 2002). From this perspective, it is important that the development of quality strategies takes into account an active negotiation process as a specific condition of quality development and supports it proactively. Quality management concepts therefore have to include a negotiation component. This requires an extended understanding of process-orientated quality development models, and requires competence development and staff professionalisation components within quality strategies.
I would like to ask if there is a quality label or something similar for different e-learning providers. There are lot of e-learning course providers, but is there any guarantee of the quality of the studies, and how this can be found out? Thank you in advance.
Tuesday, 24 October 2006
The European Foundation for Quality is an excellent partner for these kinds of question. In fact, we are just initiating dialogue throughout Europe to bring together different quality seal providers, also including the National Finnish Quality Mark. For a more substantial overview, visit www.qualityfoundation.org and download the working documents on this subject.
My experience is that, when considering quality issues rated in different services and platforms, the main focus is on the various features implemented in the systems in comparison to the technological state-of-the art. My question is, if EQUEL also considers subjects such as accessibility issues (for people with disabilities, and for people with no broadband access), how they are rated in comparison to the technological point of view? Juergen Huellen (Germany)
Thursday, 9 November 2006
Ii know of a project called eAccess that addresses issues like that; have a look at the project website. It is a project on which a colleague of mine is working. EFQUEL is an open environment that does not exclude any topic from the quality debate. We have working groups on different subjects: www.qualityfoundation.org
I agree very much with your comments about quality. It cannot be reduced to processes and structures and the stakeholders involved in the learning process have to be professional. May experience is that it is even necessary to create a quality culture in the organisation and specific skills for professional quality development in learners/clients, teachers/providers, and also all those who are involved in the educative process (management, administration, etc.). All stakeholders involved in the learning process must have a clear and shared concept of e-learning and its quality. What do you think? Is this an important issue in the actual quality evaluation of e-learning programmes?
Thursday, 9 November 2006
luislobo (rest of the world)
I think this aspect cannot be stressed enough; thank you!
1. Co-production: involvement of all stakeholders
Classical service theory conceptualises the interactive relationship between the actors of people-orientated services with the “production” and “consumption” categories (cf. Gross/ Badura 1977). It is argued that education is a symbolically mediated productive-active interaction and production process. This process involves learners together with other actors (other learners, teachers, etc.). It therefore has to be conceptualised in the form of “pro-sumption” rather than a production-consumption relationship (cf. Martens/Prosser 1998). The targets of educational services are therefore conceptualised as active “co-producers” and not as passive receptors.
2. Quality culture
Quality literacy is a concept that is very much related to the philosophy of total quality management. Within this approach, quality is seen as a continuous improvement process, involving all stakeholders in the process of permanent assessment and quality improvement. For this aim, one element is of key importance: the introduction and development of a quality culture in an organisation. This has two dimensions: First, a managerial dimension, which is rather technocratic in nature and deals with implementing tools and instruments to measure, evaluate, enhance and ensure quality. This is usually facilitated though a top-down process. Second, a dimension of quality commitment focuses on an individual level. It relates to the individual commitment to strive for quality, using tools and instruments for quality development, but first and foremost it also focuses on changing attitudes, values and developing new skills and competencies in order to enable permanent quality improvement. Individual abilities, attitudes and values add up to a collective level, which in turn leads to a quality competent organisation. This dimension relates to a bottom-up process.
Dear Dr Ehlers, did we come up with any result regarding the Quality Standards in ONLINE Education? I don’t think anybody has yet agreed on the concept of “Quality Standards of ONLINE Learning”. I will continue to contribute more on the subject. What was the result of your surveys? By the way, I have been involved in e-learning for 15 years and I am a great believer in it; however, to date, I have not yet seen ONE GOOD ONLINE course to my liking. I hope your work will change the situation.
Best regards, Muvaffak GOZAYDIN, Turkey
Friday, 10 November 2006
Our latest survey on the “Use and Distribution of quality strategies in European E-learning” is published in www.qualityfoundation.org
L’article présente les principes directeurs du système bLO, qui ont servi de base pour le développement des outils. Ce système comprend trois principaux outils, dont deux ont déjà été mis au point, et un complément pour améliorer l’applicabilité de la méthode. Il inclut un outil de profil basé sur la norme LOM, le système indicateur d’évaluation comparative, et propose une carte des compétences qui sert de mécanisme d’amélioration continue. De plus, un système d’évaluation de l’efficience et de l’efficacité a été mis au point en complément de la matrice d’indicateurs.
Le bLO a été appliqué dans deux contextes différents. Afin de tester l’applicabilité de la méthode, trois modules d’un mastère en bâtiment ont été utilisés. Les informations obtenues grâce à ce test ont joué un rôle important pour l’amélioration des outils, et en particulier du système d’indicateurs. Par la suite, la méthode bLO a été utilisée comme outil d’évaluation pour certains résultats du projet européen intitulé « E3 : Electronically Enhanced Education in Engineering », dont l’objectif était de développer des OA qui étaient échangés entre les partenaires internationaux et évalués.
Enfin, l’article propose plusieurs pistes pour un futur travail visant à améliorer le système et à l’intégrer à d’autres systèmes.
This paper was adapted from one originally presented and published at the EDEN CONFERENCE 2004.