Towards Networked Learning Analytics – A concept and a tool

Networked learning is much more ambitious than previous approaches of ICT-support in education. It is therefore more difficult to evaluate the effectiveness and efficiency of the networked learning activities. Evaluation of learners ’ interactions in netw

Towards Networked Learning Analytics – A concept

and a tool

S. Retalis1, A. Papasalouros1, Y. Psaromiligkos2, S. Siscos2, T. Kargidis3 1University of Piraeus; 2Technological Education Institute of Piraeus; 3Technological

Educational Institution of Thessaloniki

retal@unipi.gr; andpapas@softlab.ntua.gr; siskakos1978@yahoo.gr; jpsa@teipir.gr;

<kargidis@mkt.teithe.gr

ABSTRACT

Networked learning is much more ambitious than previous approaches of ICT-support in education. It is therefore more difficult to evaluate the effectiveness and efficiency of the networked learning activities. Evaluation of learners’ interactions in networked learning environments is a difficult, resource and expertise demanding task. Educators participating in online learning environments, have very little support by integrated tools to evaluate students’ activities and identify learners’ online browsing behavior and interactions. As a consequence, educators are in need for non-intrusive and automatic ways to get feedback from learners’ progress in order to better follow their learning process and appraise the online course effectiveness. They also need specialized tools for authoring, delivering, gathering and analysing data for evaluating the learning effectiveness of networked learning courses.

Thus, the aim of this paper is to propose a new set of services for the evaluator and lecturer so that he/she can easily evaluate the learners’ progress and produce evaluation reports based on learners’ behaviour within a Learning Management System. These services allow the evaluator to easily track down the learners’ online behavior at specific milestones set up, gather feedback in an automatic way and present them in a comprehensive way. The innovation of the proposed set of services lies on the effort to adopt/adapt some of the web usage mining techniques combining them with the use of semantic description of networked learning tasks Keywords

Interaction analysis, networked learning design, understanding the learner's behavior

MOTIVATION

In a networked learning environment –usually supported by a Learning Management System (LMS), e.g. Moodle, Claorline, WebCT, etc.– lecturers often provide students with details of the course structure electronically, provide online course notes, make available presentations used during face to face lectures, give references for reading material. Learners are encouraged to use these resources and to engage in collaborative tasks where they have the opportunity to express ideas, discuss course topics. Learners can collaborate, share files and suggest ways of improving group deliverables in order to complete learning tasks collaboratively. Networked learning is much more ambitious than previous approaches of ICT-support in education. It is therefore more difficult to evaluate the effectiveness and efficiency of the networked learning activities. Significant effort is required to provide systematic evaluation of networked learning environment.

(Soller et al., 2004) have proposed the collaboration management cycle to model the generic phases followed by a system (either a human or a tool) to analyse interaction in networked learning settings, giving emphasis on the analysis computer supported collaborative learning activities. For example, a teacher or evaluator assisted by automatic tools can observe the current state of learners’ interaction and provide feedback to correct those situations that are not positive for learning. For example, a tool can compute the indexes of participation in a forum of the members of a group. From the observation of these indexes, a teacher can make suggestions to individual members or to the group as a whole, in order to help them meet the desired criteria considered by that teacher. The first phase of the cycle is the collection of data about the interaction. This is a fundamental phase, as the type of data used as source configures the rest of the analysis.

Two main approaches to analysis of interaction can be distinguished: dialogue and action based approaches. Dialog-based approaches to analysis of interaction are the most popular ones, due to the high interest of the study of language as a means for knowledge construction. In spite of their popularity, dialogue-based Networked Learning 2006

Networked learning is much more ambitious than previous approaches of ICT-support in education. It is therefore more difficult to evaluate the effectiveness and efficiency of the networked learning activities. Evaluation of learners ’ interactions in netw

approaches pose some important problems to analysis. The most important is that they need a lot of experise and resources for performing such a very detailed analysis. In order to face this problem, many researchers have proposed the use of pre-codified dialogue tools, by means of sentence-openers (Soller, 2001) or pre-codification performed by the learner (Vieira, Teixeira, Timóteo, Tedesco, & Barros, 2004). This solution allows the automatation or semi-automation of the analysis of data (Inaba et al., 2002).

Another approach to evaluation is based on the use of the actions that the users perform within the networked learning environments (supported by a LMS) as the main source of data. The structured interfaces provided by the LMSs allow researchers to collect data on the actions performed by the participants of the experience. This has lead to interesting interaction analysis techniques of synchronous computer supported collaborative learning environments such as the activity recognition approach (Muehlenbrock, 2004), the OCAF framework (Avouris et al., 2002), or the Collaboration Factor proposed in Fesakis et al., (2004). These approaches have been supported by specialised analysis tools that collect actions of the users in a collaborative learning environment and show different indexes of collaboration (Koutri et al., 2004). The main advantage of these tools is that they provide efficiently feedback to their users (either learners, lecturers or evaluators), a very important issue if we aim at using them during real networked learning experiences. The main limitation is that, in order to gain efficiency, interaction analysis techniques avoid any deep analysis of the details of the dialogues that happens during the collaborative experiences.

Up to our knowledge, action based approaches have not be extensively used for learners’ interaction analysis in asynchronous networked learning environments. The majority of LMSs offer lecturers basic log file analysis (number of visits per page, traffic analysis per learner, etc). Most of the times, this analysis is useless to the lecturer since it does not contain any valuable information of learners’ behavior that could give him/her hints so as to make revisions at design and implementation level. Of course, if a lecturer sees that some units of the course material have not been visited, she will probably delete them. Other quantitative data such as the frequency and time of participation and the use of the online discussion tool (eg. number of original posts, number of replies, etc.) can be obtained. This data alone offers no deep insight into the learners’ behavior for acquiring learning outcomes.

For example, it would be beneficial to cluster learners who solved an online test after having read the theoretical parts of the online material rather than those who studied only the examples of the online material. This is a good indication of identifying the preferred learning styles as well as the usefulness of the online material. It would be also interesting for the lecturer to find out which course online units a learner accessed and studied before posting a message about a topic of discussion.

Thus, lecturers (playing the role of Evaluators) need to complement the data that the tracking system of LMS now provides with the data from a detailed and reasonably accurate interpretation of a learners’ interaction with the peers, the use of the provided resources/objects as well as with data from content analysis. It is also necessary to automate data gathering, interpretation and analysis process.

Thus, the aim of this paper is to propose a new set of services for the evaluator and lecturer so that he/she can easily evaluate the learners’ progress and produce evaluation reports based on learners’ behaviour within a LMS. These services allow the evaluator to easily track down the learners’ online behavior at specific milestones set up, gather feedback in an automatic way and present them in a comprehensive way.

Input for designing these services came from “web analytics tools” which are software tools that have been developed for the purpose of a web site traffic analysis. “Web analytics” uses a variety of data and sources (mainly the web server log file and historical data of visits to the web server) and to evaluate Web site performance, visitor behavior, and patterns at both an individual and an aggregate level (Pierrakos et al., 2003). Its goals are to improve web site performance, especially from the content perspective, enhance visitor experience, and identify issues for revisions at design and implementation level. There are plenty of web analytics tools that go beyond simple statistical reports and produce a more detailed description of the user’s navigational behavior, by applying basic data mining or most importantly in our case web usage mining techniques, such as Clustering, Classification, Association Rule Mining, Regression Analysis and so on.

Our innovative approach tries to adopt some of the web usage mining techniques combining them with the use of semantic description of networked learning tasks. It is supported by a tool, called CosyLMSAnalytics, which helps in automatically gathering and analysing data concerning learners’ access patterns, as well as making correlations among students’ learning paths, etc. The ultimate goal of this approach is to allow a lecturer and an evaluator to frequently assess the by gathering learners’ feedback when necessary, monitor individual or collective progress, so as to provide informative feedback to students, adapt instruction as needed, and ultimately improve student overall performance.

Networked Learning 2006 2

Networked learning is much more ambitious than previous approaches of ICT-support in education. It is therefore more difficult to evaluate the effectiveness and efficiency of the networked learning activities. Evaluation of learners ’ interactions in netw

The structure of the paper will be the following: Section 1 has already stated the motivation of our work. Section 2 contains state-of-the-art information about web analytics tools. It will be shown that these tools cannot adequately support the task of analysis of learners’ behaviour in networked learning environments. This is why the need of developing new tools, called networked learning analytics tools, has arisen. Section 3 gives an overview of the networked learning analytics approach. The CoSyLMSAnalytics tool, will be used as a show case of the application of the networked learning analytics approach into real practice. Finally, the paper will contain concluding remarks and topics for future work in section 4.

FROM WEB ANALYTICS TO NETWORKED LEARNING ANALYTICS

Web Analytics is a more general term that mainly seeks to identify visitors’ navigational behavior and track their access patterns in the site’s Web pages. To do this, several tools have been developed in order to define users’ ‘click streams’ and period sessions depending on the time a visitor spend on each Web page. There are other related terms, such as Website Traffic Analysis, Web Log Analysis, Log File Analysis, and Web Mining.

A complete Website traffic analysis includes the categorization and pre-processing of Web data and the extraction of correlations between and across different such kind of data (Mobasher et al., 1996).

Website traffic analysis takes as input raw Web data in the form of a log file from the web server, and processes them in order to extract statistical information. A simple analysis in the server’s log files gives little or trivial information regarding unknown patterns of the user’s navigational behavior or even the usability of the Web server.

This information concerns usage statistics, such as average time spent on page, count of visits, most downloaded files etc. and it is used basically by Web administrators for improving the system performance, facilitating the site modification task and providing support for marketing decisions (Srivastava et al., 2000).

Web Usage Mining in more detail is the application of Data Mining techniques to large Web data repositories in order to discover hidden knowledge concerning users’ navigational behavior and extract meaningful patterns that go beyond simple queries and usage statistics.

Several algorithms have been developed seeking to identify visitor’s similar browsing behavior. Most of these algorithms are implementations of a number of Data Mining Methods like the following ones:

1.Clustering is used to group together items that have similar characteristics. In Web Usage Mining,

clusters are created so as to categorize number of users with common browsing patterns. The most

common clustering algorithms are K-Means, BIRCH, ROCK, Hierarchical Clustering Algorithm,

DBSCAN and TURN.

2.Classification is a process of mapping items into pre-defined classes. In the Web domain classes

usually represent different user profiles and classification is performed using selected features that

describe each user’s category. The most common classification algorithms are Decision Trees,

Naïve Bayesian Classifier and Neural Networks.

3.Association Rule Mining is a technique for finding frequent patterns, associations and correlations

among sets of items. Such rules indicate the possible relationship with a specified support that is

the number of data sequences that contain the pattern between Web pages that are often viewed

together even if they are not directly connected and can reveal associations among groups of users

with specific interests.

(Koutri et al., 2004) provide an overview of the state of the art in research of web usage mining, while discusses the most relevant criteria for deciding on the suitability of these techniques for building an adaptive web site. Among the techniques discusses are: Clusters of document references reflect pattern of common usage, Clusters of user visits, association rule and sequential patterns (see Table 1).

Currently there is a variety of Web log analysis tools available both commercial such as Net Genesis’ E-Metrics Solutions Suite, the Funnel Web by Quest, the NetTracker provided by Sane Solutions, WebTrends Log Analyzer software by WebTrends, as well as open source such as WUM and WEKA (Pierrakos et al., 2003). Web usage mining technique Data Mining Metrics Interpretation

Clustering web document references Groups of web document

references

Patterns of common usage

reflecting mentally related web

documents

Networked Learning 2006 3

Networked learning is much more ambitious than previous approaches of ICT-support in education. It is therefore more difficult to evaluate the effectiveness and efficiency of the networked learning activities. Evaluation of learners ’ interactions in netw

Clustering user visits Groups of user visits Patterns of common navigational

behaviour reflecting users actions

and motivations

Association rule mining Associations among web

document references Patterns of common usage reflecting related web documents, as well as the notion of antecedents and consequents

Sequential pattern mining Associations among sequences

of web document references

over time Patterns of typical browsing behaviours over time

Table 1. Interpretation of Web usage mining techniques [Source: Koutri et al., 2004]

Up to our knowledge none of these systems has been used for analysing learners’ interaction within a networked learning environment. This does not mean that there are not any attempts for qualitative evaluation measurements based on log files. On the contrary, interaction analysis using quantitative approaches is still an open R&D topic in the area of computer supported collaborative learning (CSCL), despite the various attempts already made. A lot of resources about Interaction analysis in CSCL can be found in the web site of the Kaleidoscope project ().

Moore (1989) suggested three kinds of interactions in a networked learning environment. These are: learner–content interactions, learner–instructor interactions, and learner–learner interactions. These three together have been used for trying to understand how learners construct knowledge in online networked learning environments. Several research groups developed coding schemes that categorize interactions according to models of knowledge construction in an effort to perform interaction analysis more efficiently. Gunawardena and colleagues (1997) developed a model and coding scheme for online interaction with five phases of knowledge construction:

i.sharing/comparing of information;

ii.discovery and exploration of dissonance or inconsistency among ideas, concepts, or statements;

iii.negotiation of meaning/co-construction of knowledge;

iv.testing and modification of proposed synthesis or co-construction; and

v.agreement statement(s)/applications of newly constructed meaning.

Shute and Glaser (1990) propose a technique that enables the evaluator to derive global learner differences on the basis of learner interaction measures. The approach by Shute and Glaser can be summarized as involving 1) counting frequencies of actions, 2) categorizing them into ‘meaningful’ units, and 3) making comparisons across groups.

Gaßner et al, (2003) illustrate how log files can be captured, codified and analysed for providing statistics of interaction as well as activity patterns. The MatchMaker TNG tool offers a framework for activity logging. As they claim “using the MatchMaker TNG log files we can access the complete structure of the activities that took place in a session”. An analysis method like the one that appears in Mühlenbrock (2004), can help evaluators in identifying collaborative activity patterns and reflecting them as feedback. Most of the times, interaction data concern the number of the messages read, the postings to a discussion board, the file up-loads, the annotations to the uploaded files, etc. Examination of the frequency of interactions across the individual learner or among members of groups can lead to identifying patterns of contributions.

Apart from quantitative measurements, analysis of participants' postings (content analysis) should be performed in sequel, so as to reveal many of the behaviors associated with collaborative learning situations (Curtis & Lawson, 2001).

Recently, CSCL research has focused on the use of Social Network Analysis (SNA) (Wortham, 1999), as an extension to the common technique of descriptive statistics of the messages or length of contributions (Benbunan-Fich & Hiltz, 1999).SNA creates graphical scheme of the contributions where factors such as “centrality” and “density” can be used for describing the learners’ groups cohesion. For example, “density” is a

Networked Learning 2006 4

Networked learning is much more ambitious than previous approaches of ICT-support in education. It is therefore more difficult to evaluate the effectiveness and efficiency of the networked learning activities. Evaluation of learners ’ interactions in netw

factor that indicates the extent to which students respond to each other (Lipponen 2003).

Our way of thinking about networked learning analytics concerns the identification of learning resources that convey the desired content and satisfy learners either individually or in clusters of learners with same characteristics (styles, knowledge level, etc.). Moreover, it concerns the discovery of issues that helped students move toward knowledge construction, understanding of topics of the subject domain, and problem-solving skills acquisition either individually or in groups. It mainly focuses on social and personal interaction among learners and among learners and resources.

With the proposed approach we try to have a good insight to the three design components of a networked learning environment identified by Goodyear (2000). These are (1) the tasks set for students, which influence but do not determine their actual learning activity, (2) the (social) organisational forms put in place for them, out of which they develop more or less convivial learning relationships, and (3) the digital resources, tools, artefacts, etc that we make available to them, which are used by them to “customise or fit out their individual learnplaces” (Goodyear et al., 2004).

Thus, we have proposed an approach, supported by a specialised tool, for the automatic classification of learners in a according to their access patterns and behaviour in a LMS. It would be beneficial to understand what learners and lecturers are doing alone and together and how learning goals learners are accomplishing.

A NEW APPROACH FOR ANALYSING LEARNERS’INTERACTION IN A NETWORKED LEARNING ENVIRONMENT

The main aim of our approach supported by the tool is to gather information concerning learners’ access patterns as well as to extract correlations among their learning paths. The first step is to get a general view of students’ log entries and then produce usage statistics such as count of visits, average time interval spent for performing an activity.

More importantly, this approach makes a Path Analysis. It gives us the ability to form all clustering groups of learners that perform an activity of a specific activity type (e.g. study an example, solve exercise, read a case study, post a message etc.) during one or more online sessions. Thus, we can “discover” groups of learners with similar browsing behaviour. This is a first indication of learners’ access patterns, before proceeding to deeper analysis by creating even more complex queries and extracting interesting association rules from the learning paths.

Our approach is supported by an analysis tool which is called CoSyLMSAnalytics (see Figure 1). This tool has been developed in Visual Basic and its current situation has been tested for analysing learners’ behavior in Moodle LMS [Spyros thesis 2005]. More specifically, it can:

?Produce usage statistics such as count of visits, average time interval spent on an activity, and offer then in various formats such as cross tabs and charts.

?Provide more detailed information regarding discussion forum statistics.

?Exploit learners’ sequential patterns by drawing the exact paths being followed by each learner individually or in groups

?Show deviations of individual learners from the typical series of activities performed by their group ?Perform Path Analysis with the creation of more complex queries that reveal interesting correlations and association rules among students’ learning paths.

The tool exports data in a specialized format for being used by the SPSS statistical package. Our approach suggests the utilization of the Two Step Clustering Algorithm in order to group students based on their overall behavior in networked learning environment. This algorithm is designed to reveal natural groupings or clusters within a data set that would otherwise not be apparent. It has several desirable features that differentiate it from traditional clustering techniques, such as:

?Handling of both categorical and continuous variables

?Automatic selection of number of clusters

?Scalability that deals efficiently with large volume of data

Networked Learning 2006 5

Networked learning is much more ambitious than previous approaches of ICT-support in education. It is therefore more difficult to evaluate the effectiveness and efficiency of the networked learning activities. Evaluation of learners ’ interactions in netw

This algorithm has been used for finding association rules among students in correlation with the activities they have performed. In this way, we can identify clusters of students who have strong learning preferences, e.g. giving priority in reading theory than examples, etc. With this algorithm one can find out that a learner does not belong to a cluster that contains a number of performed activities. On the other hand, one can easily see that a cluster, which contains some of the overall activities, consists of a number of learners, meaning that these learners did not in turn performed these activities in some paths or sessions.

Finally, our approach accords to the philosophy of the quantitative analysis techniques of asynchronous learners’ interaction in discussion fora. More specifically, analysing data from the use of a discussion forum, means that apart from producing descriptive statistics, such number of postings per learner, number of replies, etc., the lecturer can annotate the messages using her preferable schema (e.g. such as proposal, contra-proposal, question, comment, clarification, and agreement (Barros & Verdejo, 1999; Mühlenbrock, 2004) and then getting descriptive statistical results from the analysis of the interactions based on that schema. Distinguishing when groups are talking about the concepts to be learned (conceptual) from when they are talking about procedures for completing the task (non-conceptual) provides a picture of how groups approach the tasks, be it collaboratively (conceptual) or cooperatively (non-conceptual).

Moreover, with the aid of the tool, we can export data in the appropriate format so that it can be used as input to other tools such as NetDrawTM for social network analysis.

Figure 1. A screenshot of CosyLMSAnalytics tool

CONCLUSIONS

A greater understanding of how learners communicate, complete tasks and construct new knowledge through navigation in a LMS or collaborative dialogue can help online educators create such tasks more effectively. Understanding and evaluation of learners’ interactions in networked learning environments is a difficult, resource and expertise demanding task. It is still an important topic in the field of networked learning. Educators participating in online learning environments, have very little support by integrated tools to evaluate students’ activities and identify learners’ online browsing behavior and interactions.

As a consequence, educators are in need for non-intrusive and automatic ways to get feedback from learners’ progress in order to better follow their learning process and appraise the online course effectiveness.

In this paper we proposed an innovative approach (supported by a specialized tool) for analyzing learners interaction in networked learning environments which has adopted many issues from web analytics and put them in a the networked context using the specific requirements and semantics of the field. This approach has been successfully applied with real data concerning two postgraduate courses of the University of Piraeus, Department of Technology Education and Digital Systems, namely Human-Computer Interaction and Learning Management Systems.

This action-based approach is quite useful, but its performed analysis needs to be complemented by other sources of data and analysis techniques in order to achieve the required understanding of a situation. Mixed evaluation techniques supported by automatic tools are an example of how the different perspectives can be complemented (Neale & Carroll, 1999; Martínez et al., 2003). This is actually what happened for the course on Human - Computer Interaction, where the results from the analysis using the proposed approach had been complemented by results from pre & post test questionnaires (TELL 2005).

Networked Learning 2006 6

Networked learning is much more ambitious than previous approaches of ICT-support in education. It is therefore more difficult to evaluate the effectiveness and efficiency of the networked learning activities. Evaluation of learners ’ interactions in netw

Especially, the dialogue-based approach to analyse interactions in a networked learning environment using some interaction schema has some limitations. First of all, codified dialogue can be too limited in many learning situations; and second, the codes chosen might not be acceptable by other researcher. For this reason, some recent approaches to the use of dialogue in analysis of interaction propose to use data and text mining in order to help to codify the text and avoid this demanding procedures (Padilha, Almeida, & Alves, 2004). However, this approach is still at a very early stage of development, and the results are not enough for use in real situation. Concluding this approach seems promising and has successfully applied. It can definitely be further developed in the future in various directions. Some of the future research directions could be:

?Utilization of the software in practice from other educators who use Moodle or other LMS in order to reveal other areas of improvement.

?Scalability of the system in order to take as input Web server’s log file, especially regarding Learning Management Systems that follow SCORM and IMS Manifest standards.

?Integration of this tool into a Web based adaptive educational system that would maintain and update user-profiling, interacting with the user model and in some cases make predictions and provide recommendations to learners for performing learning tasks.

ACKNOWLEDGMENTS

This work was partially supported by the “TELL: Towards effective network supported collaborative learning” project co-funded by the European Commission e-learning program (ref num: 2003-4721 TELL).

REFERENCES

Avouris, N.M., Dimitracopoulou A., Komis, V., Fidas C., (2002), OCAF: An object-oriented model of analysis of collaborative problem solving, G. Stahl (ed), Proceedings CSCL 2002, pp.92-101, Colorado, January 2002, Erlbaum Assoc. Hillsdale, NJ, 2002

Barros, B., & Verdejo, M. F. (1999). An approach to analyse collaboration when shared structured workspaces are used for carrying out group learning processes. In S. P. Lajoie & M. Vivet, editors, Artificial Intelligence in Education: Open Learning Environments , pages 449-456: IOS Press.

Benbunan-Fich, S., & Hiltz, S. (1999). Educational applications of CMCs: solving case studies through asynchronous learning networks. Journal of Computer Mediated Communications, 4(3)

Curtis, D.D., & Lawson, M.L. (2001). Exploring collaborative online learning. Journal of Asynchronous Learning Networks, 5(1).

Fesakis,G., Petrou, A., Dimitracopoulou, A., (2004), Collaboration Activity Function: An Interaction Analysis Tool for Computer Supported Collaborative Learning Activities, ICALT 2004, Joensuu, Finland, August 2004

Gaßner K., Jansen M., Harrer A., Herrmann K., Hoppe U. (2003), Analysis methods for collaborative models and activities, In (Ed) U. Hoppe, Computer Support for Collaborative Learning: Designing for Change in Networked Learning Environments, CSCL 2003 congress: 14-18 June 2003, Bergen, Norway Goodyear, P. (2000). Environments for lifelong learning: ergonomics, architecture and educational design.

Integrated and Holistic Perspectives on Learning, Instruction & Technology: Understanding Complexity (J. M. Spector & T. Anderson ed., pp. 1-18). Dordrecht: Kluwer Academic Publishers. Goodyear, P., Jones, C., Hodgson, V., Asensio, M., & Steeples, C. (2004). Undergraduate students’ experiences of networked learning in UK higher education: a survey-based study. Advances in research on networked learning (Goodyear, P. ed., ). Dordrecht: Kluwer Academic Publishers. Gunawardena, C. N., Lowe, C. A., & Anderson, T. (1997). Analysis of a global online debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing. Journal of Educational Computing Research, 17(4), 397-431.

Inaba, A., Ohkubo, R., Ikeda, M., & Mizoghuchi, R. (2002). An Interaction Analysis Support System for CSCL.

An Ontological Approach to Support Instructional Design Process. 1st European Conference on

Networked Learning 2006 7

Networked learning is much more ambitious than previous approaches of ICT-support in education. It is therefore more difficult to evaluate the effectiveness and efficiency of the networked learning activities. Evaluation of learners ’ interactions in netw

Computer-Supported Collaborative Learning(Euro-CSCL 2001), Maastricht, The Netherlands, on March 22-24, 2001.

Koutri, M., Avouris, N., Daskalaki S. (2004).A survey on web usage mining techniques for web-based adaptive hypermedia systems. In S. Y. Chen and G. D. Magoulas (ed), Adaptable and Adaptive Hypermedia Systems, Idea Publishing Inc., Hershey, 2004

Lipponen, L. (2002). Exploring foundations for computer-supported collaborative learning. G. Stahl Computer Support for Cooperative Learning. Foundations for a CSCL Community. Proceedings of CSCL 2002, Boulder, CO (pp. 72-82). Hillsdale, NJ: Lawrence Erlbaum Associates.

Martínez, A., Dimitriadis, Y., Tardajos, J., Velloso, O., & Villacorta, M. B. (2003). Integration of SNA in a mixed evaluation approach for the study of participatory aspects of collaboration, Workshop on Social Networks

Mobasher, B., Jain, N., Han, E., and Srivastava, J. (1996). Web mining: Pattern discovery from world wide web transactions. Technical Report TR-96050, Department of Computer Science, University of Minnesota, Minneapolis.

Mühlenbrock, M. (2004). Shared Workspaces: Analyzing User Activity and Group Interaction. In Hoppe, H. U., Ikeda, M., Ogata, H., Hesse, F. Hrsg.: New Technologies for Collaborative Learning, Computer-Supported Collaborative Learning Series, Kluwer

Neale, D., & Carroll, J. (1999). Multi-faceted evaluation of Complex, distributed activities. Computer Supported Collaborative Learning Conference (CSCL ’99) Mahwah, NJ: Lawrence Erlbaum Associates

Padilha, T. P. P., Almeida, L. M., & Alves, J. B. M. (2004). Mining Techniques for Models of Collaborative Learning. J. Mostow, & P. Tedesco, Designing Computational Models of Collaborative Learning Interaction, workshop at. ITS 2004 (pp. 89-94). Maceió, Brazil

Pierrakos D., Paliouras G., Papatheodorou C., Spyropoulos C.D. (2003) : Web Usage Mining as a tool for Personalization – A Survey, User Modelling and User-Adapted Interaction, 13 (4), pp. 311-372. Shute, V. J., & Glaser, R. (1990). A large-scale evaluation of an intelligent discovery world: Smithtown.

Interactive Learning Environments, 1, 51-77.

Siskos, S. (2005), Website Traffic Analysis Tools- A Case Study on E-Learning, MSc thesis, University of Paisley, October 2005

Soller, A. L. (2001). Supporting social interaction in an intelligent collaborative learning system. International Journal of Artificial Intelligence in Education, 12, 40-62.

Soller, A., Jermann, P., Muehlenbrock, M., & Martínez-Monés, A. (2004). Designing Computational Models of Collaborative Learning Interaction: Introduction to the Workshop Proceedings . Proceedings of the 2nd International Workshop on Designing Computational Models of Collaborative Learning Interaction, ITS 2004 .

Srivastava J, Cooley R., Deshpande M., Tan P., (2000). Web usage mining: Discover and applications of usage patterns from web data, SIGKDD Explotations, 1(2), 12-23.

TELL (2005). "Introducing a Framework for the Evaluation of Network Supported Collaborative Learning",(1.37 MB .pdf) TELL Project, Deliverable of WorkPackage 1. Retrieved on January 2006 from TELL project website from http://cosy.ted.unipi.gr/tell/.

Vieira, A. C., Teixeira, L., Timóteo, A., Tedesco, P., & Barros, F. (2004). Analyzing Online Collaborative Dialogues: The OXEnTCHE-Chat . Intelligent Tutoring Systems. 7th International Conference, ITS 2004. (pp. 315-324). Berlin Heilderberg.

Wortham, D. W. (1999). Nodal and matrix analysis of communication patterns in small groups. In C. M.

Hoadley, & J. Roschelle (Eds.), Proceedings of the Computer Support for Collaborative Learning (pp.

681–686). Palo Alto: Stanford University.

Networked Learning 2006 8

本文来源：https://www.bwwdw.com/article/lw91.html