Using A Group Support System in Strategic Academic Planning – A Field Experimental Study on the Influence of Facilitation 

T. Nepal
Nepal@dut.ac.za
Faculty of Accounting and Informatics
Durban University of Technology, Durban 4000, South Africa 

D. Petkov
petkovd@easternct.edu
Department of Business Administration
Eastern Connecticut State University, Willimantic, CT, USA

Abstract

This paper presents the results from a field experimental study on the role of facilitation for improving the quality of ideas generated in a GSS environment used for strategic planning within an academic department. The content of the ideas on academic management generated as part of the planning work is not discussed as it is specific for the particular department. Instead we focus here on process facilitation in group decision making and the role of the facilitator. The contributions of the paper are in the originality of the application area (academic strategic management) chosen for the use of a GSS and in the insights from the field experimental study on the effect of facilitation on the quality of the work of the participating lecturers from a university computing department who had real vested interest in the problem of concern.  

Keywords: Group support systems, facilitation, academic planning



1. INTRODUCTION

The effect of facilitation on the group process is one of the five most important theoretically and practically relevant moderators in Group Support Systems (GSS) use according to Dennis and Wixom (2002). Earlier work on the role of facilitation in GSS was focused on experimental investigations of facilitator effects on meeting outcomes (see Anson, Bostrom and Wynne, 1995; Bostrom, Anson and Clawson, 1993). Since the mid 1990s there was a renewed interest in facilitation aspects research in GSS environments (see Ackermann (1996); Dickson et al. (1996); Griffith et al. (1998); Khalifa et al. (2001) and Miranda and Bostrom (1999). Publications over the last ten years have been more reflective on reasons for failure of GSS due to facilitation (De Vreede et al. (2003). They have focused on meta analysis of GSS research including facilitation issues (see Dennis and Wixom, 2002). Content facilitation research in knowledge acquisition and learning is reviewed in Kwok, Ma and Vogel. (2002). Niederman et al. (2008) suggest an expanded consideration of elements of Adaptive Structuration Theory for structuring tactics on agendas, design patterns and micro processes as a means for getting deeper insights into facilitation of GSS sessions. It can be noted that the research interest in the GSS field after 2002 has decreased and hence most of the relevant references quoted in this paper are relatively old. On the other hand most of the published research is about laboratory experiments involving students who do not have a vested interest in the problems being addressed in the experiments. Hence we conclude that there is a need for more field studies like the one described in this research on the practice of facilitation in GSS that explore both quantitative and qualitative aspects and contribute to the continuous development of a deeper understanding of the practice of using GSS. 

Academic management is one of the areas where GSS have not been widely used in field studies. While Trauth and Jessup (2000) study effects of computer mediated discussions on issues related to gender equity at a university, we could not find any published sources on the use of GSS in university strategic management over the last 10 years. Academic management on the other hand is a well established field (see a detailed overview on this area in Fathi and Wilson (2009)). Most of the work done in that field is about empirical research that does not focus on the details of a particular problem situation and instead is attempting to develop generalizations from large amount of empirical data. While such efforts are important it becomes problematic how one translates such knowledge in the condition of a particular institution of higher education. This was another motivation for the field study on the reported here.
 
According to Bostrom et al (1993), facilitation provides structures and/or support to a group in order to positively influence how the group accomplishes its outcomes. Structures provide an overall framework or context to activate individuals or groups to behave in a particular way. Support activities are used primarily to maintain and promote these structures, encourage effective task and relational behaviours, and deal with disruptive influences in the meeting. According to Dennis and Wixom (2002), process facilitation attempts to help the group in structuring the process by which it uses the available GSS tools. Content facilitation attempts to improve task performance directly by offering insights, interpretations, or opinions about the task and various decision alternatives available to the group (Dennis and Wixom, 2002).

The goal of this paper is to explore a field study on the role of facilitation for improving the quality of ideas generated in a GSS environment used for strategic planning within an academic department. The content of the ideas on academic management generated as part of the planning work is not discussed as it is specific for the particular department. Instead we focus here on process facilitation in group decision making and the role of the facilitator. The contributions of the paper are in the originality of the application area (academic strategic management) chosen for the use of a GSS and in the insights from the field experimental study on the effect of facilitation on the quality of the work of the participating lecturers from a university computing department who had real vested interest in the problem of concern. The rules of the laboratory experiment were adhered to, in order to achieve suitable/appropriate conditions allowing one to draw more meaningful conclusions about the influence of the factors that were studied.
The paper proceeds as follows: Section two provides an overview of the role of facilitation as outlined by Ackermann (1996). Section 3 provides a brief insight into the problem of strategy formulation for an academic department in a tertiary institution. Section 4 describes the field experiment while section 5 discusses the measures of performance for the group generation of ideas. Section 6 discusses the results of the experiment and section 7 provides some concluding comments and areas for further research.
2.	ON THE ROLE OF FACILITATION IN ELECTRONIC GSS ENVIRONMENTS

The introduction of anonymous electronic brainstorming (computer-mediated communication) can supplement verbal communication in a GSS environment. Electronic brainstorming can affect process gains and losses in at least three ways: through enabling parallel communication, through the provision of group memory, and through facilitating anonymous ideas (Dennis and Valacich, 1993). A software package for group decision making and multi criteria decision analysis, Team Expert Choice (Team EC) was used in this research as it provides all of these features. Hence the purpose of this work is not to distinguish in any way Team Expert Choice from other GSS environments.

This field experiment focuses on the introductory sub stage (of the meeting stage) of the facilitation process as it is defined by Ackermann (1996) which comprises of the following learning points: 

1.	Provide an explanation to the process.
2.	Provide a clear set of objectives and a corresponding agenda.
3.	Create and display an overview of the issue/problem.
4. Manage the group’s direction and progress.
5. Ensure that participants perceive themselves to be equal for the event.

The requirements of points 1 to 3 were addressed in the experiment. Learning points 4 and 5 were excluded because it was not possible to guarantee the identical treatment for both groups. Details of the learning points are explained below:

* Providing an explanation to the process that is common to both groups: This was achieved by ensuring that the participants were familiar with the software and the electronic brainstorming environment. The participants from both groups were required to complete a pilot exercise to familiarise themselves with the technology, the process and procedures. An explanation setting out clearly what was available in terms of software and technology and what was required was provided verbally and in writing. 

* Providing a clear set of objectives and a corresponding agenda only to the experimental group: They provide a sense of direction and establish clear milestones against which the group can monitor its progress (Ackermann, 1996). A set of objectives and an agenda were given to the experimental group. 

* Creating and displaying an overview of the issue/problem only to the experimental group: The overview of the problem provides the experimental group with an understanding of how the key issues and goals/objectives relate to each other and also helps to establish the order in which to tackle the issues (Ackermann, 1996). This was achieved by providing the vision of the institution, the mission statement of the institution and the department, and also a list of activities affecting strategy formulation for the university department. Further details are discussed in the next section.

3.	THE PROBLEM OF STRATEGY FORMULATION FOR AN ACADEMIC DEPARTMENT AT A TERTIARY INSTITUTION

A strategic planning problem of a large Information Technology academic department within a South African University of Technology was selected as the subject of the electronic brainstorming session. It was presented to two homogenous groups of full time permanent academic staff members from the same department. The two groups had shared common instructions on the issues of process. The experimental group was presented with additional information. This was in line with the first three points of the introductory sub stage of the facilitation process as defined by Ackermann (1996).

The participants were required to focus on issues that will realign, and influence the development and implementation of a strategic plan for the department. To assist in the generation of ideas, three broad areas were identified as sub-goals of the strategic process: Academic Development, Research, and Community Development. These classifications were intended merely to assist the participants and they were not required to deliberate on whether a particular idea belongs to a particular category. The focus of the session was to generate as many ideas as possible. The scope of the experiment included realistic and achievable goals in a time frame of three to five years and included the question of any rationalization and/or merger with other institutions. 

Criteria of Activities affecting the sub-goals are:
Student Development; Curriculum Development; Staff Development; Teaching, learning, and auxiliary resources; Management Structure Procedures; Inter-departmental Collaboration; Inter-institutional Collaboration; Collaboration with Industry Partners; Contribution to Research Centers of Excellence; Improvement of Marketing Strategy; Provision of Computer Resources for disadvantaged schools; Computer Literacy Program for Teachers from previously disadvantaged schools; Computer Literacy Program for the Unemployed.	

The following sections will outline the details of the field experiment and the discussion of the results.

4.	DESCRIPTION OF THE FIELD EXPERIMENT SETUP

4.1	Hypotheses for the outcomes on issue generation 

The following hypotheses were formulated: 
Hypothesis 1:	 The group with a clear set of objectives, additional information and an overview of the problem will generate a larger number of unique ideas.

Hypothesis 2: The group with a clear set of objectives, additional information and an overview of the problem will generate better quality ideas. 
        
The quality of an idea is dependent on how relevant it is for the strategic planning process.
4.2	Subjects
The subjects in both groups of the experiment were full time academic staff from the same department. The two groups were formed through a matching procedure to avoid possible bias on age, race, experience or gender. The participants were familiar with strategic planning activities carried out in 1996 without GSS and in 2000 with a very informal use of the same GSS environment. A handout containing information on the meeting environment, hardware/software components and the problem statement were given to the participants in both groups. Finally, all participants were relatively familiar with the Team Expert Choice GSS environment.
The participants were divided into two groups, X being the control group to which no facilitation was provided, and Y being the experimental group, to which facilitative support was provided in the form of additional information including: 
* Mission statement and vision of the institution
* Mission statement of the department
* Overview of the staff and student growth in the Department for the past 5 years
* Gender Breakdown of IT Students in the Department
* Race Breakdown of IT Students in the Department for Year
* Graduates and Post Graduates for the last 5 years
* Academic Staff establishment: Rank vs Gender for Year
* Staff Profile: Full Time Academic Staff Qualification for Year
* Staff Profile: Race Breakdown

There were nine members in each group. Gallupe et al. (1991) used group size of nine members, and most small group experiments in GSS involved groups with a size between four and nine. For this reason, the group size in this study was considered comparable to the studies conducted by the two groups of researchers mentioned above.
4.3 	The Task
These included brainstorming on issues relevant to the strategic planning process. The first author as a facilitator provided a brief explanation of traditional brainstorming and its shortcomings. He also cited the similarities and differences between traditional and electronic brainstorming. A brief outline of the problem statement is given below:

Participants are required to focus on issues that will realign, and influence the development and implementation of a strategic plan for the Department of Information Technology in three directions: Academic Development, Research, and Community Development. The focus of the session is to generate as many ideas as possible.
Both groups were given 40 minutes to complete this task. After 40 minutes had expired the participants were asked to stop whilst the facilitator ensured that the ideas were saved on the server. Instructions were then given to the participants to save their work on the local hard drives. 

In addition to the brainstorming task, the participants were requested to complete a post session questionnaire. The questionnaire attempted to measure member satisfaction with the process; process effectiveness as reflected by equality of participation; production blocking, and evaluation. The participants were then given ten minutes to complete the questionnaire. 
4.4	The Role of the Facilitator 
The facilitator’s role was limited to providing an explanation of the problem and what was required from the groups, i.e., explaining the task, re-explaining the objective of the sessions, explaining the principles of brainstorming and how the Team EC Brainstorming and the Evaluation and Choice modules work. The facilitator did not participate, deliberate, or contribute any ideas in the group session. This was in line with the purpose of the experiment, i.e., to investigate the role of facilitation, in the meeting stage of facilitation as defined by Ackermann (1996). The instructions appeared on the printed materials given to the groups before each group session.
4.5	Treatment
The control group X was given ten minutes to read the handout containing the problem statement while the experimental group Y was given fifteen minutes to read the problem statement and the additional information and objectives. The facilitator then explained the hardware and software environment. He then provided instructions on entering ideas and viewing the groups’ contributions. 

5.	MEASURES OF PERFORMANCE FOR THE GROUP GENERATION OF IDEAS

Two dependent variables were observed in this experiment: the number of unique ideas, and idea quality. These are discussed briefly:   
The number of unique ideas: These were determined by counting non-redundant ideas.  An idea was identified as unique when it added a new piece of task-related information.  Ideas that only agreed with a previous idea or shared the same meaning were not counted (Dennis et al., 1997).  Statements that did not relate to strategy formulation were excluded from the analysis. A final transcript of the ideas generated by the control group and the experimental group were collated and given for evaluation to an expert judge. The judge was blind to the groups in the sense that he was not provided with the details regarding which data set represented which group. 

Idea quality: The same judge also assessed the quality of each idea using the five point Lickert scale: 1 – very poor; 2 – poor; 3 – average; 4 – good; and 5 – very good. 

These quality measurements were combined to produce the following possible measures of quality as proposed by Diehl and Stroebe (1987), i.e., total quality, mean quality and number of good ideas. For the purposes of this study, all these idea quality measures were calculated. Brief descriptions of each of these measures follow:

* Total quality: This was calculated by summing the quality scores for each unique idea generated by each group. This measure rewards groups for all ideas they produce, even very poor ideas with low quality scores. This measure was found to be consistently reliable in various studies by Dennis et al (1997).

* Mean quality: This was the average quality of ideas generated by each group (i.e., total quality/number of ideas). This measure rewards groups that generate high quality ideas and penalise those that produce very weak ideas. Diehl and Stroebe (1987) did not regard the mean quality as being very reliable.

* Number of good ideas: Diehl and Stroebe (1987) argue that this measure is the most appropriate. It is calculated by counting only those ideas that were rated three or higher on a five point scale. According to them, this measure attempts to strike a balance between total quality and mean quality so that groups are rewarded for all the ideas they produce, except those that are poor (i.e., rated less than three).

Finally, the post session questionnaire assessed the subjects’ understanding of the experimental instructions, and their perceptions of effectiveness of and satisfaction with the process. Two components, the quality of the session process and the quality of the outcomes were measured. The quality of the process included a measure of the equality of participation through log files, the questionnaire and facilitator observations. In addition member satisfaction, production blocking, and evaluation apprehension were explored in the questionnaire for the purposes of comparison with other studies, e.g., Anson et al.,(1995) and Bostrom et al. (1993). The measures were adapted from DeSanctis et al (1989) and Gallupe et al.(1991).  
6.	RESULTS OF THE BRAINSTORMING EXPERIMENT ON IDEA GENERATION

The ideas generated by the control group and the experimental group were evaluated and moderated. The judge determined the total number of unique ideas, the total quality scores, and the total number of good ideas. The Total Quality Index and the Mean Quality of ideas were calculated for both groups. 

6.1 Effect on the Number of Unique Ideas Generated
Although the experimental group generated 28.20% more unique ideas than the control group, the question of whether the results are due to the additional information provided or merely because of chance, must be addressed. This is done by way of the t-test for two independent samples with equal variances (Polard, 1977).
 
Hypothesis 1: The experimental group with a clear set of objectives, additional information and an overview of the problem will generate a larger number of ideas.

Let X denote the control group sample and Y the experimental group sample. The assumption for their respective populations are that X and Y are two normally and independently distributed populations having the means mx and my respectively. The aim here is to prove that the mean number of unique ideas of the experimental group Y is statistically significantly higher than the mean from the control group X. This corresponds to the alternative hypothesis, H1 in the formulation below: 
Ho: mx ?  my 
H1: mx < my
where mx and my define the means of the X and Y data sets respectively. The null hypothesis (H0) will be tested using the one-sided t-test for the inequality of means by two independent samples. The critical value of the t-distribution with n1+n2-2 (n represents the number of participants in each group) degrees of freedom at 5 % level of significance is extracted from a statistical table. 

The critical region is the lower 5% region of the t-distribution that is less than – 1.746. If the t-statistic is smaller than t-critical, then the hypothesis is accepted. The test statistic calculated is equal to – 2.4300. Since, t < tcrit it is significant and the null hypothesis is rejected. Therefore, we concluded that by providing additional information and a clear set of objectives (corresponding to data set Y), a greater number of unique ideas on average would be generated, at the 5% level of significance. This leads to the acceptance of the original hypothesis: The experimental group with clear set of objectives and additional information will produce more unique ideas than the control group.

6.2 Effect on the Quality of Ideas
Hypothesis 2: The group with a clear set of objectives and an overview of the problem will generate better quality ideas.

For the second dependent variable, idea quality, and the following measures: total quality of unique ideas, mean quality and the number of good ideas were determined and analysed. Thus, the second hypothesis is examined in three different versions. For all three measures, the null hypotheses were tested using the same statistical method as above.
 
(a) For the Mean Quality of Ideas

The mean quality of ideas was found to be marginally higher for the experimental group at 4.48 against 4.45.

Hypothesis 2.1: The group with a clear set of objectives, additional information and an overview of the problem will generate better mean quality ideas.
 
The test statistic was calculated as t = - 0.0759. Since - 0.0759 > - 1.746, we conclude that this value is not statistically significant.  Therefore, we accept the null hypothesis, H0, at the 5 % level of significance. It can therefore be concluded that the mean quality of data set X is not lower than that of data set Y. This is contrary to the original hypothesis in that the provision of additional information, a clear set of objectives, and an overview of the problem did not result in a higher quality of ideas, when measured by the mean quality. This indicates that any difference in the mean quality of ideas of both groups may be due to chance factors. This is however in accordance with previous research carried out by Diehl and Stroebe (1987) indicating the low reliability of this measure.

(b) For Total Quality of Ideas

Hypothesis 2.2: The group with a clear set of objectives, additional information and an overview of the problem will generate better total quality ideas.

The test statistic calculated as t = - 2.0481 is statistically significant. Therefore we reject the null hypothesis, H0, and accept the alternative hypothesis H1 at the 5 % level of significance. It can therefore be concluded that the total quality of ideas for the experimental group was higher than that of the control group. This result is important because as mentioned earlier, it is known as the most reliable measure of quality, as it rewards both good and poor ideas (see Diehl and Stroebe, 1987).

(c) For the Number of Good Ideas

Hypothesis 2.3: The group with a clear set of objectives, additional information and an overview of the problem will generate a greater number of good ideas.

The test statistic calculated as t = - 2.4467, is significant. Hence, the null hypothesis, H0 is rejected, and the alternative hypothesis H1 accepted at the 5 % level of significance. It can be concluded that using the number of good ideas as a measure, additional information and a clear set of objectives produces more good ideas than the control treatment at the 5 % level of significance.

6.3 Post Session Questionnaire
The opinions of the participants were expressed through a 7-point Lickert scale ranging from very dissatisfied to very satisfied. The questionnaire measured member satisfaction with the process; process effectiveness as reflected by equality of participation; production blocking, and evaluation apprehension. An analysis of results indicated that the experimental group were marginally more satisfied than the control group. 

The first question sought to establish the participants’ views on the role of the facilitator. Both groups have indicated that they were completely satisfied with the explanation provided by the facilitator. Questions 2 to 5 sought to establish the participants’ views on the software and meeting environments. The participants of both groups indicated that Team Expert Choice was a good brainstorming tool and felt comfortable working with the software. 

The questionnaire also attempted to elicit the views of the participants about the issue generation (electronic brainstorming) process. Participants of both groups indicated that the idea generation process uncovered valid considerations that they did not initially think of and that the processes made them re-evaluate the validity of their considerations.

The issues of the effects of electronic meetings were also investigated via the questionnaire. According to both groups, Team Expert Choice promoted democracy and orderliness. It also encouraged co-ordination and equal participation.  

7. CONCLUDING COMMENTS ON THE IDEA GENERATION EXPERIMENT AND AREAS FOR FURTHER RESEARCH

This paper presented the findings from a field experiment on the role of facilitation in applying a GSS in strategic planning in an academic department. For two of the three defined measures, the results led to the conclusion that, the original hypothesis can be accepted at the 0.05 level of significance: The group with a clear set of objectives and additional information will generate better quality ideas. In a similar way it was proven that the same group generated a larger number of unique ideas.

The results of the experiments confirm the assumption about the positive influence of better facilitation and more information on the quality of group discussion. They comply with the findings of  Anson et al., 1995)  and Miranda and Bostrom (1999).

The process of electronic brainstorming was seen as a learning exercise for developing a deeper understanding of the categories in the strategic plan. However, of particular practical significance are the following important results:

* Facilitation (in the context of providing additional information in the form of printed materials) positively influences the thought processes and behavioral patterns of the group. 
* The electronic meeting system enabled a useful brainstorming session. It proved to be an important instrument for more effective organizational learning, creating the preconditions for a better strategic plan. 

The fact that Hypothesis 2.1 was not accepted may be due to chance influences or that the mean quality index is not a good indicator as has been suggested previously by the studies of Diehl and Strobe (1987).

The chosen approach to facilitation ensures that the same experiment may be replicated in a different organizational environment since the subjective factor i.e., the behavior of the facilitator is eliminated from the experiment. The level of detail of printed information can always be measured against some level of detail that is ideal for a particular task on hand. It is also possible to replicate the level of detail corresponding to the ideal level of instruction for a group. The latter was assessed after consultation with a number of managers who had several years of experience in educational management.
Most of the GSS facilitation studies to date have focused on a small subset of facilitator behaviours or have tested the facilitator role by applying very specific and prescribed procedures, rather than investigating facilitation processes or influences. The existing research on academic management like Fathi and Wilson (2009) focuses on the tasks in strategic academic management but leaves out the process of doing it. 
This research studied the influences of facilitation on group processes during strategic planning via a field experiment within a real academic department and not in a laboratory context and that was its contribution. Though the application area of academic management was not addressed before in GSS research, the results from this field experimental research have mostly confirmed previously published process findings on facilitation in GSS experiments dealing with other problem areas. It is recommended however that further studies in this area are required to allow better qualitative analysis to clarify the role of facilitation in GSS meeting environments and thus contribute to making electronic meetings more widely used in the future in diverse organisational environments. 

8. ACKNOWLEDGEMENTS

The authors are grateful to the anonymous reviewers for their comments which helped them in revising the paper.

9. REFERENCES

Ackermann, F. (1996).  “Participants’ perceptions on the role of facilitators using group decision support systems”, Group Decision and Negotiation, 5, pp 93 – 112.
Anson, R., Bostrom, R. P., and Wynne, B. (1995). “An Experiment Assessing Group Support Systems and Facilitator Effects on Meeting Outcomes”, Management Science, 41(2),  pp189-208.
Bostrom, R. P., Anson, R., and  Clawson, V. K. (1993). “Group Facilitation and Group Support Systems”, in L. M. Jessup, and J. S. Vallacich (eds.), Group Support Systems: New Perspectives, Macmillan Publishing Company, New York, pp 146- 168.
Dennis, A. R. and Valacich, J. S. (1993). “Computer Brainstorms: More Heads Are better Than One”, Journal of Applied Psychology, 78 (4) pp 531-537.
Dennis, A. R., Valacich, J. S., Carte T. A., Garfield M. J., Haley, J. H., and Aronson, J. E. (1997). “Research Report: The Effectiveness of Multiple Dialogues in Electronic Brainstorming,” Information Systems Research, 8(2), pp 203-211.
Dennis, A. R., Aronson, J. E., Heninger, W. G., and Walker, E. D. (1999). “Structuring Time and Task in Electronic Brainstorming,” MIS Quarterly, 23(1), pp 95-108.
Dennis A. and B. Wixom, (2002) “Investigating the Moderators of the Group Support Systems Use with Meta-Analysis”, Journal of Management Information Systems 2001-2002, Vol. 18, No. 3, pp. 235-257.
DeSanctis, G., Sambamurthy, V., and Watson, R. T. (1989).” Computer Supported Meetings: Building a Research Environment”, Large Scale Systems in Information and Decision Technologies, 13(1), pp 43-59.
De Vreede, G.J., Davison R.M. and Briggs, R.(2003). “How a Silver Bullet May Lose Its Shine”, Communications of the ACM, 46 (8), pp 96-101.
Dickson, G. W., Lee-Patridge, J. E., Limayen, M., and DeSanctis, G. L. (1996). “Facilitating Computer Supported Meetings: A Cumulative Analysis in a Multiple- Criteria Task Environment”, Group Decision and Negotiation, 5, pp. 51 - 72.
Diehl M., and Stroebe, W., (1987). “Productivity Loss in Brainstorming Groups: Toward the Solution of a Riddle”. Journal of Personality and Social Psychology, 53 (3), pp 497-509.
Fathi, M. and Wilson, L. (2009). “Strategic Planning in Colleges and Universities”,The Business Renaissance Quarterly: Enhancing the Quality of Life at Work, 4 (1), pp 91-102.
Gallupe, R. B., Bastianutti, L., and Cooper, W H. (1991). “Unblocking brainstorms”, Journal of Applied Psychology, 6 (1), pp. 137-142.
Griffith, T. L., Fuller, M. A., and Northcraft, G. B. (1998). “Facilitator Influence in Group Support Systems: Intended and Unintended Effects”, Information Systems Research, 9(1), pp. 20-36.
Khalifa, M., Kwok, R., and Davison, R. (2001). “GSS Facilitation Restrictiveness in Collaborative Learning”, Proceedings of the 34th Hawaii International Conference on System Sciences, Hawaii. 
Kwok, R. Chi-Wai, J Ma and D. Vogel (2002) “Effects of Group Support Systems and Content Facilitation on Knowledge Acquisition”, Journal of Management Information Systems/Winter 2002-X Vol. 19. No, 3. pp. 185-229
Miranda, S.M. and Bostrom, R.P. (1999). “Meeting Facilitation: Process Versus Content Interventions”, Journal of Management Information Systems, (15:4), pp 89-114.
Niederman, F, G-J deVreede, R.O Briggs and G.L Kolfschoten (2008), “Extending the Contextual and Organizational Elements of Adaptive Structuration Theory in GSS Research”. Journal of AIS,  Vol 9, 10/11, pp 633-652.
Pollard, J. H. (1977). A Handbook of Numerical and Statistical Techniques, Cambridge University Press.
Trauth, E. and L. Jessup (2000), “Understanding Computer-Mediated Discussions: Positivist and Interpretive Analyses of group support system”, MIS Quarterly Vol. 24 No. 1, pp. 43-79