The Value of Research Projects in Undergraduate Information Systems Degrees Brian W Hollocks The Business School, Bournemouth University Poole, Dorset BH12 5BB, United Kingdom Abstract The final year research project is often seen as a significant element in the "honors worthiness" of undergraduate degrees and a key indicator of individual student performance. However, from the institution viewpoint, such a project is relatively costly in terms of academic resources and hence, with student numbers increasing, may come into question. The paper discusses the value of the research project to student learning (with the Information Systems domain particularly in view), based on reflection by supervisors and students engaged in IS degrees at Bournemouth University Business School. Some distinctive outcomes are identified and it is noted that the inclusion of final-year research projects is supported by both students and academic staff. Keywords: Curriculum design, student managed learning, learning and teaching, information systems 1. INTRODUCTION The Organisation for Economic Co-operation and Development (OECD) defines research as "creative work undertaken on a systematic basis in order to increase the stock of knowledge, including knowledge of man (sic), culture and society, and the use of this stock of knowledge to devise new applications." (OECD 1993). This view may be mainly driven by graduate or post-doctoral work, but the definition does not preclude undergraduate research. Indeed, countless undergraduate programs include an individual research project as part of their final year. With steadily increasing student numbers and the relative unit cost of academic supervision, it is pertinent to consider the value of such projects and whether their contribution is distinctive. The discussion in this paper has in view, particularly, the value to Information Systems (IS) students and programs. 2. BACKGROUND Located on England's central south coast, Bournemouth University Business School has some 1500 students and provides a portfolio of vocationally focused programs of study at both undergraduate and postgraduate level. Most full-time undergraduate programs are of 4 years duration, with the 3rd year on placement in an appropriate post in industry, commerce or (less often) the public sector. There are also one-year "top-up" programs for students already having an appropriate qualification below honors degree level. The leading Information Systems (IS) program in the School is the full-time BSc in Business Information Systems Management (BISM), which has been running since 1988. There is also a top-up BSc in Business Information Systems (BIS), and the School's Masters level programs include an MA in Information Systems Management. IS teaching across all Business School programs is provided by staff from an Information Systems Subject Group. Learning and teaching utilizes conventional lectures, seminars, and laboratory-based workshops, with other participative and student-managed learning, plus normal technological support such as e-mail and conferencing. As well as subject content, the IS programs determinedly give attention to personal and professional development and transferable skills. This feature is strongly encouraged by the employers of the IS placement students and graduates. As part of the personal development provision, all Business School programs include an obligatory individual research project as part of the final year of under-graduate programs or of the final stage of top-up or post-graduate programs. (Having regard to professional development, the BISM program also includes a group development project, involving a team of typically four students in the design, development and implementation of a real system, albeit limited in size or scope, for a real client.) The focus of this paper is the individual research project of IS undergraduate programs. 3. THE RESEARCH PROJECT Figure 1 shows the present final year structure of the BISM degree with the research unit carrying 20 credits from a final year total of 120. (The term "research study" is used in this program in order to more clearly separate it from the group project.) Broadly, Bournemouth University guidelines identify a 20-credit unit with around 180 hours of student study time. However, student feedback from a sample of IS projects indicates that some students put in up to 300 hours or more. From the staff perspective, in terms of the time allocation allowed by the Business School in planning academic workloads, the research project is some four times more costly per student than a typical 20-credit, level-3 taught unit. With the Business School operating in an environment of progressively increasing student numbers, this is an important managerial consideration. Fig. 1: BISM Level 3 Information Systems Strategy 20 Credits Advanced Database Systems 20 Credits Advanced Networked Systems 20 Credits Information Systems Development (Group) Project 20 Credits Information Systems Research Study 20 Credits Option from: Information Systems Project Management, Management of Information Services, Public Sector Information Management, Computer-supported Co-operation & Decision Support 20 Credits The formal aims stated for the BISM research project are unsurprising: a) to further develop research capability; b) to provide students with an awareness of sources of information relevant to research issues; c) to allow students to pursue 'in-depth' research into an area of interest in relation to Information Systems; d) to develop students' ability to present coherently the results of a research project to a professional standard; e) to support the development of investigation and research skills. Many hundreds of these studies have now been completed. The remainder of this paper presents reflection on the learning outcomes, based on supervisor and student views. Supervisor views were polled informally. Student views were polled by questionnaire immediately at the end of their research project. The questions were: (1) In what ways do you believe you have personally benefited from carrying out the research study? (2) On reflection, what would you have done differently in carrying out the study? (3) How much total effort do you estimate you spent on the research study? (4) Would you have preferred to do another (specialist) unit instead, assuming it included an assignment and written exam? (5) Any other comments? 4. VALUE OF THE RESEARCH PROJECT Understanding "Research" The first major obstacle characteristically encountered by students is the concept of "research". Undergraduate students do not necessarily grasp at once the distinction of "discovery" from the more straightforward "analysis" which their systems education to that point has encouraged. (In the BISM context, this may be aggravated by students spending the preceding year on placement in the less rigorous world of industry!) The notion of a research process (even broadly, such as: Literature Review - Primary Research - Critical Discussion) is commonly an early revelation, including key steps that most students find conceptually new (in whole or part). A new terminology, such as epistemology and positivism, certainly sets the area apart - but not in an immediately transparent way. However, the whole weight of the process that the students undertake delivers an understanding of "research" that few of the students fail to grasp (although not necessarily quickly). Subject Selection Each student is assigned a personal supervisor for their project, but the student is responsible for choosing their subject. Students have no trouble in recognizing the importance of the goal, ie the choice and expression of the initial question or hypothesis, as the first step in the research project. This is the dominant concern for the student at the start: identifying an IS issue where they personally can add something original by providing an answer to the question or a test to the hypothesis. As an undergraduate they do not have, in normal research terms, a great deal of time to carry out the task. Therefore the topic must be, not only relevant to their course, but one which can realistically be pursued and written up within the available timetable - and with an effort commensurate with the project weighting in the degree. A good choice of topic sometimes looks detailed or minor to the student but, if an aim of the research unit in the course is for students to individually develop and demonstrate research skills, then the topic is simply a vehicle and only has to be adequate in its potential as a source of challenge and information. It is worth noting, at this point, a conflict of learning strategy. To some educationalists, the free identification of the topic by the student is a key part of the learning process, in contrast to the selection of the topic from a confined list or, at the extreme, the specific nomination of a topic by a tutor. However, self-definition is commonly a worry and delay for many students in the light of their very limited understanding of what constitutes a "good" topic. Indeed, those students clear and adamant about their personal topic frequently have an unsuitable choice - too large, complex or vague. A good compromise appears to be: (a) offer the students a choice from a range of fairly specific topics, (b) allow them to frame the topic more specifically for their circumstances, eg designate a case study company as one where they have ready access. In selecting and framing a subject, students need to carefully think through what they would actually have to do in order to answer that research question (even with an initially limited understanding of methodology and methods). For example, could they actually do it with the resources or access to sources of information that they will have available? In practice, some topics may be easier for them to pursue than others; there may be some field for which they have an inside track. (In particular, their placement company may be an excellent opportunity for easier access to people or information and provide a Case Study.) Notably, they seem to have little difficulty in maintaining an "IS" focus to their goal, as distinct from the technology itself, but there is a tendency for some to drift toward a business strategy or human resource management (HR) focus. Students unsurprisingly elect most often for subjects which they feel interest them, and probably do a better job of research as a result (in line with Cresswell's "personal interest" criteria for research (Cresswell 1994)). This process of goal definition proves a significant learning experience for students as they work through the remit, plan and ensuing consequences in the project. In the IS domain they increasingly note that the research subject can feature in their curriculum vitae (CV) and, even at undergraduate level, be helpful in securing a first job. Use of Literature Sources The broad notion of the research "process" introduces students to the idea of using literature to establish their topic's background and "state of the art". This is readily understood, but the natural tendency then appears to be to simply consult a few books on the subject - perhaps reflecting the "recommended texts" nature of earlier taught units to which they have been exposed. Students commonly require prompting to pursue depth and breadth in their reading, as well as to see further goals to this secondary research, such as: models or frameworks to structure their thinking or argument, the basis for the questions that they may ask in their primary research, or potential results from that inquiry. The notion of credibility as an aspect of literature sources needs to be emphasized. Students then quickly appreciate a hierarchy from peer-refereed journals, through trade press (frequently significant in the IS/ICT domain), down to web-sites - unless the latter are presenting information which is in one of the higher categories (eg electronic journals). However, students may then still take some time to actually engage with academic/peer-refereed journals in the relevant domain. The Internet/WWW has much to offer the researcher as a powerful means to identify information sources, but it is a major temptation to the undergraduate as "easy money". Of themselves, web-sites are the least credible information source - as a site could post any expression of a view without corroboration - and students can waste considerable time in being seduced by the detours and deviations the technology encourages. There are clearly now good on-line sources for searching for relevant literature, including indexes, bibliographies, journal contents, and electronic journals. However, students need to become aware of the population that the sources cover and, consequently, the potential bias in the search outcomes. The demands of the research process on the literature review lead to another significant learning outcome for undergraduate students, namely in the identification and critical review of subject material. The sources and disciplines involved are not likely to be developed by the students/graduate once in industry or commerce. Methodology and Methods An aim of the research project unit is the development and demonstration of research skills and a further key learning domain for students is the pivotal element of the research process, namely methodology and methods. In essence, research is about securely extending knowledge, even by a small amount. However, the basis of "securely" rests in such factors as validity, reliability and reproducibility (Saunders et al 1997) - and this notion of evidence comes only slowly to some students. They are disposed to simple assertions, rather than conclusively demonstrating their result through an objective process. They lose prejudice only slowly. (This is worst seen in mature part-time students.) Most students have little difficulty in appreciating the difference between "methodology" and "methods". However, the discipline of care and rigor in the process does not come as easily and is a major contribution in this aspect of learning. The ensuing execution of the methodology/method(s) requires the exercise of reasoning, another key component of learning (Walliman 2001). Experience in the Bournemouth University BISM degree shows a range of methodological approaches employed, including: Experimentation, Ethnography, Action Research, Grounded Theory, Survey and Case Study. The latter two are by far the most common which, given the context, is not surprising. Since their placement experience gives them potential access to an IS-active organization, the Case Study approach, well documented by Yin (1994) and others, has a particular appeal to students. In addition to care in subject definition and approach selection, students also learn care at a detailed level through the research project. For example, questionnaires are a common tool in research studies and it is not uncommon for students to take the design and implementation tasks lightly if left unguided. It can be necessary in project supervision to provide "blinding glimpses of the obvious", such as the student being clear about what they really want to find out. Also, many students seem to have little real dialogue in interviews, whether telephone or face to face, and do not get to the "bedrock" information. This may be partly a matter of confidence, but cross examination skills need to be cultivated and can be another material learning outcome. Report Writing The final deliverable from the BISM research project is a report of nominally 6000 to 8000 words, but typically extending to 10,000 to 12,000. This is another significant value domain. Very few undergraduate students have written such a lengthy report before - even at the lower of the levels mentioned above. The basic components of the research report are proscribed by the programs themselves, but students usually need encouraging to see that the various component parts are not independent sections. There should be a logical flow of argument in the report from Introduction to Conclusion, as if telling a story in chapters. This itself proves a learning experience. It requires a clear view and sound grasp of the topic and the secondary and primary research, so that a cohesive argument is presented. Most students take little pressing to start their report writing early. They appreciate that this spreads the load and reduces the likelihood of material being lost or forgotten. Encouraging a systematic approach, eg starting with the Contents section (helping them structure ideas and material), Introduction (getting the report writing "flow" going), and Methodology (articulating what they are doing), proves valuable. However, time still proves a problem for most students as work progresses (see Self-Management below). The biggest casualty of time pressure is the extent of discussion and reflection in the work. Students have little difficulty in distinguishing actual results, say from a questionnaire-based survey, from discussion of those results. However, they have more problem distinguishing what those results say from what they mean and what can be deduced from the overall research. The latter requires reflection across the whole content and process, with synthesis and judgment, not just analysis, and with appropriate use (or creation) of models or frameworks. This is a major learning benefit of the research project. Not all students secure this learning - it is a key differential between the good and the excellent. Again, potential areas of learning arise also at a detailed level. For example, results presentation is commonly dominated by tables, pie charts and histograms and students need encouragement to use more imaginative presentations (such as in Tufte 1983) - which may indeed lead them to further insights in their inquiry. However, this outcome could be achieved readily in other student course-work. The question "what does good look like?" is a reasonable one at the start of any new task. To give some flavor of general grading criteria for research reports (and, by implication, of the research itself), the author supplies his own supervisees with the personal view of criteria shown in Figure 2. Self-Management A further contribution from the research project is its stimulus to self-management skills on the part of the student - in particular, time management, priority assignment and self-discipline. Some regrettably leave components of this learning outcome rather late (if not too late). In a busy final year it is easy for students to give priority elsewhere, such as assignments. Time is readily lost in a research project, be it in hesitancy in subject selection, a lack of urgency in literature review, delays in primary research, or procrastination in report writing. It is difficult to recover without loss of quality. Most students have to be encouraged to think through what they need to do to complete the project, step by step, and produce a plan, meeting required delivery dates and other goals. They also need encouragement to plan the detail; for example: who are they going to ask what to, why, when and how?... what will they do with that information when they get it?... what are the potential obstacles? Milestones in the plan, with specific calendar dates and a focus on clear interim deliverables (such as draft literature search, survey plan, proposed questionnaire) are, of course, helpful within supervisory meetings. However, this does not come naturally with many students and is ignored by others. Students commonly underestimate the time required for some steps in the cycle, such as developing a questionnaire; gathering and analyzing data, and writing the report. The disciplines of planning, and the subsequent time/activity management to meet that plan (or otherwise), provide a major learning experience. A Sense of Achievement The research project is typically the largest, intellectually demanding, individual task that any of the students have ever done. Student feedback overwhelmingly reports a strong sense of achievement through the research project task with it adding materially to their self-confidence and their view of self-worth. This rôle is a possibly unanticipated benefit of the project but strengthening student confidence just prior to their final examinations and job hunting should not be disregarded! Subject Knowledge Students themselves reflect that the project gives them a level of subject understanding and insight that they would not otherwise have gained. Even within specialist taught units, their scope for personal inquiry and perspective is contained by the channeling effect of the syllabus. Ultimately, the insight of their literature review and primary research strengthens their prospects of employment in appropriate domains. This is particularly a feature of IS programs. 5. CONCLUSION The learning value to students of the Research Project appears to rest in a number of domains. * Understanding the concept of "research" * Goal/problem definition skills * Wider, more critical, perspective on literature sources * Development of research skills, in particular in methodology/methods and rigorous inquiry and reasoning * Report writing * Self-management and confidence * In-depth understanding of a particular domain (and thus support to possible career opportunities). The project has an integrative nature, both within its own focus and potentially bringing together material dealt with earlier in their program - some taken to greater detail, some providing perspective. It thus promotes "deep learning" (Ramsden 1992). It, further, is a material contribution to developing "independent learning" (Baume & Baume 1997). The nature of the project is such that it offers components that provide for the varying preferred approaches to learning reflected by Kolb (1987) in the Learning Cycle (Figure 3). Fig.3: Kolb's Learning Cycle Experience Testing ideas Observation & methods & reflection Concepts & theories The research project is very much an individual task and is generally seen as a significant element in "honors worthiness" at undergraduate level and a key indicator of individual performance. The evidence of past student achievement supports this view of the discrimination value of the project, as well as its development role. Such projects can be significant factors for potential employers comparing job candidates and, for the student, it is a marketable opportunity to consider if a particular IS/IT field is really of interest to them. Conspicuously, the learning outcomes identified above are not peculiar to the IS arena. However, the nature of IS, with its continuously changing technological environment and very broad-range of application fields, makes the research project particularly beneficial. In reflecting on the student view, it is noteworthy that, although the project is a very demanding task (intellectually, emotionally and in time-consumption) students emphasize that they would not prefer to have another taught unit instead. Further, academic staff typically like to be engaged in project supervision. From a wider perspective, such projects may, if focused appropriately, contribute information within the wider research interests of the staff of the IS academic group. At 200 hours or so per student, a substantial resource is available. The project may also promote in students an interest in research which they may then pursue in an MPhil/PhD. However, from the Bournemouth University Business School viewpoint, individual research project supervision is a non-trivial issue, with a staffing cost some four times that of a taught unit. With the increasing number of students, there are correspondingly increased pressures on supervisors. Some post-graduate programs in the School seek to decrease staff input by delivering collective methodology/methods lectures and seminars and carrying out supervision in "sets" with an action learning model. IS evidence indicates that the former has a limited impact on supervision demand (if any) and the latter tends to degenerate into individual supervision. Other actions may be considered to mitigate the staff/student cost, such as: * reducing the number of other units in level 3 of the degree to compensate for the project's demands * revising the content/structure of the research project to a more contained deliverable (eg using the "journal article" as the model, rather than the "research dissertation") * starting work in year 2 (although, to date, it has proved difficult to convince a year 2 student of the importance of something that does not bite for 18 months) * only offering the research project as an option. * abandoning the project as no longer viable. Most of the learning outcomes identified with the research project are distinctive - at least in terms of the depth or force with which they are delivered. However, the weight given to that product, and hence to the future of the research project, may be down to a local value judgement. 6. REFERENCES Baume C. and D. Baume, 1997, "The Art of Inspiring Independent Learning." The New Academic, 6, 3, Autumn Cresswell, J. W, 1994, Research Design: Qualitative and Quantitative Approaches, Sage Publications. Kolb D, 1984, Experiential Learning, Prentice Hall OECD, 1993, The Frascati Manual, Organisation For Economic Co-operation And Development, 2 rue André-Pascal, 75775 Paris, Cedex 16, France Ramsden P, 1992, Learning to Teach in Higher Education, Routledge Saunders, M., P. Lewis and A. Thornhill, 1997, Research Methods for Business Students, Pitman Tufte, E, 1983, The Visual Display of Quantitative Information, Graphics Press Walliman N, 2001, Your Research Project, Sage Publications Yin, R.K., 1994, Case Study Research: Design and Methods, Sage Publications. The Value of Research Projects in Undergraduate IS Degrees 2