Introduction to Business Systems Development Students Perspective of a Problem Based Learning Approach John F Bentley Glenn R Lowry Geoffrey A Sandy School of Information Systems, Victoria University of Technology Melbourne, Victoria, 8001, Australia John.Bentley@vu.edu.au Abstract This paper presents the results of a qualitative research study aimed at understanding students' perspectives of a trial of Problem Based Learning (PBL) in a traditional information systems course structure. Issues that arose and difficulties that were encountered by students are discussed. The major themes identified from the students' perspective of the Problem Based Learning approach were: improved problem solving, improved time management, self learning, improved research skills, improved group work skills, and the use of realistic problems. Issues included a focus on factual knowledge, problems encountered with group work participation, weaker students requiring more direction, and preparation and motivation for PBL. Keywords: Problem based learning, computing education, information systems education 1. INTRODUCTION Curriculum issues relating to content are frequently discussed in information systems arenas. A model information systems curriculum has been revised and issued as IS'97 under the auspices of the ACM (Association for Computing Machinery), AIS (Association for Information Systems) and AITP (Association of Information Technology (formerly DPMA)). Rarely is there discussion regarding the appropriate delivery of the curriculum. To our knowledge, problem based learning (PBL) has not been meaningfully adopted for information systems education, though a final year project is incorporated in many undergraduate information systems courses and a number of courses use case study methods. Problem Based Learning seems to offer an approach that provides a closer match of the learning approach with the world of work of the Information System Professional. Stretton (1985, p.60) states from an industrialist's viewpoint I believe that if we were able to bring education and practice closer together, many of the current dissatisfactions in both camps [education and industry] would be eliminated ... Why should education and practice be separated at all? Education is a facilitator to the learning process. Learning and living are inexorably interlinked. Therefore education and living should be similarly interlinked. Their current separation is a legacy of specialization and reductionist approaches of the past ... It seems to me that problem based learning could (should?) be the most appropriate vehicle for closing the gap. There is an increasing body of knowledge in information systems. This results in more content being crammed into an already bulging curriculum. At Victoria University in the Faculty of Business and Law most of the teaching and learning is teacher-centred and course based, the dominant mode being lecture-based delivery on a two-hour lecture and a one-hour tutorial/workshop model. In a traditional approach to teaching there is an emphasis on students acquiring knowledge rather than the development of critical thinking and the problem solving skills required of an information systems graduate. What we have is essentially a behaviouristic, psychometric approach to learning and assessment. Jones (1990) suggests this leads to unrealistic workloads for students that may be intellectually unchallenging, encouraging passive learning that is unlikely to motivate many students. Lecturers in the School of Information Systems have attempted strategies to enhance student outcomes. However, these are often isolated, individual efforts that, while laudable, probably do not encompass a broader curriculum perspective. There must be a better way to develop graduates more suited to the practice-based, project-oriented world of the information systems professional. We believe that PBL is one approach that offers a match between curriculum delivery and the professional information systems work environment. The School of Information Systems commissioned an exploratory trial of problem based learning, related in the sections that follow. 2. METHOD Initially the trial was to be a comparison of students in normal delivery mode against a group following a PBL approach via a pre-test post-test experiment. In subsequent discussions with educationalists in the university it was decided to modify this approach to avoid serious methodological questions. There were only a small number of students in the trial and the problem of cross interference between groups was an issue. The main intention of this limited trial within a single subject became for the researchers a way to become familiar and learn how PBL works within information systems courses. This pilot study provides the foundation for a full study of the option of PBL within the information systems curriculum. Trial participants were first year students undertaking "Introduction to Business Systems Development". This introductory systems analysis and design subject lends itself well to a PBL approach as it involves the presentation of problems requiring use and development of PBL skills to solve systems development problems and opportunities. Students self-selected into the PBL group. It is acknowledged that this limitation, which was accepted for ethical reasons, may bias any comparison with a non-PBL group in that many of the PBL students may have a predilection for problem solving. The group was viewed by the instructors prior to commencement of the study, to be an anomalous and generally weaker group of students, located at an outer campus of the university where lower entrance requirements apply. Students were provided with access to computer learning resources such the as the Internet, Intranet, self-assessment tests, discussion server and e-mail, as well as traditional information sources including textbooks and libraries. PBL students completed weekly planning sheets to guide their learning for the following week. The PBL students were required to submit weekly reflective learning diaries (Appendix 1) in which they recorded their reflections on their learning and activities related to the subject. The trial was further constrained by an Ethics Committee requirement that students in both groups receive the same assessment. PBL assessment is different from traditional assessment. In PBL the process is assessed as well as the content learning and outcomes. There has been much discussion in the PBL literature regarding the forms of assessment and evaluation of PBL students. This study used a qualitative approach. A phenomenological research approach as described by Patton (1990) and Leedy (1997) was undertaken. Phenomenology as defined by Leedy (1997, p.161) is "a research method that attempts to understand participants' perspectives and views of social realities". The study sought to understand students' perceptions of the effect of problem based learning as an approach to learning. Student perceptions were obtained from face-to-face interviews, focus groups, tutor observations, course evaluation forms, and examination of diaries and planning sheets. This approach is similar to that used by Coll and Chapman (1998) who interviewed ten students in an investigation into students' perceptions of the benefits and barriers of an international co-operative education programme. Seven individual interviews were conducted with PBL students. This is consistent with Leedy's view (1997, p.162) that a phenomenological approach "typically involves 5 to 10 in-depth interviews". The interview structure was designed to evoke more personal and in-depth responses and to allow the interviewers to probe into issues raised. Questions were open-ended to elicit personal responses and experiences. All the PBL students were invited to participate in a focus group. Only three students attended the focus group, in part due to a requirement to return to the campus after examinations had finished. Ottewill and Brown (1999) discuss the use and issue of low participation by students in voluntary focus group sessions. Only one of the focus group participants had been though an in-depth interview. Each interview was approximately 45 minutes in length. The focus group and interviews were audiotaped. The tapes were fully transcribed. The content of the transcripts were analyzed, looking for shared themes, experiences, key words, and phrases relating to perceptions of PBL. Further data were obtained from reflective diaries, tutor observations, academic results and the weekly planning sheets. Each of these is discussed below. 3. RESULTS The major themes identified from the student's perspective regarding the PBL approach were: * Improved problem solving; * Improved time management; * Self learning, improved research skills; * Improved group work skills; * Realistic problems relevant to the work of an information systems professional. Issues arising included: * A focus on factual knowledge; * Problems with group work participation; * Weaker students requiring more direction; * Preparation and motivation for PBL. Improved problem solving Though the improvement was not strongly articulated by most students, most students believed that their approach to problem solving had improved. Jane reflected "I learnt how to solve problems and that was another aim about going for the PBL approach. I had a problem myself with looking at a problem and breaking it down." Some students were using the approach to tackle problems in other subjects. As Bob said, "... the process of defining the problem that you showed us, to me was very valuable, in that I started thinking to myself not only in this but in every assignment for the different subjects. The first thing that I did was sit down and define the problem." He also stated "... the problem based learning type techniques I have found have been valuable to me looking at the next set of problems outside of this particular class, I'm getting that value because I can see the value in it ..." Improved time management Nearly all students reported that they believed their time management had improved. They attributed this to the use of planning sheets, diaries and working in groups. As one student stated in their course evaluation form that one of the best aspects of the course was "Being able to plan and reflect on my work and achievements every week." Mary also reflected this stating, "Because you have goals and actually achieve them". Self Learning All students in their diaries and in the interviews perceived that the onus was on them to be responsible for their learning, though at some stages they did not see this as positive and would have preferred to be "spoon fed". Mary stated "with my work it has made me more independent, instead of always running to the teacher, learning stuff for yourself" and Joe said, "the difficult thing was coming to realize that you have actually got to learn". Rex summed up many students' feelings when he said in the interview: "I understand that throughout the semester like we would ask questions or raise some issues and I understand that you didn't as the administrator you didn't want to tell us the answer 100% full stop. Because that would bring it back to a lecture/tutorial kind of thing, I understand that, but I think that might have added, probably as you might expect added to the confusion that the students were feeling, like I wish he would just tell me, because in other subjects they do just tell us. But by the same token I appreciate that you didn't tell us because then I had to work it out for myself and it actually stuck." Realistic problems It was observed as the course progressed that students commenced problems enthusiastically and were engaging the problem situation. Students were asked their views about the problems used. Bob, a mature aged student, said that this was "Engaging to me because I could see the real life scenarios happening and I had no real problems with the way that they were presented nor did I with the way or the tasks that we were being asked to do within the problems." As Henry insightfully noted, "the problems were situations that seemed as though they could happen, not like some cases [in other courses] where you could tell they are artificial." Improved research skills Most students perceived an improvement in research skills. Max stated "In PBL you have to explore yourself ... in other lectures you have got everything you just take it and write it down, and this (PBL) makes you go into more, you have to look for it yourself". Focus on factual knowledge as learning Even though the students were initially enthusiastic about the change in the approach to learning, some became skeptical about the approach as they were still focussed on factual learning. This perhaps indicates that they have to unlearn prior conceptions of the educational process and their part in it to better apply the PBL approach. As Rex reported, "it took us a while to find our feet but when it did I thought it was pretty good". The students seemed to have difficulty with the fact that content knowledge was not being emphasized as in other subjects. Early in the trial the diaries contained frequent questions or comments relating to "have we learnt enough to pass the exam?" and "will the pace of learning pick up?" The students did not seem to perceive the improvement they were making in process skills as being a significant element of their learning. The students' prior concept of learning seemed to affect their acceptance of PBL early in the trial. This might have been expected, as students may be reticent to change from a learning style with which they have developed a level of comfort for twelve years of their life. Bob summed up students' feelings saying, "A lot of people towards the end of the PBL felt that they weren't learning enough or didn't have the knowledge compared to the people that had done the lectures." However, this was not bourne out in the examination performance of the two groups. Motivation and preparedness for PBL From a tutor's perspective there was more time than expected spent on developing students' learning strategy and problem solving. Some students struggled with the approach whilst others blossomed and pursued other areas of interest in analysis and design. Students' reasons for undertaking the course varied from "I was sick of just sitting in class, sick of being there like for two hours, three hours and you just go to sleep", "a new way to learn", to "thought it would improve my problem solving skills". Rex, contemplating his preparedness and reason for undertaking PBL said, "If I had been a little bit weak in my subjects in the first semester I probably wouldn't have tried it. I thought that if I'm struggling anyway I don't think that I want to try and innovate too much. But I did well so I thought I would give it a go." We may need to consider student's attitude and maturity for PBL, though in this group it may be that as they are a weak group that may not possess the attitude and motivation required for effective tertiary study. Jane, reflecting on the group, suggested "Goes back to attitude. Mature students need to be in PBL for sure because you have the people that take it seriously, and ones that don't care and it just ruins everything." Group work Collaborative learning and group work was sometimes seen as difficult but students began to realize the benefits. A significant level of bonding was observed between students and their collaborative approach to learning in both the PBL course and other courses. Students suggested that group work allowed them to interact with others and to open up. They stated that they have begun to learn they have responsibilities and that they can let others down. A student reported, "group interaction - working with others" as one of the best ways that the subject was taught. Communication between group members was frequently mentioned in diaries and interviews. Some students were quite strong in their insistence that others should use email and discussion lists. Even though the students were enrolled in a computing course they were not making use of the university email system. This may be due to the current state of the system as Joe said "I really don't use the student service (e-mail system) that much because I find it not very appealing. I don't check email". Joe uses a free provider for his own personal email. However the students did make extensive use of the Internet for fact-finding. Group work was seen by some students as being able to allow them to open up. Max, a very quite individual who said little in the early part of the semester, stated "... you have to get really involved in it. But if you are shy it is probably a good subject to take because you have to get involved." Henry, who in other courses generally keeps to himself, energetically said "I got out of myself by having to work with others." The following series of extracts from Jane's weekly reflective diary to the questions "How well do you feel your group is progressing?" and "How could group interaction be improved?" illustrate typical feelings towards group work and PBL. Week 2 - In the first stage of understanding PBL I can see that this group will work well together as we have all shown initiative and have all worked together to collate ideas and obtain knowledge from each other. Week 3 - I feel the group work is progressing well. However some members do remain skeptical of the whole PBL. I feel if everyone had the same attitude then it would work really well. Week 4 - All group members are trying hard to work together. We try to meet in our own time. I think everyone is becoming more comfortable with idea of PBL. Week 6 - When trying to organise times to meet there is always some sort of reason as to why we can't make it. I am really getting sick of people not being enthused and not taking PBL seriously. I feel we are running out of time and by now we should have found group work more easier to grasp. ... Need to use email a lot more. Need to organise more meetings. Week 7 - I think everyone is accepting the group work aspect of the subject and most are trying their best. Week 8 - More use of email, and everyone needs to attend meetings because when we are assigned specific tasks it is unfortunate that some can spoil it for others. Then those that do complete their work and attend get put off and unmotivated. Week 9 - Group work has come a long way. We have spent a lot of time with working together. Week 10 - Most members' attitudes have changed and they have found that there is a need to participate and learn from each other. Week 11 - We are all trying really hard and for those that do put effort have achieved satisfactory results. ... Need to communicate with each other as much as possible and meet more regularly. Week 12 - Those that are working together are cooperating and acknowledging each other's work. To have a successful group all members need to participate, communicate and work together. It is not happening with everyone. 4. DISCUSSION Introducing PBL into the second semester of first year probably means that students have already framed their perceptions of university education by sitting through subjects in semester one that consist of lectures and tutorials. It might be better to introduce PBL or elements of PBL in semester one for commencing students so that they can perceive that university education is different and not necessarily focused on content learning. This will also mean that they will be able to develop their self-teaching and collaborative skills more gradually. One major problem regarding group work reported by students was the poor attendance by members to meetings outside of regular class times. It is suggested that timetabling an extra hour where students can meet for non-tutor presence group meetings should be considered. In PBL it is important to identify weaker students and to provide them with encouragement and feedback so that they are not left behind. Three students clearly identified that PBL was not an approach that suited them. They wanted more direction and structure imposed on their learning. Two of these students decided to undertake Technical and Further Education courses in 2000 rather than higher education studies. The weakest students suggested that they preferred directed learning. Specifically, they desired that resources such as set tutorial questions and that they need to be pushed or forced to do the work. An unintended consequence of PBL may be a self-realization to assist students in identifying the best approach to learning that suits them and whether they wish to be professionals or technicians as their possible career path. 5. CONCLUSION This paper has presented some of the perceptions of undergraduate students in an information systems subject encountering PBL for the first time. Many of the issues and perceptions in this paper have been reflected in other disciplines (Chen 1991, Duke et al 1995, Mackenzie 1997, Warburton & Whitehouse 1998, Dahlgren & Dahlgren 1999). This information can be read and digested but until academics actually experience PBL and the nuances of PBL for particular students in an information systems undergraduate situation the issues cannot be fully appreciated. PBL is an approach that might better prepare students for the world of information systems professional practice by aligning curriculum delivery with information systems practice and will be investigated in further studies. This trial has been an encouraging first venture. In summarizing the impact of PBL, Jane in a recent selection process she had undertaken for a professional information systems position captured the value of the PBL experience: "They gave us a team work problem and we had to sit there and work it out. There was a candidate there who jumped right into it and I just broke it down and did the things that we learnt in PBL and it came across well ... the problem that they gave us was similar to the problems that we had [in PBL]." Jane was offered the position in the company. 6. REFERENCES Dahlgren, M. A., and Dahlgren, L.O. (1999). Portraits of PBL: students' perspectives of problem-based learning in three academic settings. In Conway, J., Melville, D. and Williams, A. (Eds.), Research and Development in Problem Based Learning. Vol 5. Sydney: Australian Problem Based Learning Network, pp. 137-148. Chen, S.E. (1991). Profile of an integrated problem-based architecture course: students' perceptions and preferences. The technology of design. Proceedings of 1991 ANZA School of Architecture Conference, Adelaide, pp.175-187. Coll, R. K., and Chapman, R. (1998). International co-operative education placements: Benefits & barriers. Paper presented at the Asia Pacific Conference on Co-operative Education, China. Duke, M., Forbes, H., Hunter, S. and Prosser, M. (1995). The perceptions of problem-based learning (PBL) held by undergraduate students of nursing: A progressive analysis. In Little, P., Ostwald, M. and Ryan, G. (Eds.), Research and Development in Problem Based Learning. Vol 3. Sydney: Australian Problem Based Learning Network, pp. 137-148. Jones, J. (1990). Reflections on the undergraduate curriculum. In Moses, I. (Ed.), Higher Education in the late twentieth century: Reflections on a changing system. Saint Lucia: University of Queensland Press. Leedy P. D. (1997). Practical research: Planning and design. (6th ed.). New Jersey: Prentice-Hall, Inc. Mackenzie, L. (1997). Perceptions of occupational therapy students about the nature of their problem based learning occupational therapy curriculum. In Conway, J., Fisher, R., Sheridan-Burns, L. and Ryan, G. (Eds.), Research and Development in Problem Based Learning, Vol. 4. Newcastle: Australian Problem Based Learning Network p.366-371. Ottewill R. and Brown D. (1999). Student Participation in Educational Research: experimenting with a focus group. Journal of Further and Higher Education. Vol.23. No. 3. pp.373-380. Patton, M. Q. (1990). Qualitative Evaluation and Research Methods. (2nd ed). Newbury Park: Sage Publications, Inc. Stetton, A. (1985). Problem Based Learning and the Academic-Practitioner Gap. In D. Boud (Ed.), Problem-Based Learning in Education for the Professions (pp. 239). Sydney: Higher Education Research and Development Society of Australasia. Warburton, B., and Whitehouse, C. (1998). Students' perceptions of a learner-centred approach using problem-based learning on an undergraduate general practice course at the University of Manchester, Medical Teacher, Vol. 20, Iss. 6, pp.590-591. APPENDIX 1 Reflective Diary Questions Student Number: Assignment/Workgroup Group: 1. What new thing(s) have you learned in the past week? 2. What reading have you done in the past week? 3. How well did you follow your individual plan for the week? 4. What was good about your planning and how could it be improved? 5. How do you feel the group work is progressing (towards problem solution)? 6. How could group interaction be improved? 7. What have you contributed to the group in the past week? 8. On a scale of 1 to 10, how confident are you currently feeling (in terms of your knowledge and skills) about: a) Microsoft Access 1 2 3 4 5 6 7 8 9 10 b) Strategies for systems analysis and design 1 2 3 4 5 6 7 8 9 10 c) Systems development techniques (data modelling and process modelling) 1 2 3 4 5 6 7 8 9 10 d) Data gathering and fact finding techniques 1 2 3 4 5 6 7 8 9 10 e) FAST methodology 1 2 3 4 5 6 7 8 9 10 f) System documentation 1 2 3 4 5 6 7 8 9 10 g) Project management 1 2 3 4 5 6 7 8 9 10 9. Other comments or things you wish say (optional) 10. About how many hours outside of the tutorial did you spend on activities related to this subject in the last week? Less than 1 1 2 3 4 5 6 7 8 9 10 11 12 or more hours The on-line diary form and other course material are available at: http://www.business.vu.edu.au/bco1048 1