Learning Outcomes in Use 
– Project-Oriented Education from a Teacher’s Perspective

Tommy Wedlund
tommy.wedlund@liu.se
Department of Management and Engineering, Linköping University
Linköping, Sweden

Karin Axelsson
karin.axelsson@liu.se
Department of Management and Engineering, Linköping University
Linköping, Sweden and

Swedish Business School at Örebro University
Örebro, Sweden

Ulf Melin
ulf.melin@liu.se
Department of Management and Engineering, Linköping University
Linköping, Sweden

Abstract

This paper describes how a teacher can apply learning outcomes in a syllabus in project-oriented education. We present these learning outcomes in the context of a project-oriented course and give practical examples of how a teacher can apply the examples by coordinating (1) learning outcomes in the syllabus, (2) learning activities in the course, and (3) processes in a project-oriented course. The examples are applied to processes in a project-oriented course, including the students’ individual learning process, project team process, and project process. Involving students early on, at the start of the course, by presenting the learning outcomes and then continuously working with and referring to them gives the students a learning structure. The course starts with learning outcomes building on team experience and knowledge, i.e., the project team process. After that come learning outcomes for the project process, making the project the carrier of new knowledge for the team, and finally learning outcomes ending in the students’ individual learning process, by which the students can deepen their learning, applying their knowledge in new situations and various learning activities. Project-oriented education was used in an undergraduate course at a Swedish university; experiences from this course are discussed and analysed here.

Keywords: Learning outcomes, syllabus, course design, learning, project, information systems education, student-centred education


1. INTRODUCTION
A cornerstone of the Bologna Process (European Association for Quality Assurance in Higher Education, 2005) is the development and publication of explicit, intentional learning outcomes. Defining and using the learning outcomes (e.g., in a syllabus) can then be the starting point for designing, starting, executing, and ending a course. Learning outcomes are statements expressing what a learner is expected to know, understand, and/or be able to demonstrate after completing a learning process (European Commission, 2004). 
This paper focuses on the difficulties of teaching according to how students learn and how to facilitate the explicit learning outcomes in the syllabus. It describes and analyses how, in practice, to apply learning outcomes in three simultaneous processes in project-oriented education (Wedlund et al., 2006) and gives examples of the learning activities in a course that support these learning outcomes, activities undertaken during course design, at the start of a course, during course execution, and at the end of the course. Good teaching supports those learning activities that lead to the attainment of the explicit, intentional learning outcomes. “Learning activity” refers to an activity that supports student learning. All nine learning outcomes in the cited syllabus are presented in the studied course. In this paper, “teacher” refers to a person with overall responsibility for a course; a teacher may be an individual course leader, an instructor team, or an examiner, depending on the type of university course.
This paper touches on topics relevant to university teachers who teach courses that include student projects, such as general courses touching indirectly on information systems (as often is the case in information systems courses; Schwalbe, 2005; Marchewka, 2006) and specific information systems courses.
The paper is arranged in the following sections: section 2 presents the research approach, section 3 describes the theoretical frame, section 4 the empirical examples, and section 5 presents concluding remarks and identifies areas for future research.

2. RESEARCH APPROACH
In order to elaborate on the problems discussed above, we have conducted a R&D project. A major point of departure of the present R&D project, named “A learning outcome model – reflective assessment”, is learning outcomes as part of the Bologna Process, focusing on keywords such as knowledge, understanding, ability, skills, assessment, and perspectives. When applying learning outcomes in higher education courses, the need to assess student achievements in light of learning outcomes is considered to be a key issue. The use of learning outcomes has great potential, though it poses several challenges. Assessing student achievements in the present project calls for, among other things, the development of a framework. This framework would relate to learning outcomes from multiple perspectives, such as employability, student learning outcomes, research, and subject-oriented profiles. The project is grounded in and related to didactic practice and pedagogical research. Though the R&D project is conducted in the information system subject area at four Swedish universities, it is relevant to other subjects of similar kind.
The research approach is qualitatively grounded in a case study (cf. Yin, 1995), both in theory and in the knowledge areas (Kolb, 1984; Tuckman, 1965; PMI, 2008) of the three processes. The context of this paper is an undergraduate course including a development project at the Department of Management and Engineering, Information Systems, Linköping University, Sweden. 
The results presented here come from a new course that includes an IT project. The students’ program was re-constructed in 2008, and students can now choose between three different program profiles, which is the reason why this course was developed, designed, and implemented. This course, the students’ second IT project course (their first was in their first year), is a ten-week undergraduate course. The course was evaluated by both students and teachers, who focused on the learning outcomes of the course. Evaluations were conducted at the beginning, in the middle, and at the end of the course. Three teachers and one assistant facilitated the students in the course. The teachers have at least fifteen years teaching experience, and the course obtained high evaluations from students.
3. A PROJECT-ORIENTED COURSE
Teachers must develop a deeper understanding of how learning outcomes can be used in project-oriented courses. This could be done by developing a thorough understanding of how to work with the learning outcomes. Considerable knowledge and practice can be applied in working with learning outcomes in project education, for example, knowledge of project life cycle phases and the development of team member skills and knowledge (Cederling et al., 2000). The Project Management Institute (PMI) (2008, p. 5) defines a project as “a temporary endeavour undertaken to create a unique product, service, or result”. A project-oriented course normally comprises a range of processes, each containing certain stages, steps, or phases. Each phase comprises various activities, which are components of the work performed in the phase. A project-oriented course can therefore be divided into different time periods, often corresponding to the same phases as the generic project life cycle. The project life cycle defines the phases that connect the beginning of the project to its end. These phases make the project more measurable and manageable, as in Figure 1, which presents the following five phases: project definition, project planning, project execution, project completion, and project evaluation.
Project-oriented education includes skills training in which students improve their skills while completing the activities in the processes included in the course’s project phases. These skills can be categorized as of two types, depending on the decisions made about what activity to do and how to do it. This is also an important issue in industrial projects, such as global product development projects (Wedlund, 2000). Key considerations in project-oriented education include how teaching methods, project environment, and student learning backgrounds predict the quality of learning outcomes. Learning outcomes can be used when designing the course and completing course work. One purpose of a project-oriented course is to foster student-centred learning (Gibbs, 1995).
Ramsden (2003) argues that higher education is part of what he sees as a global shift to a new way of creating and using knowledge. Project-oriented education is a good example of this new way of creating and using knowledge. It encourages teachers to use more knowledge areas than they would in a normal university course. The knowledge areas of the three processes, however, usually belong to different faculties in a university. Project-oriented education also encourages a deep learning approach (Biggs, 2003) on the part of students. The teacher designs learning situations that motivate students and encourage them to be active in their own learning processes. The students then participate in activities in the processes, in problem solving, and in summarizing and digesting new information to change how they think about and use information in fundamental ways. The information is used in various phases in the project while the students are working on analysis, design realization, and testing in the course. 
The empirical case used in this paper comes from a continuing education course dealing with IT projects. The main project phases in the course are analysis, design, realization, and testing. In the course, the students develop an object-oriented website for e-commerce. The course is provided full-time over a ten-week period and is taken by approximately 30–60 students each year. The course components mentioned are integrated naturally during the course so that students become familiar with and gain abilities in the various phases of a systems development project.
3.1.	The students’ individual learning process 
The students’ individual learning process is the first process involved in project-oriented education. Kolb (1984) created his model, inspired by Lewin (1951), out of four elements: concrete experience, observation and reflection, forming abstract concepts, and testing in new situations. Kolb argues that the learning cycle can begin at any one of the four steps and that it should be approached as a continuous movement (see Figure 2).

However, we suggest that Kolb’s learning cycle often begins when, for example, a student carries out a particular action and then sees the effect of the action in a new situation. This process often begins with students’ concrete experience. A teacher who has learned to teach in line with this cycle may well have various “rules of thumb” about what to do in various learning situations. It is important to start with students’ “here and now” experience. In a project-oriented course, written and oral feedback is given to students at project milestones, normally every second week. A workshop, designed by the teacher based on the students’ concrete experience, is used. If learning has taken place, the process can be seen as a spiral. Once the action has been completed in a different set of circumstances, the teacher can anticipate its possible effects. The teacher’s earlier experience is crucial, particularly when students solve problems in the information systems area.
3.2.	The project team process
The project team process is the second process involved in project-oriented education. Tuckman (1965) described the four stages of project team (i.e., group) development. He recognized the distinct phases the group (students in this case) passes through, and suggested that a group must experience all four stages before achieving maximum effectiveness. In project-oriented education, project teams comprise approximately four to six students. Normally the students go through two stages of group development; if the project team is successful, students can sometimes reach the third stage. The four stages of project team development are forming, storming, norming, and performing (ibid).
In stage 1, forming, students want to be accepted by other students and avoid conflicts. Serious issues and feelings are avoided, and students often focus on project details, such as who does what and when to meet. However, students are also gathering information and impressions about each other and about the project scope. In stage 2, storming, students in the project team become less careful and formally polite with each other as they start to address important issues. Some students’ patience breaks early on, and minor confrontations arise. These may relate to the project work of the team itself, or to roles and responsibilities established in the project plan. Every project team normally reaches this stage. In stage 3, norming, the rules of the project team, which evolved in stage 2, become established, and the scope of the project team’s tasks or responsibilities is clear and agreed upon. Having had their arguments, team members now understand each other better and can appreciate each other’s skills and experience. Students listen to each other, appreciate and support each other, and are prepared to change pre-conceived views. Few project teams reach this stage. Stage 4, performing, is very seldom reached by project teams and is characterized by interdependence and flexibility. Everyone in the project team knows the others well enough to work together, and they trust each other enough to allow independent activity. Project roles and responsibilities change according to need almost seamlessly. Project team identity, loyalty, and morale are all high, and everyone is equally task and student oriented. “Improved performance” in Figure 3 indicates that the project team is able to reach further developmental stages, while “worsened performance” means the opposite. It normally applies when a project team reaches stage 2, storming.
3.3.	The project process 
The project process is the third process involved in project-oriented education, based on a project team’s areas of expertise as defined by the PMI (PMI, 2008). In this case, the teacher corresponds to the project team. The components that must be understood are the project environment, project management knowledge, and application area (see Figure 4).
All projects are executed in particular project environments, which comprise information about the applicable organizational culture. This culture is reflected in factors such as stakeholder beliefs and project expectations, business processes in the organization, goals, problems and strengths, and views of authority (e.g., the project sponsor). Project management consists of unique knowledge described in nine knowledge areas (PMI, 2008): integration, scope, time, cost, quality, human resources, communications, risk, and procurement. In total, there are 42 project management processes, important ones including developing the project management plan, monitoring and controlling project work, verifying scope, and controlling costs (ibid.).
Each application area is generally governed by a set of accepted standards. The course, an information systems course in the present case, is one such application area. The standards applicable to an information systems course are usually defined in terms of information systems development work (e.g., in software development in the studied course or in engineering courses in other cases). In the studied course, students use unified modeling language (UML) and rational unified process (Kruchten, 2003). Rational unified process consists of the inception, elaboration, construction, and transition phases. With “UML-like” diagrams the students then create objects, and interact with their methods in the software tool. This creates the opportunity for direct experimentation with objects, which makes it easy to understand. In university courses in information systems, students commonly use systems development models, such as rational unified process. The milestones of such models are life-cycle objectives, life-cycle architecture, initial operational capability, and product release.
4. MANAGING LEARNING OUTCOMES
This section presents examples of how a teacher can manage learning outcomes involved in the three processes in a project-oriented course. It is important to provide information about the intended learning outcomes for a course, and the learning opportunities that are available for the students. This information from the course means that teachers have to take careful attention to curriculum and programme design and content. In this paper it is empirical examples, from the studied course. The start of the course is about two weeks long; after that follows course execution, which takes about six weeks in this case. Finally, the course wraps up over two more weeks. Figure 5 summarizes the processes related to the course flow. It is a generic figure; project execution may take more or less time in another course in which students work on a project.
The learning outcomes in the syllabus have been translated from Swedish into English, and all nine learning outcomes of the course are presented in tabular form below. The three teachers in the course have developed and implemented the learning outcomes in the course. The students have evaluated the learning outcomes during the whole course.
4.1.	At course start
At the start of the course, the students must gain an overview of the learning outcomes of the course. One way to do this is to present the three processes involved in project-oriented education, and to describe what students will learn during the course. The explicit learning outcomes set forth in the syllabus are related to the activities in the course schedule and the various examinations administered during the course. All five examinations are related to and discussed in light of the learning outcomes of the course.
In the studied course, the learning outcomes start in the project team process. The students are divided into project teams on the first day of the course and are lectured about how to build the project team. The lectures introduce knowledge about project organization, project team composition, different roles in the team, communication profile models, development phases in the team, norms and rules, values and attitudes, project team rules, feedback, leadership, management, situational leadership, conflict resolution, and expectations and demands. Table 1 presents the learning outcomes applicable at the start of the course.
Table 1. Learning outcome applicable at the start of the course 
Process
Learning outcome
The project team 
process
After passing the course, students should be able to:
1. Identify various kinds of situation-adapted leadership in an IT project
The project teams then choose at least three optional subjects that are covered in the lectures; the teams study these subjects in greater depth, and present the results orally and in writing in a workshop at the end of the second week. It is important to motivate the students and involve them in the learning outcomes as soon as possible. In this case, the students learn from their classmates in the project teams, creating a good learning climate.
4.2.	Course execution
During course execution, the project teams start directly working on the course project and engage in lab lessons, lectures, and exercises involving fundamental concepts of object orientation. The project teams start hands-on work on the various use cases involved in the shopping cart in the e-commerce website – the project in the third week of the course. The design and use case diagrams are presented to the teacher in the fourth week of the course. Open source software is used as the IT tool during the project. After that, the project teams implement the design. The teacher then schedules milestone meetings every second week. The project teams receive more interactive lectures from the teacher. This improves project team learning and understanding (Bernhard, 1997), for example, by presenting and explaining object-oriented concepts in the programming language (e.g., objects, classes, methods, and parameters). These interactive lectures also facilitate two-way communication with the students, which motivates the students. 
The interactive lectures start by focusing on student questions concerning the project. One question project teams normally ask concerns visualizing the class structure. It is important to choose a tool environment that can visualize important concepts (Barnes and Kölling, 2006). The teacher identifies and discusses different types of solutions with the project teams, supervises them, and provides feedback every week.
The object-oriented website for e-commerce is presented at a seminar, held after approximately seven weeks of the course, together with the other project teams. The students also present lessons learnt from the project. After the seminar, the teacher assesses the project and provides feedback to the students. The students also get more information about the project environment during project implementation. In this case, they have lectures on how to solve easier legal issues arising in an IT project and how to determine when legal assistance will be needed. Processes for managing project procurement include contracts, which are legal documents between a buyer and a seller, which include terms and conditions. The project team seeks support from legal specialists, for example, in contracting, purchasing, and IT law (PMI, 2008). This support is a focus of the course. Another important question concerns the legal aspects of using open-source software when developing an e-commerce website. Table 2 presents the learning outcomes applicable during course execution.
Table 2. Learning outcomes applicable during course execution
Process
Learning outcomes




The 
project 
process





After passing the course, the student should be able to:
2. Analyse and design an IT system using object-oriented, unified modeling language (UML) design elements
3. Understand the principles underlying object-oriented analysis and design
4. Use IT support in implementing an object-oriented system
5. Solve simple legal issues in an IT project 
6. Determine when legal expertise should be consulted in an IT project
4.3.	At course end
At course end, the students’ individual learning process is central. It is important that the students are motivated during the project, which is the carrier of new knowledge in the course. The students have learnt a lot from the project, and now have an opportunity for deeper learning. It is important to involve the students in challenging tasks, some of which should be optional. There should be the possibility of students obtaining a higher grade, such as passing with distinction. This is optional for the students, and in this course the students must complete an individual project-related task.
Some of the students’ learning should also be reflective. Students can, for example, reflect on the knowledge they acquired when starting and executing the course. The students also have the chance to reflect on their choice of program profile. The students should be able to display their learning results in both oral and written forms. Table 3 presents the learning outcomes applicable at the end of the course.
Table 3. Learning outcomes applicable at the course end
Process
Learning outcomes

The 
students’ individual learning 
process

After passing the course, the student should be able to:
7. Use a language and set of concepts in explaining an IT project to leadership 
8. Deepen their knowledge within the chosen program profile
9. Present basic legal concepts concerning IT projects
4.4.	Evaluating learning outcomes
As mentioned above, teachers must manage nine learning outcomes in this course, all of which must be planned in detail before the course starts. The students later evaluate these outcomes, normally starting two weeks into the course. This evaluation is conducted as a “muddy cards evaluation”. “Muddy Cards are a variation of the One-Minute Paper technique (Angelo and Cross, 1993) specifically designed to determine gaps in student comprehension”. Each student writes constructive criticism on a small card; the cards are collected, and the students then receive written and oral responses to the criticism. The teacher usually resolves minor issues quickly, discussing them with students during a lecture. In this course the students also evaluated the learning outcome after two weeks. Mid way through the course, another evaluation is conducted, and when the course ends a final evaluation is conducted by the students. The questions used in these evaluations are based on, among other matters, the learning outcomes of the course, and the evaluation questions can be categorized according to the three processes. The teacher starts to plan the next version of the course, during course planning.
5. CONCLUSIONS AND FURTHER RESEARCH
This paper has highlighted three central concepts: learning outcomes in the syllabus, learning activities in a project-oriented course, and the processes involved in a project-oriented course. The teacher supports student learning by coordinating these central concepts (see Figure 6).
First, the teacher must design the course by using the learning outcomes set forth in the syllabus. The teacher starts by asking two questions:
“Why are these learning outcomes in the syllabus and what are their purposes?”
By answering these two questions, the teacher can clearly communicate the learning outcomes, throughout the course, to the students. 
Student learning should start at a low conceptual level, indicated by verbs in the learning outcome such as “identify”, and end at a high conceptual level, indicated, for example, by the verb “reflect”. Different learning outcomes are therefore emphasized at the start and end of the course. It is also important for the students to be aware of the learning outcomes. This means that, before the course starts, the teacher must have a broad overview of when to apply particular learning outcomes, which requires preparing and understanding the course flow. By answering why particular outcomes are part of the course syllabus, the teacher can explain, for example, how the course relates to other courses in the program. 
Second, the teacher should formulate a detailed strategy for how to facilitate the learning outcomes, which is done using the learning activities in the project-oriented course. To this end, the teacher asks these two questions:
“How can a student use learning activities during the course, and what learning activities support particular learning outcomes for the student?”
When answering these two questions and deciding what type of learning activity to use, the teacher should consider the following factors (the examples are based on the course studied here):
* Student learning is based on prior experience
o learning activities include the workshop in the second week
* Student learning is an active process
o learning activities include lab lessons, lectures, and exercises dealing with fundamental concepts of object orientation; project teams start working hands-on during course execution
* Student learning should be organised using realistic projects
o learning activities include interactive lectures focusing on student questions relating to the project, i.e., the shopping cart in the e-commerce website
* Student learning should encompass multiple perspectives
o learning activities include lectures that present information about the project environment; in the studied case the students had lectures on IT law
* Student learning is unique to each student
o learning activities at the end of the course should be of various types, which could be optional and be presented in oral and written form
Finally, the teacher starts dealing with the processes in the project-oriented course by asking two questions:
“When should a student or a project team work on given learning outcomes and activities, and who, student or a project team, should work on these?”
Team building and project team energy must be effectively managed by the teacher. Student learning is influenced by other students. The teacher must create a good learning climate, which could be done from the outset of the course. Then the teacher must guide the students so that they can achieve the learning outcomes during course execution. This entails questioning the project team and providing feedback on an ongoing basis. After that, the students work individually via their learning process (see Figure 7), allowing them to deepen their learning. This could be done by using a range of learning activities up to the end of the course.
The management of learning outcomes in regular university courses needs to be further refined; evaluation, verification, and testing in regular course design are key aspects of this. Central analytical concepts could also be set forth in more mature forms and provided with a stronger basis in theory; for example, the results presented here could be related to frameworks for course design (Melin et al., 2009), theories of educational taxonomy (Bloom et al., 1956) and instructional design theory (Merrill, 1994).
ACKNOWLEDGEMENT
First, we would like to thank the other teachers of the course, Bengt Lennartsson and Georg Hellgren, for their contributions. Second, this study has been financially supported by the Swedish Agency for Networks and Cooperation in Higher Education. The study is a part of a R&D project called “A learning outcome model – reflected assessment”. The project is managed by the Information Systems Division, Linköping University (LiU). LiU is also an active part in the project, together with information systems researchers, teachers and student representatives from Dalarna University, Jönköping International Business School and University of Borås. We dedicate this paper to our late project member and research colleague Dr. Göran Hultgren, Dalarna University.
REFERENCES
Angelo, T. A. and Cross, K. P. (1993) Classroom Assessment Techniques, 2nd ed., Jossey-Bass, San Francisco.
Barnes, D. J. and Kölling, M. (2006) Objects First with Java, 4th ed. Prentice Hall/Pearson Education, Upper Saddle River, New Jersey.
Biggs, J. (2003) Teaching for Quality Learning at University. Open University Press, Berkshire, England.
Bloom, B. S., M. D. Englehart, E. J. Furst, W. H. Hill, D. R. Krathwohl (1956) Taxonomy of Educational Objectives: The Classification of Educational Goals. McKay, New York.
Cederling, U., R. Ekinge, B. Lennartsson, L. Taxén, and Wedlund, T. (2000) “A Project Management Model Based on Shared Understanding.” Proceedings of Management Minitrack in the Organizational Systems and Technology Track of the Thirty-Third Hawaii International Conference on System Sciences (HICSS-33), January 4–7.
European Commission (2004) ECTS Users’ Guide: European Credit Transfer and Accumulation System for Lifelong Learning. DG Education and Culture, Brussels, Belgium.
European Association for Quality Assurance in Higher Education (2005) Standards and Guidelines for Quality Assurance in the European Higher Education Area. DG Education and Culture, Helsinki, Finland.
Gibbs, G. (1995) Assessing Student Centered Courses. The Oxford Centre for Staff Development, Oxford.
Kolb, D.A. (1984) Experiential Learning; Experience as the Source of Learning and Development, Prentice-Hall, Englewood Cliffs.
Kruchten, P. (2003) The Rational Unified Process. Addison-Wesley, USA.
Lewin, K. (1951) “Field Theory and Learning.” Cartwright, D. (Ed.) Field Theory in Social Science: Selected Theoretical Papers, Social Science Paperbacks, London. 
Marchewka, J. (2006) Information Technology Project Management Providing Measurable Organizational Value. John Wiley and Sons Ltd., New York.
Melin, U., Axelsson, K. and Wedlund, T. (2009) Project-based Learning - An Emergent Framework for Designing Courses, Information Systems Education Journal, 7(34), http://isedj.org/7/34/, pp. 3-11. (Also appears in The Proceedings of ISECON 2006: §2144. ISSN: 1542-7382).
Merrill, M.D. (1994) Instructional Design Theory, NJ, Educational Technology Publications, Englewood Cliffs. 
Project Management Institute (PMI) (2008) A Guide to the Project Management Body of Knowledge (PMBOK® Guide), 4th ed. Newtown Square, Pennsylvania, USA.
Ramsden, P. (2003) Learning to Teach in Higher Education. RoutledgeFalmer, London.
Schwalbe, K. (2005) Information Technology Project Management. Thomson Learning, Cambridge, Boston, UK.
Tuckman, B. (1965) “Developmental Sequence in Small Groups.” Psychological Bulletin 63, pp. 384-399.
Wedlund, T. (2000) Global Product Development Supported by Groupware. Proceedings Product Models, Linköping.
Wedlund, T., Axelsson, K. and Melin, U. (2006) “Project-Oriented Education: Managing Three Simultaneous Processes.” Higher Education Close Up 3 (HECU3), July 24–26, paper 42.
Yin, R.K. (1995) Case Study Research, Design and Methods Sage Publications, Thousand Oaks CA.


Appendix


Figure 1. A generic project life cycle (Wedlund et al., 2006)



Figure 2. Kolb’s steps of a learning cycle (Kolb, 1984)


Figure 3. Tuckman’s stages of project team development (Tuckman, 1965)


Figure 4. Areas of expertise needed by the project team (PMI, 2008)

Figure 5. Managing three simultaneous processes in a project-oriented course (Wedlund et al., 2006)


Figure 6. Coordinating student learning in a project-oriented course


Figure 7. Different cognitive levels of learning outcomes in a project-oriented course