An Inquiry-Based Learning Framework Reconciling_2004.doc Created on 2/19/2004 8:32 AM Reconciling a Traditional Syllabus with an Inquiry-Based Introductory Course Katrin Becker Computer Science Education Group Department of Computer Science University of Calgary 2400 University Drive NW Calgary, Alberta Canada T2N 1N4 (403)220-5769 email: becker@cpsc.ucalgary.ca Abstract This paper outlines a new approach that permits the use of an inquiry-based style of learning while still meeting the requirements typical of a more traditional lecture and content-based format. The students are informed of the overall course objectives and given the freedom to choose how they will meet these goals. The goals and outcomes of the course or unit are described in detail using a rubric; a large set of problems to solve is collected or created, and the solutions to the problems are analyzed and mapped onto the course rubric. By providing students with this very large set of pre-analyzed problems from which to choose, it is possible to permit learners a great deal of freedom. The mechanism for verifiable accountability is created through the mapping of the solutions onto the course rubric. This design was used in a first-year single semester course in the fall of 2003, and a preliminary assessment of this method’s viability is provided. Introduction The phrase “Inquiry Based Learning” has become quite popular in recent years, but when pressed, many will admit that they aren’t really sure what that means. Perhaps the best example of Inquiry-Based Learning (IBL) is the Ph.D. or Master’s Thesis. In this Page 1 of 15 An Inquiry-Based Learning Framework Reconciling_2004.doc Created on 2/19/2004 8:32 AM scenario, the learner chooses a topic or area of study, develops a plan for carrying out that study, pursues the work, and finally writes it up and defends it. [Coll1983] Unfortunately, this degree of freedom is not feasible in most typical undergraduate courses, especially in the current climate of assessment and accountability. [Boud1997] As a result, when the concept of inquiry based learning is employed within the context of a college course spanning a single semester, the freedom of a pure inquiry based approach must necessarily be somewhat curtailed. This does not mean however, that we cannot use this approach. Using the framework outlined in this paper, it is possible to meet the goals and objectives of a traditional course or unit within an inquiry format. 1. Inquiry vs. Topic Based Syllabi One of the key elements of an inquiry-based approach is that the learners drive the content of the course or unit using inquiry (i.e. by asking questions that they wish to answer). Freedom of choice is essential. Another key element is that the instructor generally does NOT teach according to a pre-determined schedule, and the traditional lecture format is rarely employed. Instead, much of the contact time in such a course is devoted to helping individual students understand and solve the problems they have chosen to solve. The role of the instructor is that of coach, or facilitator, rather than that of a teacher using the more typical tell-test approach. [Tayl2002] Inquiry based learning involves a significant shift in approach, from one that is typically topic or content based and teacher centered, to one that is learner centered and emphasizes significant learning that is lasting and valued both for its intrinsic qualities as well as for its applicability to a future professional career [Fink2003]. This leaves many who would like to use this approach in a quandary, since it is difficult to define the Page 2 of 15 An Inquiry-Based Learning Framework Reconciling_2004.doc Created on 2/19/2004 8:32 AM outcomes of such a course in a manner that allows for defensible assessment and comparison against more traditional, topic- or content-based courses. In practice, many inquiry-based courses are described using vaguely defined objectives. This is one of this method’s chief detractors when it comes to its use within a carefully structured curriculum. Further, students in these courses often complain that they lack direction and guidance. Learners find it difficult to track their progress and are often unsure of how they compare against those acquiring the same material in more traditional ways. There is a high level of discomfort in completing a course and being unable to articulate what the learner achieved, both for the students and for the instructors. [Bran2000] Since most university courses must still fit into the traditional, content-based system, there is a need to maintain some ‘control’ over the course content, even in an IBL course. In other words, specific goals and outcomes must be demonstrably achieved by learners in an IBL course, and these goals and outcomes must stand up to scrutiny when compared against those of their more traditional counterparts. This is especially true in the current climate of accountability. [Walv1998] The proposed approach maintains ‘control’ over the content of the course by providing the students with a large set of predetermined problems (questions) from which to choose. Instead of permitting the students to pose any problem, they are to choose a problem from this pre-defined set. In order to maintain the inquiry-based approach, the set of potential problems must be large (several hundred would not be excessive). Further, solutions to these problems must not only be known 1 , but also analyzed in specific ways. 1 Problems with no known solutions may still be acceptable, provided it is possible to analyze the solution attempt. This means that a problem may be posed for which no known solutions either exist, or exist yet, provided that the problem solving attempts can be used to demonstrate the achievement of one or more of the stated course outcomes. Page 3 of 15 An Inquiry-Based Learning Framework Reconciling_2004.doc Created on 2/19/2004 8:32 AM ‘Control’ over content is achieved by knowing in advance which parts of the course material each problem addresses, and to what extent. The students are given access to a detailed description of the course objectives and a guideline explaining which objectives can be met by each of the available problems. The learners are then be permitted to choose any sub-set out of the entire problem set, as long as the union of achieved outcomes addressed by the specific problems they have picked meets the overall course requirements. 2. Rationale for the Use of Inquiry Based Learning and This Approach A course designed in this manner can provide a viable inquiry-based alternative to a more traditional course. It provides an opportunity for students to engage in self-directed learning activities that are known to enhance and augment their problem analysis, problem solving, and solution implementation abilities. [Reig1983] Because students have an opportunity to choose the problems that interest them most, they are more likely to take ownership of those problems and become engaged in a way not normally seen when a problem is simply chosen for them. This design also provides learners with regular feedback on their accomplishments, allowing them to see what they have accomplished and what they have still to-do at any point in the course. The course outcomes / goals are clearly and fully defined in a manner that permits learners to track their own progress and achievements. It also provides a mechanism for clear administrative accountability. Each of the goals and outcomes is assigned a relative weighting. As the student accomplishes and demonstrates mastery of each of the goals, they are assigned a value proportional to the degree of mastery, and these values are then used to calculate a final grade. 3. Key Features Page 4 of 15 An Inquiry-Based Learning Framework Reconciling_2004.doc Created on 2/19/2004 8:32 AM There are five key features of this approach. First, the fundamental framework is provided by the careful selection of a large set of problems, and students will be required to choose a minimum number of problems from this set according to publicly stated criteria. For example, students may be required to solve at least 6 problems, where at least one is from category "1", and at least one is at difficulty level "4". Second, each problem must be solved in advance, even though these solutions will not be made available to the students. Where applicable, several alternate solutions must be produced. Third, the solutions must be analyzed and sorted according to numerous key elements (approach; level of difficulty; time commitment, content). The rigor and richness of the course "content" depends largely on the number and selection of problems. If there is a rich selection of problems, then the course will meet the requirements of an InquiryBased framework; with too few problems it reverts to a more traditional course (worse in fact, because the structure of a schedule of lectures or lessons is now missing). Fourth, the entire course outline is described using a detailed set of instructional objectives [Gron2000], and finally, the solutions to the problems submitted by the students are mapped onto this rubric to demonstrate achievement and mastery. 4. Course Content In keeping with the inquiry-based nature of such a course, content flow will be largely determined by the needs of the learners in the class. Topics can be addressed as they become relevant in the process of examining and solving various problems. Questions asked by the students in the process of solving their chosen problems will drive the content (with possible "guidance" from instructor). As appropriate questions present themselves Page 5 of 15 An Inquiry-Based Learning Framework Reconciling_2004.doc Created on 2/19/2004 8:32 AM they may from time to time form the basis for a more formal lecture, however lectures are not scheduled except on an as-needed basis. 4.1. Procedures & Criteria for Tracking Students' Achievements Table 1: A Sample Rubric for the Topic “Number Systems” 1 2 3 1 Element Understands Number Systems: Convert numbers between arbitrary 3 bases [to & from base 10]. 2 Explain an arbitrary base (such as base 4 5 or 13) without first having been shown that base. Show / tell / demonstrate conversion of 5 specific numbers from base X to base Y. Count in an arbitrary base. 6 9 Exceeds Requirements 16 base 2, 8 [3.0] base 2, 8, 16 [4.0] can do one base with advance preparation [3.0] can do most bases less than 10 [4.0] base 2, 8 [3.0] base 2, 8, 16 [4.0] base 2, 8 [3.0] base 2, 8, 16 [4.0] base 2, 8 [3.0] base 2, 8, 16 [4.0] Meets Requirements Perform simple arithmetic in an 7 arbitrary base. 4 Exemplary 20 any arbitrary base other than 10 [5.0] any base other than 10 [5.0] any arbitrary base other than 10 [5.0] any arbitrary base other than 10 [5.0] any arbitrary base other than 10 [5.0] 4.1.1 Mapping Content Onto Assessment Instruments: Table 1 is a portion (one topic/concept) of a "Global Course Rubric" written for an introductory Computer Science course. The course learning outcomes are written as a global, detailed rubric. A rubric is a two-dimensional table that describes (publicly and in advance) the criteria by which the learners will be judged. The columns describe the level of mastery (such as meets or exceeds requirements; exemplary). Which levels of mastery are to be used (as well as how they should be labeled) for any given unit or course is to be determined by the course designers. The rubric shown describes a single main topic (Number Systems), and the rows beneath describe various dimensions of quality that are deemed important. The first row identifies the level of mastery associated with each Page 6 of 15 An Inquiry-Based Learning Framework Reconciling_2004.doc Created on 2/19/2004 8:32 AM column (meets, exceeds, exemplary). The second row describes the desired outcome for this topic (understanding Number Systems), and the total number of points that the learner must ‘earn’ in order to attain each stated level of mastery (earning 16 points indicates that this student has exceeded the minimal requirements for an understanding of number systems). For each row then, the mastery level is described, and a number of points are assigned. The number of points assigned to each level of mastery will be relatively consistent from row to row but need not be identical. This allows for any specific row to be weighted more heavily than others. This is useful in those situations where one particular quality is deemed more important than others. (Huba2000) 4.1.2 Explanation of the Numerical Aspect (the point system) Row 1 describes the main element (topic/concept/skill) and the numbers in this row indicate the MAXIMUM number of points that can be achieved by fulfilling requirements in that and all previous columns. For example, a student earns 9 points by meeting the requirements in column 2 (“meets requirements”), and rows 3, 4, & 5. In order to earn more points, they would have to meet some requirements from column 3 or 4. This ensures that students can only earn a higher number of points by meeting more demanding requirements, rather than meeting multiple minimal requirements. In terms of the traditional letter grade system of assigning marks, "Meets Requirements" is essentially equivalent to a grade of "C". Exceeds Requirements = "B", and Exemplary = "A". In order to pass, a student must MEET requirements. Those requirements are deemed minimally acceptable for a pass. Letter grades can easily be mapped onto the requirements in such a way as to permit reasonable flexibility. It is also possible to describe the circumstances under which an individual would receive a "D" Page 7 of 15 An Inquiry-Based Learning Framework Reconciling_2004.doc Created on 2/19/2004 8:32 AM grade (for example: meeting more than half of the minimal requirements, but less than 80% could be a "D"). This would be in addition to the point totals and would eliminate the risk of awarding a passing grade to an individual who may excel in just a few areas but fails to meet the requirements in most. Using this approach, designers can decide what the minimal point requirement will be. Points in any one "element" are distributed in such a way as to require the student to meet designated standards. The entire course is described using such a rubric. For each topic, we must answer: “What do we want them to know/do with it?” and “How can they demonstrate they have achieved this goal?” If the course is to be a prerequisite for other courses, or is part of a core curriculum, then this rubric must be in agreement with the requirements laid out for the traditional version of this course. Using this approach has many advantages, not the least of which are: • Students know exactly where they stand in terms of mastery. • Students have a written definition of what the instructor feels is "adequate", "good", "excellent". • Students have some freedom to choose where they wish to devote their energy for achieving mastery, while still knowing what they must do to maintain competence in other areas. • Students can determine what they have left to do at any given point in time. • If the student's submission "misses the mark" in some areas, it is still possible to award a portion of the points according to the main rubric. Page 8 of 15 An Inquiry-Based Learning Framework • Reconciling_2004.doc Created on 2/19/2004 8:32 AM In order to ensure mastery in some areas that are considered to be key or foundational, the passing requirements can include a requirement to exceed requirements in at least one row. • Requirements can be chosen in such a way as to ensure a fairly wide range of competencies, while still setting attainable goals. 4.2. The Problem Set The detailed rubric establishes a verifiable connection between the achievements of students in this class, and the overall curricular requirements. The selection of problems establishes the domain of inquiry. Each problem is analyzed to determine which aspects of the curriculum it addresses/exercises and to what extent 2 . For example, in order to be able to write a program that (perhaps among other things) converts base 10 integers into any arbitrary base correctly, the programmer will have to know all the steps involved. So, a program that converts numbers to and from base 10 indicates that the student who wrote it would at least meet the requirement in row 3 of Table 1 above. The details of the specific solution will determine which level of mastery has been achieved. 4.3. A Student Solution that Doesn't "Fit The Mold" One of the hallmarks of an Inquiry-Based approach is that learners are given a great deal of lee-way both in terms of which problems they choose to tackle as well as how they choose to tackle a particular problem. This of course means that students are likely to submit solutions that were unpredicted. In other words, the advance analysis of the problem did not include this solution. It is also possible that the student may interpret the problem somewhat differently from that which was intended or predicted. In that case the solution will need to be analyzed when it is submitted. While this can be time consuming, 2 This is a predicted mapping is based on the analysis of the generated solutions. It MUST be adjustable. Page 9 of 15 An Inquiry-Based Learning Framework Reconciling_2004.doc Created on 2/19/2004 8:32 AM it is unlikely to happen regularly if the problem set has been analyzed and multiple solutions have been produced when possible. It is often possible for a domain expert who also possesses teaching experience to predict how novices are likely to approach a particular problem. On the other hand, allowing unexpected solutions provides a number of benefits: students retain flexibility and creativity is not stifled; the submitted solution can become part of the problem set – which enhances the set; and if the solution is a result of a misinterpretation, and it happens often, it can be used to help build a collection of "MissInterpretations", which would in turn indicate an area of common difficulty that may require a more structured approach. It also encourages students to increase their stake in the course material by creating their own problems to solve. [Shar1999] 5. Assessment A common form of assessment in inquiry-based courses is a student portfolio. A criticism of this form of grading is that it does not provide the measure of ‘coverage’ that exams are believed to provide [Huba2000]. Since the focus of the current design is to permit an inquiry based approach to be used within a more traditional curriculum, a means of mapping the traditional content onto a portfolio is required. This is accomplished with the connection of the problem set to the course objectives. To provide evidence of breadth, students were required to submit a minimum number of solutions of a predetermined level of difficulty. In the current implementation, it was decided that students would be required to submit a minimum of 6 programs, of which at least two had to be classified as intermediate difficulty and one as “wasabi” (challenging). Students submit completed programs at regular intervals, some of which must be demonstrated for the instructor, and then each submission is mapped onto the rubric for the Page 10 of 15 An Inquiry-Based Learning Framework Reconciling_2004.doc Created on 2/19/2004 8:32 AM course to determine which elements of the course requirements were met. Since elements of the rubric will each have associated point values, the points are then used to calculate the final mark. There is a component of the mark allotted to group work, attendance, and general participation. This is NOT an independent study course. There are no formal exams. There are no optional components, and although there is a minimal requirement, there is no upper limit on the number and kind of problems the student chooses to solve. This design was implemented in a first-year course, so it was essential that the students’ progress be monitored, and that they were given assistance in organizing their time and in focusing their efforts so they would be able to achieve the grade they desired. There were no hard deadlines, so it was essential to connect with each student in a face-toface meeting to ensure that the course expectations had been clearly communicated, and were understood. The instructor met with each student individually twice during the term: once before the middle of the term, and once shortly before the end. The meetings lasted approximately 10-20 minutes and were used to inform the student of his/her progress, identify areas of potential concern, and to clarify what they needed to do to complete the course. The correspondence of the assessment to the prescribed content was achieved in the following manner: • The total weights of the problems chosen by the student had to fall within a specified range. • A detailed rubric was produced which outlined the learning outcomes required by the end of the course. Page 11 of 15 An Inquiry-Based Learning Framework • Reconciling_2004.doc Created on 2/19/2004 8:32 AM Each problem was mapped onto this rubric and the students were responsible for ensuring that all elements of the rubric were adequately addressed. Formal, common exams are inappropriate in this setting since students are not all in the same place at the same time. Consistency is achieved because the overall requirement is that they have all achieved a stated level of competence by the end of the term. [Tooh1999] As the instructor was working closely with the students for more than the usual number of contact hours per week, she was in a good position to conduct a subjective, yet defensible assessment of each individual. 7. First Run Reaction An approach such as this holds great promise for attracting and inspiring the better students who are often apathetic in more traditional courses. This design was implemented and employed in the fall semester of 2003 at our institution and admission was by invitation only. The course consisted of 21 of the top high school graduates. The reaction was universally positive. As this was the first implementation of a novel design, regular feedback from the students was essential. Students were asked to provide weekly feedback in the form of a “Critical Incident Questionnaire”. [Diam1997] They were asked five questions: 1. What was the moment in class this week where you felt most engaged (most involved)? 2. What was the moment in class this week that you felt most distanced (least involved or isolated? OK, maybe even repulsed)? 3. What action did you find the most helpful? 4. What action did you find the most confusing? Page 12 of 15 An Inquiry-Based Learning Framework Reconciling_2004.doc Created on 2/19/2004 8:32 AM 5. What surprised you most this week (related to the course, or things we did)? The questionnaires were extremely useful for maintaining focus and direction, both for the students and for the instructor. Students especially appreciated the freedom and the degree of choice they had in determining how to master the material required of them. There were also a few unexpected benefits of the format of the course. The lack of hard deadlines and the ability to resubmit assignments fostered an environment where participants were free to share information as well as to revisit a problem with new information and revise it. This allowed them to apply new knowledge to a known problem and to see the effect of a greater depth of understanding. [Mint1997] The requirement that they submit any solutions that meet the course outcomes also had the side effect of reducing the risk on any specific assignment. This freed the students to try things they might not otherwise have attempted. In this context, all they would loose is time. In a more traditional setting with hard deadlines for assignments, a student taking a chance on some new approach may wind up with a solution that didn’t work at all and be required to accept a failing grade on that assignment. Here they always had another chance. This also had a positive effect on the group work they were required to complete. Students were required to complete at least two of their programs as part of a group. It is often the case that the best students prefer to work alone, perhaps because they can find few true peers and often end up ‘carrying’ any group of which they are members. In this class, because they were not required to submit any specific work, they always had the option of quitting work on a program or with a group, and moving on to another. It would cost them time, but the stake they had in a specific assignment or group was reduced – they retained the choice of giving up and trying a different one. This freedom seemed to Page 13 of 15 An Inquiry-Based Learning Framework Reconciling_2004.doc Created on 2/19/2004 8:32 AM have the effect of encouraging the more able members of the group to take to time to teach the less able member(s) what they were doing rather than simply getting the job done. The comments on the group work from all members of the class was that they all found the group work to be beneficial – some for what they learned about programming and problem solving, and others for what they learned about group work. This course can perhaps be best summed up with the following student comments: “I love the class... it allows me to learn what I want at a pace that I want. I could not be happier!” and “To be honest I don't want the semester to end cause I know I will miss this course.” 8. References Link to course web site: http://pages/cpsc.ucalgary.ca/~becker/235-IBL [Boud1997] Boud, David, and Grahame Feletti, (Eds) 2E, "The Challenge of ProblemBased Learning", 1997 Kogan Page, ISBN 0-7494-2560-1 [Bran2000] Bransford, et. al., Ed. National Research Council, "How People Learn: Brain, Mind, Experience, and School", Expanded Edition, 2000, National Academy Press, ISBN 0-309-07036-8 [Coll1983] Collins, Allan, and Albert Stevens, “A Cognitive Theory of Inquiry Teaching”, in: "Instructional-Design Theories and Models: An Overview of Their Current Status, Reigeluth, Charles M. Ed. 1983, Lawrence Erlbaum, ISBN 0-89859-275-5 [Diam1997] Diamond, Robert M., "Designing & Assessing Courses & Curricula: A Practical Guide", Revised 1997 ISBN 0-7879-1030-9 Jossey-Bass Inc. [Fink2003] Fink, L. Dee, “Creating Significant Learning Experiences: An Integrated Approach to Designing College Courses”, 2003, Jossey-Bass ISBN 0-7879-6055-1 [Gron2000] Norman E. Gronlund, “How to Write and Use Instructional Objectives”, 6E Merrill, Prentice-Hall, 2000, ISBN 0-13-886533-7 [Huba2000] Huba, Mary E. and Jann E. Freed, “Learner-Centered Assessment on College Campuses: Shifting the Focus from Teaching to Learning”, 2000, ISBN 0-20528738-7 Allyn & Bacon [Mint1997] Mintzes, Joel J., James H. Wandersee, Joseph D. Novak, Ed., “Teaching Science for Understanding: A Human Constructivist View”, 1997, ISBN 0-12498360-X, Academic Press [Reig1983] Reigeluth, Charles M. Ed. "Instructional-Design Theories and Models: An Overview of Their Current Status, 1983, Lawrence Erlbaum, ISBN 0-89859-275-5 Page 14 of 15 An Inquiry-Based Learning Framework Reconciling_2004.doc Created on 2/19/2004 8:32 AM [Shar1999] Sharan B. and Rosemary S. Caffarella, "Learning in Adulthood: A Comprehensive Guide", 2nd Ed., Merriam, 1999 ISBN 0-7879-1043-0 Jossy-Bass [Tayl2002] Taylor, Peter C., Penny J. Gilmer, & Kenneth Tobin, Editors Transforming Undergraduate Teaching: Social Constructivist Perspectives, 2002 Peter Lang Publishing (Counterpoints Vol 189) ISBN 0-8204-5293-9 [Tooh1999] Toohey, Susan, "Designing Courses for Higher Education", 1999 ISBN 0335-20049-4 Society for Research into Higher Education [Walv1998] Walvoord, Barbara E. & Virginia Johnson Anderson Effective Grading: A Tool for Learning & Assessment, 1998 Jossey-Bass ISBN 0-7879-4030-5 Page 15 of 15