Cutting-Edge Research by Undergraduates on a Shoe-String Katrin Becker Overview Typical Undergraduate Research Model Why have undergrads do research? What sort do they do? Challenges to Supporting Undergrad Research Ideal Undergraduate Research Model Addressing the Challenges Why Should Undergrads Do Research? What do Undergrads Bring to the Table? One Approach – Cutting Edge Research Reaction, Problems and Results CCSC-NW 2005 2 Typical Undergraduate Research Model Faculty directs topic of research. Highly prescribed. • Senior Project or Capstone Course • One on one with individual faculty members. • Part of a grant-funded project. • Undergrads fill the role of “junior assistant to the deputy-assistant”. CCSC-NW 2005 3 Why have undergrads do research? • NSF actively promotes it (SME&T). • Students become better qualified for grad school. • Advances faculty research. • Assists faculty publication. • Fulfills internship requirements / experience. CCSC-NW 2005 4 What sort of research do undergrads do? Research-oriented (not unlike kosher-style*) • Replication of completed work. • Completing a predetermined task that is part of a larger research endeavor. • The student is the only one who doesn’t already know the outcome. • Small, incremental advances to heavily researched problems. *Has trappings of research, but isn’t really. CCSC-NW 2005 5 Challenges to Supporting Undergrad Research • Undergrads lack depth & breadth. • Insufficient time for background research. • Faculty invest considerable time (mentoring). – “Lost” if students don’t finish. – “Wasted” if students don’t go on to grad school. • Lack of adequate funding. • Demand for departmental and institutional support. • Lack of research infrastructure (especially for teaching institutions). CCSC-NW 2005 6 Answering the Challenges • Undergrads lack depth & breadth. – True. Not enough reason not to try. • Insufficient time for background research. – True. Learning to target background research is a valuable skill. Worth learning. • Faculty invest considerable time (mentoring). – Not the point. Is advancing my career more important than positive rolemodeling? Than training the next generation of CS professionals (including those who do not benefit us directly)? • Lack of adequate funding. – Not a reason. This doesn’t require funding. • Demand for departmental and institutional support. – True. Can be accomplished without either, but is harder. • Lack of research infrastructure (especially for teaching institutions). – Not a deal-breaker. Insufficient to prevent success. CCSC-NW 2005 7 Why Should Undergrads Really Do Research? • Engagement in authentic, relevant work. • Research is about more than grad school prep. • Promotes meaningful undergraduate experience. • Integration of research into teaching promotes meaningful understanding and greater relevance. CCSC-NW 2005 8 Why Should Undergrads Do Research? (really) • Helps prepare future professionals for lifelong learning. • Facilitates development of higher order thinking skills. – Critical assessment of resources – Novel and creative approaches to problems – Complex inquiry • We learn by trying, and by making mistakes. • Helps develop students skills and professional abilities. CCSC-NW 2005 9 What Do Undergrads Bring? • Naiveté. • Undisciplined approaches. • Different perspectives. • Energy. • Optimism. • Expertise (sometimes) – In unexpected areas – In access to resources – In tool use CCSC-NW 2005 10 One Approach to Cutting Edge Research Context: Junior-level (3rd year) course in advanced data structures (data & file architecture, file formats, compression, etc.) Application area: Educational Object Repository Focus: Examining & analyzing images, sound, video CCSC-NW 2005 11 Features 1. No Known Answers (Can’t know when done) 2. The Work is Risky (May lead no-where) 3. Process Counts More Than Results 4. Groupwork Not Featured 5. Assessment Details Known in Advance 6. Criterion-Based Assessment 7. Active Reflection & Self-Assessment 8. Post-Mortem 9. Wide Range of Choice 10. Not Tied to Specific Content 11. Use of Existing Solutions Encouraged CCSC-NW 2005 12 1. No Known Answers Instructor doesn’t know the right answer – Risky for students • may ‘fail’ – Risky for instructor Won’t know when the work is done. – Good chance it can’t be completed in the time given • lack of control May not know if it’s “right”. CCSC-NW 2005 13 2. The Work Is Risky • Failure *IS* an option – Cannot be allowed to translate into a failing grade – Must provide ‘safe’ environment where students can take risks that may not lead to results. • Part of research • Part of life • May Lead No-Where • Not just about marks for effort – The process counts CCSC-NW 2005 14 3. Process Counts More Than Results Project Marks: – 1/3 code & results; – 1/6 proposal – 1/2 post mortem • reflection & self-assessment Emphasis on – Writing and reporting – Project planning – Collection and assessment of resources – Verification & justification CCSC-NW 2005 15 4. Groupwork Not Featured Assumed: – Other courses emphasize and train in group work. – Don’t need to do it here Emphasis on: – research processes and skills. If students work in groups, then individual contributions must be outlined. CCSC-NW 2005 16 5. Assessment Details Known in Advance • Serves as guide to planning and work. • Must be general, as details of project differ. • Serves as focal point for discussions of progress. Objectives outlined in detailed rubric: • Exemplary • Exceeds Requirements • Meets requirements • Fails to meet requirements. CCSC-NW 2005 17 6. Criterion-Based Assessment Criterion-Based vs. Relative Assessment Described in terms of successful completion of work. • Are claims supported? – Speaks to research process. • Quality of self critique. – Speaks to awareness of process. • Analysis of data. – Was data appropriate to project? – Were conclusions logical, reasonable? CCSC-NW 2005 18 7. Active Reflection & SelfAssessment Learners required to mark themselves. – Part of the requirement (get marks for it) – Penalized if too easy or too harsh – Markers mostly need to explain discrepancies between their marks and participants’ marks. What was learned? What would you do differently? What went right? Wrong? CCSC-NW 2005 19 8. Post-Mortem Mechanism to encourage reflection on process and experience (as opposed to simply what was produced). – What went right? – What went wrong? – What would you do differently if you did it again? CCSC-NW 2005 20 9. Wide Range of Choice Allows for personally relevant projects within instructor-controlled bounds. 2 main ‘streams’: 1. 2. Student generated research project. Replication of existing work. CCSC-NW 2005 21 Student-Generated Research Project • Was original project description • Proved too unstructured for some • Students given – Domain of inquiry – General problem category • Students required to demonstrate – Inquiry is directly tied to course content – Project involves active research CCSC-NW 2005 22 Replication of Existing Work Nature of research project was too unstructured for some. (see problems, later) Solution: – Offer selection of research papers. – Students could opt to replicate some or all of the work reported in the paper. Not the same benefit, but – Need to read research paper for understanding. – Need to carve out appropriate piece to replicate • 1 paper may involve several year’s work & numerous individuals – Assess quality of work, both their own & that of publication. CCSC-NW 2005 23 10. Not Tied to Specific Content Directly related to course content, but – Each project may focus on different aspects. Project should not be only work done in course – Must still ensure adequate contact with content CCSC-NW 2005 24 11. Use of Existing Solutions Encouraged Often considered ‘cheating’ in other courses. – Here, focus is on furthering specific research agenda Part of goal is to focus on core of project – Use of existing code and utilities saves time – Focus on inventing, not re-inventing Fancy / pretty interfaces not focus of project – Functionality is key – Fitness to task CCSC-NW 2005 25 Reaction • Terrifying. • Fascinating. • Increased confidence in managing and conducting own research. • Highest student ratings for course, ever. • Self-assessment helped students to focus – many used it like check-list. CCSC-NW 2005 26 Problems • Large class (~100 students) – Instructor had to rely on teaching assistants. – Insufficient time for instructor to guide students. • Teaching Assistants – Assigned by department • Some were unable to assist (although they were grad students, some were unable to conduct research, therefore could not help students). • Unfamiliarity with the literature in the area. • Departmental Support – Financial support is great, but not essential – Philosophical support is important. – If this is the only (or first) course in the program of its kind, then students will require far more direction and coaching. Lack of departmental support makes it difficult. CCSC-NW 2005 27 Results • Several became interested in research as a result of their experiences. • A few were able to publish their work. • We have at least 1 who ended up with 6 refereed publications by the time he graduated. He’s currently employed at Amazon. • ~ 10% went on to further project courses in order to continue their work. • ~ 5% went on to grad school. • TAs (grad students) found the project structure helpful for their own work. CCSC-NW 2005 28 Undiscovered Countries in CS • Non-text based searching (images, audio, video). – With terabyte storage here, searching is hot • Filtering for complex searches. • Security. • Content filtering. • Games (serious, and entertainment) – AI – Game technology applications • Architecture of massive data sets – for searching CCSC-NW 2005 29 Summary • Challenges exist. • Benefits to student are great. • Closer to real life. • Crucial to provide opportunities where failure of results do *not* translate to failure of assignment. Thanks CCSC-NW 2005 30