Int. J. Nurs. Educ. Scholarsh. 2016; 13(1): 1–11 Margaret Ann Quance* Nursing Students’ Perceptions of Anecdotal Notes as Formative Feedback DOI 10.1515/ijnes-2015-0053 Received August 3, 2016; revised June 15, 2016; accepted June 20, 2016 Abstract: Anecdotal notes are a method of providing formative feedback to nursing students following clinical experiences. The extant literature on anecdotal notes is written only from the educator perspective, focusing on rationale for and methods of production, rather than on evaluation of effectiveness. A retrospective descriptive study was carried out with a cohort of 283 third year baccalaureate nursing students to explore their perceptions of anecdotal notes as effective formative feedback. The majority of students valued verbal as well as anecdotal note feedback. They preferred to receive feedback before the next learning experience. Students found the quality of feedback varied by instructor. The anecdotal note process was found to meet identified formative feedback requirements as well as the nursing program’s requirement for transparency of evaluation and due process. It is necessary to provide professional development to clinical nurse educators to assist them develop high quality formative feedback using anecdotal notes. Keywords: clinical evaluation, formative feedback, anecdotal notes, nursing students, verbal feedback Feedback is commonly considered an essential element of the learning process (Hattie & Timperley, 2007; Juwah et al., 2004; Nicol & Macfarlane-Dick, 2007). Feedback has been conceptualized by Hattie and Timperley “…as information provided by an agent regarding aspects of one’s performance or understanding” (p. 81) and is necessary to the development of self-regulated learning strategies (Clark, 2012; Hattie & Timperley, 2007). “Good” feedback practices include: teacher and peer dialogue, statement and clarification of expected/desired goals and standards of performance, opportunities for students to improve their performance, quality information *Corresponding author: Margaret Ann Quance, School of Nursing and Midwifery, Mount Royal University, 4825 Mount Royal Gate SW, Calgary, Alberta T3E 6K6, Canada, E-mail: mquance@mtroyal.ca http://orcid.org/0000-0001-7460-2512 provided to students about their learning, and encouraging positive motivation and self-esteem of students (Juwah et al., 2004; Nicol & Macfarlane-Dick, 2007) and must be related to the particular learning context (Hattie & Timperley, 2007, p. 82). Formative feedback must involve students in strategies such as personal goal setting, monitoring and reflection, thus promoting the development of self-regulated learners (Clark, 2012). Formative feedback “has been shown to improve students’ learning and enhance teachers’ teaching to the extent that learners are receptive and the feedback is on target (valid), objective, focused, and clear” (Shute, 2008, p. 182). It is also clear that “assessment drives learning” (Van Der Vleuten et al., 2012, p. 207) and multiple assessment points for learning are ideal (Van Der Vleuten et al., 2012). Narrative feedback helps students to identify strengths and weaknesses in their performance and can be effective in developing strategies to improve performance (Govaerts, Van de Wiel, Schuwirth, Van der Vleuten, & Muijtjens, 2013). Theoretical courses provide opportunities for formative feedback in part through imbedded assessment strategies. Appropriate feedback allows students to see how well they are doing/learning. This feedback will allow/ encourage students to undertake self-remediation activities or seek help from professors to improve their learning and performance. Provision of opportunities for formative feedback are considered so important that academic regulations in many Canadian university settings specify how much feedback students should receive (in the marking of assignments, tests and papers) before a final exam. What is not clear or well understood is how formative feedback is provided to students who learn and practice in experiential or practice settings. At a nursing education conference in Western Canada where the results of this study were disseminated, session participants indicated that verbal feedback to students was the format most used in practice settings. Participants stated documentation of feedback usually occurred only when the student performed in an unsafe manner and/or was in danger of failing. This practice is consistent with research findings (Duffy, 2003; McCarthy & Murphy, 2008) perhaps indicating “educators were only concerned with safe practice and not with the knowledge and 2 M. A. Quance et al.: Nursing Students’ Perceptions of Anecdotal Notes behaviours and attitudes necessary to support it” (Deegan, Burton, & Rebeiro, 2012, p. 46). The practice of initiating documentation only when the student is not performing well and therefore not meeting course objectives also violates identified principles of good feedback practices (Juwah et al., 2004; Nicol & Macfarlane-Dick, 2007) and may indicate a lack of standards or guidelines for evaluation in the clinical setting (Orchard, 1994). Literature review Nursing education textbooks recommend anecdotal notes (AN) as one way of providing written feedback to students about their clinical learning experiences (Billings & Halstead, 2012; DeYoung, 2009; Emerson, 2007; O’Connor, 2006). The majority of the literature on AN has been written from the perspective of the educator, detailing the purposes and process of AN. One academic calendar statement recommends that instructors “keep anecdotal notes on all students and encourage students to keep their own notes about clinical situations” (University of Saskatchewan College of Nursing, 2014). However, schools of nursing generally have not published or made readily available the evaluative process used for experiential courses. AN are described as individualized objective written narrative accounts or observations of each student’s behaviour and/or interactions in clinical settings (Bonnel, 2009; Hall, 2013; Hall, Daly, & Madigan, 2010; Liberto, Roncher, & Shellenbarger, 1999). They are one way to provide formative feedback to the student with the intent of supporting student learning in order to meet the intended goals and outcomes of a clinical course. As Smith, McKay and Richardson (2001) comment: Feedback provides students with the information needed to overcome weaknesses, improve overall performance, and gain confidence in their clinical competencies. This feedback should identify the adherence to and deviation from established policies, standards, and procedures. Clinical instructors should document the feedback provided, along with suggestions and recommendations for improvement where applicable, and a specific time frame for behavioural change. (“Principle 4,” para.1) The available literature does not specify how AN are created. The development of process and policy related to AN as a method of formative feedback appears to have been left to each school of nursing to develop for itself and is not reported in the literature. Hall et al. (2010) identified a lack of published literature on the use and value of AN. In the past ten years, there is a noted lack of research and writing pertaining to about AN. No research has been found that focuses on students’ perceptions of the value of AN. This creates a concern for those education programs using AN for experiential learning. If students do not find AN an effective way of providing necessary and helpful feedback, then educators need to find another and/or different way of providing feedback that is useful to students for their learning. There is an established link between the need for documentation (which can be AN) and student progression in nursing courses and/or program (Emerson, 2007; Larocque & Luhanga, 2013; O’Connor, 2006; Smith et al., 2001). Smith et al. (2001) discuss several legal cases where the outcomes were supported by well written, factual, non-judgmental and objective AN describing student clinical performance. There is also a human rights requirement for due process in clinical evaluation of students with “all relevant procedures…applied fairly, equitably and with clearly defined rights and duties” (Scanlon, Care, & Gessler, 2001, p. 24) for faculty and students. This is not unlike the documentation of nursing care, with the requirement of similar principles and the possibility of scrutiny by professional and judicial bodies (e. g. Canadian Nurses Protective Society, 2007; College and Association of Registered Nurses of Alberta, 2006). Well written, factual, non-judgmental and objective AN provide a transparent picture of the evaluative process for the student as well as for faculty, meeting part of the requirement for due process. Students should come to their midterm (formative) or final (summative) evaluations, confident in his/her knowledge about the final mark. Utilizing a documented evaluation process would provide transparency for the student ensuring there are no unexpected or unwelcome “surprises” (near failures or failures) that would impact student progression through a nursing education program (Buck, Wilkinson, & Phillips, 2014). Two studies were found describing the use of AN in clinical evaluation of nursing students. Hall et al. (2010) conducted a descriptive study investigating the use of AN by clinical faculty. Of 64 responses from six American schools of nursing, 68.8 % reported weekly use of AN with another 28.1 % reporting occasional use of AN during the semester. AN were mostly used to describe student practice with regard to medication accuracy, attention to patient safety and professional behaviours. The authors comment, “…it is of some concern that accuracy in medication administration was rated highest of all reasons to use an anecdotal note. Although medication administration is obviously central to patient safety, other aspects of care may have even more significant M. A. Quance et al.: Nursing Students’ Perceptions of Anecdotal Notes implications for overall patient well-being” (Hall et al., p. 158). One weakness of this study concerns the lack of description of the processes that were used for the AN. This study presupposes that AN were being used the same way across all programs. However, this assumption confounds understanding of the results. We do not know what the AN processes are for each school and should not assume they are similar. Hall (2013) expanded the previous descriptive study, including Canadian and American schools of nursing. An electronic survey was sent out to faculty in 10 % of all Bachelor of Science in Nursing (BSN), Associate of Science in Nursing (ASN) and Canadian nursing programs, with 784 responses (23.3 % response rate). Similar to Hall et al. (2010), 62.4 % of faculty used AN “almost always” (p. 274) and 28.2 % reported occasional use. Faculty most often used AN to record student attention to patient safety issues and least to document incorporation of discharge planning. Qualitative feedback from faculty (309 responses) indicated they used AN to provide formative feedback to students (18.7 %), as evidence when completing final evaluations (12.3 %), self (educator) organization (12.3 %) and demonstration of student progression in competence in clinical settings (12 %). When considering the number of responses in the qualitative feedback, it is concerning how few faculty reported using AN for formative feedback (18.7 %) or as data for final evaluations (12.3 %). How are formative and final evaluations being developed? How is transparency of evaluation and due process demonstrated? The same weakness is noted in this study as in Hall et al. (2010). No information is provided about the processes of AN development and we cannot assume they are similar. Anecdotal note processes Over 1,600 students were enrolled in a four-year bachelor of nursing program at a Canadian school of nursing. Students begin their clinical practicum in the first year of their studies. As part of their workload, full time faculty are expected to have a certain number of hours of clinical supervision. However, the majority of clinical supervision was and is done by sessional clinical nurse educators (CNEs). In nine of the ten clinical courses in this nursing program (See Table 1), a CNE will be present on the clinical unit (acute care or community settings) with a maximum of eight students. This model of clinical instruction allows for development of long-term teacher- 3 Table 1: Clinical courses by program year. Year of program Course  Nursing  Professional Practice I Nursing  Professional Practice II Nursing  Professional Practice III Nursing  Adult Health Nursing  Family Newborn Health Nursing  Child Health Nursing  Mental Health Nursing  Seniors’ Health Nursing  Integrated Professional Practice Nursing  Transition to Independent Practice    Clinical hours Length of course   weeks   weeks     weeks or  weeks  weeks  weeks      weeks  weeks  weeks  weeks   weeks student relationships during each clinical course which is beneficial to student learning (Bok et al., 2016, p. 92). The final course in the program, Nursing 5114, is the only clinical course using a preceptor model of instruction. In the development of the Bachelor of Nursing curriculum, a detailed method of AN development was formalized that provides narrative weekly formative feedback from the CNE to each student in the clinical group. This process provides opportunity for student reflection, clarification and weekly goal development, and creates “data” for midterm and final evaluations. Through this AN process, transparency of evaluation was thought to be facilitated and due process of evaluation could be seen by the student. AN were written on pre-formatted forms customized for each clinical course (See Table 2) that contain program outcomes and specific course objectives. CNEs write their narrative observations on the AN document and send an electronic copy to each clinical student in the group, usually 2 days after the completion of the clinical experience. Each student is expected to review and then respond to the CNE’s comments in the AN. The student sends the individualized AN back to the clinical nurse educator for review before the next clinical experience. The intent is for each student to use his/her individualized AN to reflect on his/her clinical practice, to “see” if they were meeting course expectations and develop plans to correct performance issues, with the CNE’s assistance. CNEs are expected to use AN documentation as “evidence” when developing formative (midterm) and summative (final) evaluations. The need for “data” for 4 M. A. Quance et al.: Nursing Students’ Perceptions of Anecdotal Notes Table 2: Excerpt of anecdotal notes document for Year 3. Benchmarks and objectives Learning Objectives Instructor Observations and Feedback . Practice evidence based, ethical, safe & effective nursing care, integrating theoretical and practical knowledge, with individuals, families, communities and population (IFCP) groups in nursing practice & simulated settings. Objectives: a) Demonstrate critical thinking, problem solving, and decision making skills using relevant evidence informed knowledge. b) Demonstrate self-preparation for clinical practice, while seeking appropriate instructor guidance when required. c) Collaborate with the interprofessional team. d) Effectively apply the nursing process in complex care situations. e) Demonstrate safe and ethical nursing care for individuals, families and communities. Demonstrate: – articulation and adherence to the Canadian Nurses Association Code of Ethics and College and Association of Registered Nurses Professional Practice Standards. – safe performance of all care measures and skills with growing accuracy – adherence to university and agency policies and procedures. – preparation for each clinical practice day including application of relevant evidence-based literature. Student Response Identify in collaboration with the IFCP: – planning care in response to the changing health needs of client(s), giving evidence of knowledge-based practice. – reflection and evaluation of care appropriateness and effectiveness. Evaluate effectiveness and outcomes of nursing practice formative and summative evaluations, and the requirement for transparency of evaluation and due process are considered important enough that AN use, timely return to the student and expertise are embedded into the position descriptions and evaluation documents of CNEs. Clinical course coordinators (full time faculty) regularly check to ensure CNEs are doing individualized and timely anecdotal notes. These processes meet the suggested requirements for formative feedback (Clark, 2012; Govaerts et al., 2013; Hattie & Timperley, 2007; Van Der Vleuten et al., 2012). There is little published literature relating to the faculty’s use of AN or the value of AN as a formative feedback process in experiential/clinical evaluation. It is important to understand the student perspective in order to determine the value of AN to student self-reflective learning. A quality improvement survey was conducted a year before this study. Third year nursing students were surveyed about their experiences of AN over the first three years (seven of ten clinical courses) of their program. Two hundred and thirty-six students reported on their experiences with AN. It was found that half the CNEs were not initiating AN according to the program expectations. Student perceptions included the following findings: 66 % of students found it helpful to reflect on the CNE’s AN and the majority found it helpful to receive AN before the next week’s clinical experience. Only 46.6 % of the students preferred clinical nurse educator initiated AN. Many students (45.6 %) preferred no AN, but expressed a preference for only verbal feedback on their clinical performance. It was speculated this finding reflected the delay in formal student feedback as well as a reflection on the amount of students’ work required to produce AN. In contrast, instructor verbal feedback requires no physical student work. A targeted educational program was implemented for the CNEs to increase the incidence of CNE-initiated AN. The next step was to conduct a formal study of the students’ perceptions of the AN process. This descriptive survey of nursing students’ experience of AN would begin to fill the gap in the nursing education literature. The research question was: what were students’ perceptions of AN as a formative feedback tool? M. A. Quance et al.: Nursing Students’ Perceptions of Anecdotal Notes Methods Data collection methods This cross sectional exploratory study was a survey of third year nursing students (N = 283) in 2011 using three forced choice questions and two open ended qualitative questions. The quality improvement survey served as a pilot test for the formal study with study questions revised to improve student understanding of the questions. Ethics approval was received from the educational institution’s Human Research Ethics Board. All third year students were invited to attend an orientation to the nursing course “Adult Health”, the last course in the third year of their four year nursing program. Once the orientation was complete, the researchers introduced the study, using a powerpoint format and answered questions. The survey was handed out to all students at the same time. Students were asked to complete the survey and hand it back. They were informed that their names should not appear on the survey, providing student anonymity. If students did not want to participate, they were instructed to hand in the blank survey. This study design used the process of implied consent. The data collection process took 20 min. Demographic data was not collected as it was the intent of this study to gain an overall impression of students’ perceptions of the value of AN to their learning and evaluation processes. A more simplistic, exploratory study design was appropriate. Question 1 asked how AN had been used in each clinical course. Specific information was sought as to whether AN were CNE initiated, student initiated or there were no AN provided. Question 2 asked each student for his/her preference for AN to be CNE, student initiated or no AN. Question 3 asked what kind of feedback students found most useful for their clinical learning with forced options given for daily verbal feedback, weekly anecdotal or feedback through midterm and final evaluations only. Question 4 asked when it was most useful for students to receive their AN. Forced options were: before the next clinical experience, during the next clinical experience or in the preconference time of the next clinical experience. Question 5 asked students if CNE feedback, provided in the AN, assisted them in setting goals for the next clinical week. There was opportunity for qualitative responses for Questions 2 and 5. Data were analyzed through descriptive statistics and thematic analysis of the qualitative feedback. 5 The qualitative comments were analyzed with the responses were first sorted by responses to: instructor initiated, student initiated and no AN (Question 2) and “Yes”, “No” and “Other” (Question 5). Each researcher read all responses to individually and sorted the responses by frequency of reoccurring words and phrases. The researchers then came together and reviewed their word/phrase sorting together, determining overarching themes. When there were questions about the meaning of words and phrases, the literature was consulted to understand the most recent meaning. This process continued until consensus was achieved between the researchers. Results There was a response rate of 79.4 % (n = 235) of 283 students registered for Nursing 3104, Adult Health. It is not known if the 48 non-participating students refused to participate or were not present for the orientation session. The surveyed students had experience with two ways of receiving AN formative feedback. During the first three clinical courses in first and second year of the program (Professional Practice I, II and III), students would write AN notes and send them to the CNEs for comment. This was called “student initiated” AN. The School made a policy change after the third clinical course that required the CNEs to write AN and send them to the students for comment. This was called “instructor initiated” AN. In Question 1, students were asked how AN had been used in seven clinical courses in first, second and third year (See Figure 1). Of the potential respondents (n = 235), the number of “no responses” to this question by course ranged from a maximum of 4.4 % (Nursing 3134) to a minimum of 1.3 % (Nursing 3124). This was interpreted to mean that the majority of students remembered and could report on how AN were used in a particular course. Figure 1: Student experience of anecdotal notes by course. 6 M. A. Quance et al.: Nursing Students’ Perceptions of Anecdotal Notes There were a number of students who reported not doing or receiving any AN, with three courses showing more significant rates. Students reported receiving “no AN” 13 % of the time in Nursing 3124, 15.5 % of the time in Nursing 2216, and 18.4 % of the time in Nursing 3134. The remaining clinical courses had “no AN” rates ranging from a .44 % (Nursing 1214) to 7 % (Nursing 3114). These latter course rates were considered by the researchers to be acceptable, given the large number of students in the cohort. These differences in rates between courses presented important questions for faculty and administrators: how were students who receiving feedback about their clinical performance? How were CNEs collecting “evidence” in order to do midterm and final evaluations if they had no “evidence” provided by AN? These results raised concerns about students’ evaluative processes and if the principles of transparency and due process were provided to these students. 200 180 160 140 120 100 80 60 40 20 0 opportunity for students to modify their behaviour to meet the course expectations as interpreted by the CNE. Although CNEs are charged with comparing student behaviour to program and professional standards, it is not unreasonable to assume that students spend significant time to figure out that particular CNE’s individual expectations of student behaviour. This is reflected in our study, evidenced by the following statements: [AN] helps the student understand the CNE’s perspective, because they have more knowledge and criteria as to how the student should perform in clinical. Every CNE expects something different. If they initiate [the AN], it is easier to reflect appropriately. Students found that AN provided a one to one, confidential feedback mechanism with the CNE that not only identified weaknesses in their clinical practice but also areas of strengths. In fact, students in our population mentioned AN commented on their strengths many more times than their weaknesses. Students identified that recognition of their strengths helped them to develop a sense of confidence. …. Sometimes I forget examples of awesome things I did. Clinical Nurse Educator Student Inititated Inititated No Anecdotal Notes Figure 2: Student preferences for anecdotal notes. Question 2 asked students to indicate their preference for AN from three options and asked for rationale for their choice (See Figure 2). Overwhelmingly, students preferred clinical nurse educator initiated AN (n = 174, 74 %), followed by a preference for no AN (n = 40, 17 %) and student initiated AN (n = 16, 6.8 %). A large majority (88.9 %) of the students provided comments giving rationale for their choice with over 73 % commenting on instructor initiated AN. Qualitative student comments included statements such as: I liked to see what the clinical nurse educator noticed that may not be as obvious to myself. But I like adding to their comments on skills that they might have missed. CNE have the knowledge as to what is important to focus on when students are preoccupied and everything. It also gives the student an idea of what the clinical nurse educator knows and thinks about their (sic) practice. Keeps us updated on our status. Generally, students saw value in “seeing” their behaviour documented in the AN. This feedback provided Transparency of performance is one way of ensuring fairness and equitability of evaluation. Students found that AN made student performance “transparent” for both the clinical nurse educator and student. As one student commented, [AN] help me to know what they think I need to work on so I’m not taken by surprise at evals [evaluation time]. Students identified it was helpful to have areas for improvement identified verbally before the ANs were received. This meant students were not surprised by the CNE feedback in the AN, which sometimes was perceived as a negative experience. For those students (17 %) who identified a preference for no AN, many chose this preference based on their negative experiences with AN. Student initiated AN were identified by students as a significant additional workload for students. Dissatisfied students who had CNE initiated AN noted: Some CNE give little feedback. The anecdotal [notes] seem general and similar every week and [every clinical] course with no gain to learning. However, one student simply said, “Because I like verbal feedback instead”. Question 3 asked students for their preferences for feedback related to their clinical learning. Many students M. A. Quance et al.: Nursing Students’ Perceptions of Anecdotal Notes 7 indicated more than one preference. However, most students (n = 164, 69.8 %) liked daily verbal feedback, followed by weekly AN (n = 78, 33.2 %) and midterm and final evaluations only (n = 30, 12.8 %). This result clearly indicates that use of only one form of feedback is not fulfilling student needs. CNEs require skills in the provision of verbal and written feedback. Question 4 asked students to indicate their preferences for timing of receiving feedback through AN (see Figure 3). Most students (n = 188, 80.7 %) chose before the next clinical experience with no clear preference expressed for any of the other options. A number of students (n = 40, 17.1 %) indicated AN did not help them set goals and, in fact, increased their workload. This student group had three years of experience of AN from several different CNEs. Many student qualitative comments were related to the quality of CNE comments and the accountability of the CNE to do “good” AN. Students stated that “good” anecdotal notes were detailed and related directly to the individual student’s experience. They contained exemplars of positive behaviours as well as areas that required development. One student commented: 200 180 160 140 120 100 80 60 40 20 0 Students also found inconsistency of AN between CNEs and were able to identify those CNEs did not like doing AN and/or did them poorly. The differences in CNEs’ ability to “do” AN produced frustration and confusion. Often, feedback is vague and appears generalized. The opportunity to respond is restricted and fails to provide an opportunity to learn, or understand the depth. Before the next clinical experience During the next clinical In pre-conference of the experience next clinical experience Figure 3: Student preferences when to receive anecdotal notes. 180 160 140 120 100 80 60 40 20 0 [AN] ensures the instructor is attending to and engaging with each student in the clinical [area]. Not all CNEs are consistent with anecdotals and it’s very irritating about feedback. Some CNEs are verbal and don’t care for anecdotal and others are SUPER picky about them. Students were also able to identify when instructors were taking shortcuts with AN, using “copy and paste” from other student’s AN to develop AN that were vague, not personalized to the student and were similar to everyone else’s AN. As one student commented, [AN] show if the instructor gets to know you. I’ve seen multiple notes given to the wrong student. Yes No Sometimes Figure 4: Usefulness of anecdotal notes to set learning goals. Question 5 asked students if AN helped them set goals for subsequent clinical learning experiences (See Figure 4). There were a number of significant number of qualitative responses (n = 84, 36 %) with 63 % of the responding students providing positive narrative examples. Overwhelmingly, students indicated “yes” (n = 171, 73.4 %). Students commented: It helps you shape your practice and helps you be a better student as it aids your learning. Plus it is helpful and nice to hear positive things about your progress. It identifies my strengths as well as areas of improvement. Once I know my challenges, I can plan to work on them in the following clinical [experience]. The remaining students appeared to be ambivalent in their responses (n = 7, 3 %). For example, one student commented: Yes and no. It depends if you are doing well or not. One unexpected finding gleaned from the student qualitative feedback was the students’ the need to give voice to their perceptions of clinical experiences. Students used AN to tell the CNE of their experiences, good or bad. The instructor is not always with their (sic) students when we perform skills so he/she may not know of all the events that happened through clinical. Often the instructor does not see or know what I do each day and her anecdotal notes do not reflect my full practice. It appears that some students used AN notes for selfreflection although AN notes were not intended to be a 8 M. A. Quance et al.: Nursing Students’ Perceptions of Anecdotal Notes formal method of self-reflection. AN allowed students to “tell their story” about something that happened in the clinical setting of which the CNE may not have been not aware. Discussion The number of students who did not receive clinical nurse educator initiated AN was of interest. It is unknown how these students were receiving formative feedback on clinical performance though it is speculated that verbal feedback was provided. It is not known how “data” was gathered for the creation of midterm and final evaluations. Considering the number of students who found AN useful in planning their personal clinical goals (n = 171, 73.4 %), the lack of CNE initiated AN was concerning. Following CNE professional development activities for AN (after the qualitative improvement study the previous year), the number of CNE initiated AN increased by 50 %. This suggests that clinical faculty may benefit from periodic professional development related to provision of student feedback in clinical courses. However, there continued to be specific clinical courses where students were reported they were initiating AN (contrary to school policy) or they were not receiving any AN (See Figure 1: Nursing 2216, 3124 and 3134). Other strategies have been implemented to ensure CNEs comply with program expectations for AN. Specific information regarding the construction of AN notes is now included in the formal orientation for new CNEs. It is also an expectation for clinical courses to have the weekly AN posted through the clinical course Blackboard site, allowing AN to be monitored by the clinical course coordinator (a full time faculty member) for quality and compliance with the process. CNEs continue to be evaluated on a regular basis with demonstrated competence with AN as part of the required CNE competencies. These strategies are recommended for a programmatic assessment strategy (Van Der Vleuten et al., 2012), which AN are intended to be in this nursing program. This study indicates that using a very structured process for AN production and dissemination to students provided students with “good” (valuable) formative feedback. Students identified AN provided an opportunity for teacher and student dialogue, statement and clarification of expected/desired goals and standards of performance, opportunities to improve their performance from week to week, and quality information is usually provided to students about their learning (Juwah et al., 2004; Nicol & Macfarlane-Dick, 2007). Students also identified this kind of formative feedback encouraged positive motivation and their self-esteem as well as development of personal goal setting and reflection (Clark, 2012). An unexpected finding was many students found AN gave them a formal place for their voices. Students used AN notes to describe the experiences that CNE could not or did not see. Students identified AN provided feedback about their clinical performance that translated into transparency in the evaluation of their performance. It is likely that the concept of “due process” concerns faculty and administrators more than students, until a student is failing. When a student is struggling or failing, AN can serve students, faculty and administrators well by determining if students received documented feedback about clinical performance, what was needed to improve, and if improvement was demonstrated. AN can provide a process document (in our program, on a weekly basis) that then can be used to support or deny an appeal for a clinical failure. This facilitates the provision of due process procedures for faculty and students (Scanlon et al., 2001). Students in this study preferred daily verbal feedback (n = 164, 69.8 %), followed by weekly AN (n = 78, 33.2 %). This preference indicates that CNEs require skills in provision of verbal feedback and written feedback. However, verbal feedback requires significant skill and practice. Educator experience affects the kind of student feedback provided (Govaerts et al., 2013; Van Der Vleuten et al., 2012; Wenrich, Jackson, Maestas, Wolfhagen, & Scherpbier, 2015). Educators with less experience may act as cheerleaders focusing on positive rather than negative feedback, take a passive teacher role, have much concern about students’ fragility, want to create a “safe” environment to deter student discomfort, have limited goals and strategies, are oriented toward students’ current (rather than future) needs and use a private, one-to-one feedback time. More experienced educators are coaches rather than cheerleaders, act in a dynamic and selectively active role by determining which feedback should be given immediately and which can be delayed, understand that students are resilient, create a challenging but safe environment, are strategic and goal oriented, know what skills the students should have at different stages of their program/development, and foster an environment of “team” feedback, using patients and peers to do so (Wenrich et al., 2015, p. S93). Another study used an extensive literature review and Delphi process to develop a description of twenty-five high quality verbal feedback behaviours of educators (Johnson et al., 2016). M. A. Quance et al.: Nursing Students’ Perceptions of Anecdotal Notes Some exemplars of these behaviours include: educator’s comments were based on observed performance, were discussed as soon as was practicable, comments are intended to help, not criticize, the learner needs to be engaged in the discussion and identify similarities between the learner’s performance and the target performance, and educator’s comments are focused on the learner’s actions, not personal characteristics. Zsohar and Smith (2009) state, “…how the feedback is delivered to learners is just as important as what feedback is delivered” (p. 242). Although verbal feedback allows for immediate formative feedback and building of confidence, (which are student interests), the disadvantage of verbal feedback is that it cannot be “replayed” or replicated. When final grade appeals or allegations of unfair assessments are received, the reliance on only verbal feedback quickly can move to a “he/she said, he/she said” scenario which is difficult to adjudicate. A written feedback process allows for delayed reinforcement of verbal formative feedback, and permits students to review and refer to the clinical performance data more than once. There is then evidence of transparent evaluation processes and therefore due process, which are interests of the faculty and administrators. Strengths and limitations Strengths of the study are in the number of students who were surveyed and the response rate. These numbers give some assurance that the results of the survey accurately reflect the experiences of this cohort. Another strength of the study is evident in the number of qualitative responses, both positive and negative, indicating the student engagement with the survey tool and the desire to have their voices heard. The survey tool and method were pre-tested in the quality improvement study. One limitation to this study is the number of students (48) who did not participate. Attendance was not taken at the course orientation. It is not known how many students chose not to participate in the study or who were not present to participate. During the data analysis, it became evident that the survey tool requires revision. Question 3, asking “What type of feedback if most useful to your clinical learning” was confusing to students. Many students identified many more than three feedback opportunities. A Likert-type scale and/or some other tool that allows multiple choices would have been more valuable and would have increased the knowledge of students’ experiences of formative feedback beyond written 9 and verbal methods and formative and summative evaluations. Another limitation of this study might be the ability of the students to accurately recall their experiences of instructor feedback over a 3-year period. However, given the numbers of students who were able to remember their experience of AN in the previous seven clinical courses of (non-responders varied by course, ranging from 1.3 to 4.4 %) as well as the amount of qualitative feedback, this is not likely a concern. Conclusion It is curious that more formal strategies of providing formative feedback have not been taken up with other nursing programs, given its identified benefits for students, faculties and programs. One reason might be because this process of AN development by CNEs is time consuming. The time it takes to complete AN on each student in a clinical group was identified by conference attendees at a nursing education conference in Western Canada as the major barrier for implementing such a strategy. The availability of qualified clinical nurse educators is always a concern for nursing education programs, particularly when there is more than one program in the same area. Creating time consuming means of documenting student performance may disadvantage a nursing program’s ability to hire sufficient clinical educators. Not all education programs, particularly in international settings, will have the ability to hire qualified clinical educators for small groups of students. These programs will likely need to rely on the preceptor model of clinical instruction for larger numbers of students. Preceptors will not have the time to provide more than cursory verbal feedback and could not be expected to create written feedback documents for several students whose care he/she has likely not observed. Perhaps other strategies should be developed that would somehow capture the intent of written formative feedback, but without the time consuming narrative component. This study makes it clear that formative written and verbal feedback provides benefits to students. Effort should be invested in creating another method that would shorten the time involved in the creation of such documentation, yet meet the principles of formative feedback. This study was conducted with one cohort of students who shared a similar experience of feedback about their clinical performance using AN developed specifically for this nursing program. It is time for this study to be replicated in this program. There has been 10 M. A. Quance et al.: Nursing Students’ Perceptions of Anecdotal Notes significant turnover of faculty, both full time and in the CNE cohort. Additional research should also be conducted with regard to verbal feedback mechanisms, given students’ perceptions of its value. For example, what are the faculty, students’ and administrator’s perceptions of verbal feedback? How are clinical failures appeal processes in other programs managed when verbal feedback is the primary means of providing formative and evaluative feedback? The results of the study are a beginning exploration of the student experience of formative feedback in experiential/clinical settings using a written form, that of AN. Students identified that verbal and written formative feedback from the clinical nurse educator provided valuable and required information that students used to build their confidence in their nursing activities and correct their performance. The written feedback also allowed students to give voice to their clinical experience, and demonstrate their competence for knowledge and skills that the CNE was unable to witness for him/herself. This becomes a blending of feedback with reflection on student practice from the perspective of the CNE and students, meeting the needs of students as well as the nursing program. Students found that “good” anecdotal notes provided before the next clinical experience were of value to their learning. Timely verbal feedback while in the clinical setting was also valued. Students felt that this nursing program’s way of using AN provides transparency of their performance and assisted students with ideas for ongoing modification of their behaviour prior to the next scheduled clinical experience. However, students also felt that verbal feedback should be provided first and with that feedback reinforced in AN format. These strategies met several pedagogical principles, particularly related to the need for formative feedback. Acknowledgements: Dr. Maureen Mitchell, EdD RN, for her contributions to and support of this project. References Billings, D. M., & Halstead, J. (2012). Teaching in nursing: A guide for faculty. St. Louis, MO: Elsevier Saunders. Bok, H., Jaarsma, D., Spruijt, A., Van Beukelen, P., Van Der Vleuten, C., & Teunissen, P. (2016). Feedback-giving behaviour in performance evaluations during clinical clerkships. Medical Teacher, 38, 88–95. doi: 10.3109/0142159X.2015.1017448 Bonnel, W. (2009). Clinical performance evaluation. In D. M. Billings & J. A. Halstead (Eds.), Teaching in nursing: A guide for faculty (3rd ed., pp. 449–466). St. Louis, MO: Saunders Elsevier. Buck, B., Wilkinson, S. T., & Phillips, H. (2014). Preceptor development: Providing effective feedback, part 2. Hospital Pharmacy, 49(6), 521–529. doi: 10.1310/hpj4906-521 Canadian Nurses Protective Society. (2007). Quality documentation: Your best defence. Retrieved from http://www.cnps.ca/uploadfiles/pdf_english/documentation.pdf Clark, I. (2012). Formative assessment: Assessment is for selfregulated learning. Educational Psychology Review, 24, 205–249. doi: 10.1007/s10648-011-9191-6 College and Association of Registered Nurses of Alberta. (2006). Documentation guidelines for registered nurses. Retrieved from http://www.nurses.ab.ca/Carna-Admin/Uploads/ Documentation%20for%20Registered%20Nurses.pdf Deegan, J., Burton, T., & Rebeiro, G. (2012). Clinical assessment and the benefit of the doubt: What is the doubt? Australian Journal of Advanced Nursing, 30(1), 42–49. DeYoung, S. (2009). Teaching strategies for nurse educators (2nd ed.). Upper Saddle River, NJ: Prentice Hall. Duffy, K. (2003). Failing students: A qualitative study of factors that influence the decisions regarding assessment of students’ competence in practice. Caledonian Midwifery and Nursing Research Centre, Glasgow Caledonian University: Scotland. Retrieved February 16, 2015 from http://science.ulster.ac.uk/ nursing/mentorship/docs/nursing/oct11/failingstudents.pdf Emerson, R. J. (2007). Nursing education in the clinical setting. St. Louis, MO: Mosby Elsevier. Govaerts, M., Van de Wiel, M., Schuwirth, L., Van der Vleuten, C., & Muijtjens, A. (2013). Workplace-based assessment: Raters’ performance theories and constructs. Advances in Health Science Education, 18(3), 375–396. Hall, M. A. (2013). An expanded look at evaluating clinical performance: Faculty use of anecdotal notes in the U.S. and Canada. Nurse Education in Practice, 13, 271–276. doi: 10.1016/j.nepr. 2013.02.001 Hall, M. A., Daly, B. J., & Madigan, E. A. (2010). Use of anecdotal notes by clinical nursing faculty: A descriptive study. Journal of Nursing Education, 49(3), 156–159. doi:10.3928/01484834– 20090915–03 Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77, 81–112. doi:10.3102/ 003465430298487 Johnson, C., Keating, J., Boud, D., Dalton, M., Kiegladie, D., Hay, M., … Molloy, E. (2016). Identifying educator behaviours for high quality verbal feedback in health professions education: Literature review and expert refinement. BMC Medical Education, 16, 96. doi: 10.11/s12909-016-0613-5 Juwah, C., Macfarlane-Dick, D., Matthew, B., Nicol, D., Ross, D., & Smith, B. (2004). Enhancing student learning through effective formative feedback. The Higher Education Academy. Retrieved December 30, 2014 from http://www.jisctechdis.ac.uk/assets/ Documents/resources/database/id353_senlef_guide.pdf Larocque, S., & Luhanga, F. L. (2013). Exploring the issue of failure to fail in a nursing program. International Journal of Nursing Education Scholarship, 10(1), 1–8. doi: 10.1515/ijnes-2012-0037 Liberto, R., Roncher, M., & Shellenbarger, T. (1999). Anecdotal notes: Effective clinical evaluation and record keeping. Nurse Educator, 24(6), 15–18. McCarthy, B., & Murphy, S. (2008). Assessing undergraduate nursing students in clinical practice: Do preceptors use assessment strategies? Nurse Education Today, 28(3), 301–313. M. A. Quance et al.: Nursing Students’ Perceptions of Anecdotal Notes Nicol, D., & Macfarlane-Dick, D. (2007). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199–218. doi: 10.1080/03075070600572090 O’Connor, A. B. (2006). Clinical instruction and evaluation: A teaching resource. Toronto, ON: Jones and Bartlett. Orchard, C. (1994). Management of clinical failure in Canadian nursing program. Western Journal of Nursing Research, 16, 317–331. Scanlon, J., Care, W. D., & Gessler, S. (2001). Dealing with the unsafe student in clinical practice. Nurse Educator, 26(1), 23–27. Shute, V. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189. doi: 10.3102/0034654307313795 Smith, M. H., McKay, Y. D., & Richardson, J. (2001). Legal issues related to dismissing students for clinical deficiencies. Nurse Educator, 26(1), 33–38. 11 University of Saskatchewan. (2014). College of Nursing policies. Retrieved December 30, 2014 from http://www.usask.ca/nur sing/policies/index.php Van Der Vleuten, C., Schuwirth, L., Driessen, E., Dijkstram, J., Tigelaar, D., Baartman, L., & Van Tartwijk, J. (2012). A model for programmatic assessment fit for purpose. Medical Teacher, 43(3), 205–214. Wenrich, M., Jackson, M. B., Maestas, R., Wolfhagen, I., & Scherpbier, A. (2015). From Cheerleader to coach: The developmental progression of bedside teachers in giving feedback to early learners. Academic Medicine, 90(11), S91–S97. Zsohar, H., & Smith, J. (2009). The power of And and But in constructive feedback on clinic performance. Nurse Educator, 34(6), 241–243. Copyright of International Journal of Nursing Education Scholarship is the property of De Gruyter and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.