Reminders for collegial reviews and other datafolio reviews

Dear AATNs,

Attached please find some notes and reminders that may be helpful when conducting collegial reviews and ongoing datafolio reviews. These notes were based on some typical situations and trends in datafolios that were reviewed at Benchmarking.

Happy Holidays to All!

Regards,
Kelly

Kelly Ickes
Special Education Specialist
Measured Progress
1-800-431-8901 x-2219
kickes@measuredprogress.org

NYSAA Benchmarking

December 4-6, 2007

AATN Notes

The following is a list of reminders and recommendations based on the most common issues found in the content area samples reviewed during the recent NYSAA Benchmarking activity. Please review and share this list with your regions directly and if possible during your collegial review sessions. Errors in documentation and transcription of data may jeopardize a student receiving a reportable score. Please note, these recommendations are intended to stress key points in the administration guidelines for 2007-08 NYSAA. They are not to be interpreted as scoring decision rules.

STEPS FOR COMPLETING A DATAFOLIO:

· Review the student’s date of birth and confirm the appropriate grade was assessed based on the NYSAA Birth Date Chart on page 6 of the NYSAA Administration Manual.

· Review the Steps for Completing a NYSAA Datafolio on page 14 of the NYSAA Administration Manual (September 2007) and confirm that the steps are being followed by teachers. Most importantly, confirm that steps 4 and 6 are followed in the correct order. In some cases, choosing a sample assessment task before choosing an Alternate Grade Level Indicator (AGLI) may cause issues with the connection to grade level content as indicated on the scoring rubric.

· Confirm that the verifying evidence clearly reflects the indicated assessment task (i.e., it is confusing when a task indicates symbols and all evidence shows words)

· Review the verifying evidence, assessment task, and AGLI to ensure that they are aligned and make adjustments as necessary if a question arises.

o Scenario #1: An assessment task indicates that the student will work on identifying quadrilaterals, but the verifying evidence shows the student identifying quadrilaterals and triangles. This is OK, but it would be best if the assessment task is adjusted to reflect both of the skills being worked on. Be sure that the alignment to the AGLI is still evident.

o Scenario #2: An assessment task indicates that the student will work on identifying quadrilaterals. The first piece of verifying evidence shows the student identifying quadrilaterals and the second piece of verifying evidence shows the student identifying triangles. In this case, the second piece of verifying evidence does not connect to the assessment task as written. An adjustment should be made to make the connection of the verifying evidence to the assessment task clear.)

· Be sure to double check the order of documents in each content area. Misplaced documents are difficult to review and score. Utilize and refer to the Table of Contents sample in the Administration Manual. Please remember, Observer Verification Forms should be placed directly behind the Data Collection Sheet that they are supporting.

DOCUMENTATION REMINDERS ON THE VERIFYING EVIDENCE:

· Complete Observer Verification Form in full, including the required elements for all verifying evidence, observer title, signature, and date of signature.

· Be sure that the staff initial and staff key sections of the Data Collection Sheets are complete. The staff key should include the name(s) of anyone who recorded data on the Data Collection Sheets.

· The person whose initials appear in the staff key and below the data cannot be the person who signs the OVF (the observer)

· Be sure that the data and dates on the verifying evidence, VE label, and Data Summary Sheets match.

· Annotate work samples to help scorers understand the verifying evidence. Some worksheets do not include directions or the directions are not clear as to what the student was actually asked to do.

· Double check that all calculations of level of accuracy and level of independence are correct and are applicable percentages based on the assessment task being demonstrated in the verifying evidence. Please refer to page 28 of the NYSAA Administration Manual. If the percentages don’t make sense based on the evidence (i.e. there are 4 questions/answers, but the teacher indicates 80% for the level of independence) it could jeopardize the student’s score.

· Please remember that student responses that are generated by providing the student with hand-over-hand assistance affect the independence rating. However, if the student is indicating his or her answers and hand-over-hand assistance is provided to paste the indicated answer on a worksheet, this would not affect the independence rating.

· Confirm with the teachers that an Informed Consent for Photographing and Video/Audio Tape Recording form for the student is completed by the parent/family/guardian if photographs, video tapes, or audio tapes are included in the datafolio. Confirm that an Informed Consent for Incidental Photographing and Video/Audio Tape Recording of Classroom Peers is completed by the parent/family/guardian of the peers if they are seen or heard on the verifying evidence. Both consent forms are kept on file at the school district. (i.e., a piece of verifying evidence contains a student attendance chart with pictures of each of the students in the classroom. All of the students on the chart would need to have consent forms on file at the school district.)

· Do not include verifying evidence for the first date of student performance recorded on the Data Summary Sheet. Submit only two pieces of verifying evidence that correspond to the last two dates of student performance data for each assessed AGLI.

· Verifying evidence labels MUST be affixed to the actual VE (either the front if it does not obstruct student work or labels can be placed on the back of evidence).

· Do not use correction fluid/tape, black out, or any photocopies (in part or whole).

· One piece of verifying evidence is required for each of the last two dates of student performance documented on the DSS. When submitting charts/calendars please refer to page 22 of the NYSAA Administration Manual. Charts/calendars can only be submitted as one piece of evidence for a single date.

DOCUMENTATION REMINDERS ON THE DATA SUMMARY SHEET:

· Double check that all information is accurately recorded on the Data Summary Sheets, including demographic info, choice components check, AGLI codes and AGLI texts, etc.

· Do not use correction fluid/tape, black out, or any photocopies (in part or whole).

· It is helpful to indicate where the assessment task came from by marking the box where it indicates if the assessment task was one that is the same or comparable to a sample assessment task from the Frameworks (Appendix G-September 2007) or if it is an original assessment task.

· Remind teachers to indicate the sample assessment task (SAT) code as well as the page number on which the sample assessment task was located in the Frameworks (Appendix G-September 2007).

PROFILE^TM REMINDERS:

· Double check that the information entered in ProFile^TM is accurate. It is recommended that teachers print out and carefully review all documents completed using ProFile. Make sure that if a change is made to one document, that all related documents are also update. (i.e. In one case, it appears a change was made on a Data Collection Sheet, but it was not saved in a manner that would update the Data Summary Sheet information. This made it difficult to confirm the student performance ratings.)

· Remind teachers that the VE labels printed out using ProFile^TM will print the last two dates of student performance data recorded on the Data Summary Sheets. The VE labels for these dates must be separated and attached to each piece of verifying evidence.

· When using ProFile^TM review all printed pages to ensure that all information has printed properly. Teachers may need to review the printing guidelines in the User Manual (available on the ProFile log in page). Margins may need to be formatted or the page orientation may need to be adjusted depending on your printer.