AI-Flagged Paper Evaluation Process

As the use of artificial intelligence tools in academic writing continues to grow, faculty should take a proactive and fair approach when evaluating AI-flagged student work. One important precaution instructors can take is to explicitly require students (as prat of the course syllabus) to keep a time-stamped copy of all their drafts, in either Google Docs or Word or whatever format the student is using to construct the document. This is for the student's protection and could serve as "proof" that their document was not entirely AI-generated.

To further ensure fairness and accuracy in evaluating  AI-flagged papers, faculty can follow this flowchart. This guide was created in collaboration with the Stony Brook University Academic Integrity Judiciary Committee. Best practices include carefully reviewing AI detection reports, cross-referencing flagged content with course guidelines, engaging students in discussion about their writing process, and documenting findings thoroughly. It can help faculty make informed decisions that uphold academic integrity while supporting student learning.

• Step 1: Review the AI Report

  Action:

  Examine the flagged paper within the Turnitin AI detection tool.

  Focus:

  • Use TurnitIn's AI Writing Detector and Paraphrase Detection to identify flagged sections.
  • Assess the AI-generated percentage and determine if the flagged content aligns with AI usage concerns.
  • Understand of AI Detector's limitations (E.g.: file size, text length, format, etc.)

  
  Decision Point:

  If the flagged content seems minor or likely a false positive, you may decide no further investigation is needed. Otherwise, proceed to the next step.
• Step 2: Cross-check with Assignments

  Action:

  Compare the flagged sections with:

  • Course assignment guidelines and requirements.
  • Course AI policy in your syllabus.
  • Known patterns or prior work submitted by the student (E.g., writing style or complexity).

   

  Focus:

  Evaluate whether the flagged content deviates significantly from expectations or previous submissions.

  Decision Point:

  If inconsistencies suggest potential AI misuse, move to the next step.

• Step 3: Meet with the Student

  Action:

  Arrange a meeting with the student to discuss the flagged paper.

  Focus:

  • Present the flagged sections and ask for clarification or an explanation.
  • Ask questions to gather further information: "Did you use any translation apps, like Google Translate? Can you tell me why you think that could have happened? I'd like to see if we can go over your 'revision history'. Did you happen to 'correct' your paper with Grammarly?"
  • Assess the student's understanding of the flagged content (E.g., their ability to explain thought processes or provide drafts, additional notes, or revisions).

   

  Outcome:

  Use this discussion to determine if the flagged content is genuine, AI-generated, or unintentionally problematic.

• Step 4: Determine Next Steps

  Action:

  Decide on the appropriate course of action based on the evaluation.

  Options:

  • Teaching Moment: If the issue appears unintentional, provide guidance on proper citation, AI usage, or academic integrity.
  • Warning: For minor issues, issue a formal warning and document it in the student's record.
  • Formal Report: If there is substantial evidence of intentional AI misuse, proceed with an official academic integrity report.
• Step 5: Document Findings

  Action:

  Thoroughly document the evaluation process using the institution's accusation reporting template.

  Focus:

  • Include observations from the AI report.
  • Summarize discussions from the student meeting.
  • Attach supporting materials such as drafts or earlier submissions.

   

  Goal:

  Create a detailed narrative that supports your findings and decisions.

• Step 6: Submit Report

  Action:

  If formal reporting is necessary, submit the completed report and supporting documentation through official university channels.

  Focus:

  • Ensure all required fields in the reporting template are filled.
  • Attach all relevant files, including the AI detection report and meeting notes.

   

  Outcome:

  The university's Academic Integrity Office will handle further processing and follow-up.

Additional Notes:

• Instructors who assign written documents should explicitly instruct students (as part of the course syllabus) to keep a time-stamped copy of all their drafts to serve as "proof" that their document was not entirely AI-generated.
• Throughout the process, maintain clarity and fairness, ensuring that students understand the evaluation and decision-making process.
• Emphasize transparency and documentation to minimize confusion or disputes.
• Use this flowchart as a quick reference guide for consistent, informed actions across all flagged cases.