AI Prompt Injection Poses New Document-Review Risk
Recent court decisions highlight the emerging risk of 'prompt injection,' where hidden instructions in documents can manipulate AI review tools, creating reliability and integrity issues in litigation and transactional workflows.
'Prompt injection' is an emerging technique where hidden text embedded in a document instructs an AI system to disregard a user's prompt, favor a party, or otherwise skew its analysis. This poses a significant integrity risk for legal and business workflows that rely on AI to review contracts, discovery, or due diligence materials. Recent court decisions show that judicial bodies are taking this conduct seriously. In one Connecticut case, a pro se plaintiff who embedded invisible instructions in pleadings had their e-filing privileges rescinded. In a Brazilian case, a tribunal's AI detected a similar attempt, leading to a fine and professional referrals. Sophisticated counsel should be aware that this risk exists across all externally sourced documents and cannot be addressed by technology alone. Best practices include treating all third-party documents as untrusted, using security preambles in prompts, instructing the AI to flag suspicious content, and ensuring rigorous human oversight to verify AI-generated outputs against source materials.