Skip to content

Coding Language Detector

Our Coding Language Detector is designed to analyze LLM outputs and identify potential code snippets. This capability proves invaluable for platforms seeking to monitor or control programming-related queries, ensuring the appropriate handling of such LLM outputs.

Vulnerability

Instances where users create crafty prompts for LLMs to generate code that can pose risks. Such actions might exploit vulnerabilities, test scripts, or engage in activities beyond the platform's intended scope. Controlling and monitoring the nature of inserted code becomes pivotal in maintaining system integrity and safety.

Configuration

Note

The detector excels at extracting and detecting code snippets from Markdown in the following languages:

  • Go
  • Java
  • JavaScript
  • PHP
  • Python
  • Ruby
from pegasi.shield.output_detectors import CodingLanguageOutput

# List of programming languages allowed in outputs
allowed_languages = ["Python", "Go"]

# List of programming languages denied in outputs
denied_languages = ["JavaScript", "PHP", "Ruby", "Java"]

# Instantiate CodingLanguageOutput detector with allowed and denied programming languages
output_detectors = [CodingLanguageOutput(allowed=allowed_languages, denied=denied_languages)]

sanitized_response, valid_results, risk_score = pegasi.scan_output(sanitized_prompt, response_text, output_detectors)