Update Fiddler Guardrails API to match new specification

[kevinfiddlerai]

• Change Safety API from 'prompt' to 'input' field
• Change Faithfulness API to use strings instead of lists
• Add fdl_roleplaying category to safety detection
• Add unit tests to verify request payload format

Signed-off-by: Kevin Truong kevin@fiddler.ai

Description

Made changes to account for new Fiddler Guardrail API formatting and fdl_roleplaying category

Related Issue(s)

None

Fiddler Reviewer: @copperstick6

Checklist

• [ x] I've read the CONTRIBUTING guidelines.
• [ x] I've updated the documentation if applicable.
• [ x] I've added tests if applicable.
• [ x] @mentions of the person or team responsible for reviewing proposed changes.

[greptile-apps]

Greptile Overview

Greptile Summary

Updated Fiddler Guardrails API integration to match the new specification. The changes simplify the request format by using strings instead of lists and add support for the new fdl_roleplaying safety detection category.

Key changes:

• Changed Safety API request format from {"prompt": [message]} to {"input": message} for both user and bot message checks
• Changed Faithfulness API request format from lists to strings: {"response": [message], "context": [chunks]} to {"context": chunks, "response": message}
• Added fdl_roleplaying to the list of safety detection categories evaluated by the guardrails
• Added comprehensive unit tests to verify the new request payload formats match the API specification

The changes are well-tested with three new unit tests that mock the API calls and verify the exact structure of the outgoing requests, ensuring backward compatibility is broken intentionally to match the new API spec.

Confidence Score: 5/5

• This PR is safe to merge with no identified risks
• The changes are straightforward API format updates with comprehensive test coverage. All three new unit tests directly verify the request payload structure, ensuring the integration matches the new Fiddler API specification. The changes are internally consistent (relevant_chunks is already a string in the codebase), and all existing tests were updated to include the new fdl_roleplaying category.
• No files require special attention

Important Files Changed

Filename	Overview
nemoguardrails/library/fiddler/actions.py	Updates API request format from 'prompt' (list) to 'input' (string) for safety checks and from lists to strings for faithfulness checks; adds fdl_roleplaying safety category
tests/test_fiddler_rails.py	Adds fdl_roleplaying to existing test mocks and includes comprehensive unit tests verifying the new request payload format for safety and faithfulness endpoints

Sequence Diagram

sequenceDiagram
    participant Client as NeMo Guardrails
    participant SafetyAction as call_fiddler_safety_user/bot
    participant FaithAction as call_fiddler_faithfulness
    participant API as Fiddler API

    Note over Client,API: Safety Guardrail Flow (User/Bot Message)
    Client->>SafetyAction: context with user_message or bot_message
    SafetyAction->>SafetyAction: Extract message from context
    SafetyAction->>SafetyAction: Build payload: {"data": {"input": message}}
    SafetyAction->>API: POST /v3/guardrails/ftl-safety
    API-->>SafetyAction: Response with safety scores (fdl_harmful, fdl_violent, etc., fdl_roleplaying)
    SafetyAction->>SafetyAction: Calculate max score across all categories
    SafetyAction->>SafetyAction: Compare score >= threshold
    SafetyAction-->>Client: Return boolean result

    Note over Client,API: Faithfulness Guardrail Flow
    Client->>FaithAction: context with bot_message and relevant_chunks
    FaithAction->>FaithAction: Extract bot_message and relevant_chunks
    FaithAction->>FaithAction: Build payload: {"data": {"context": relevant_chunks, "response": bot_message}}
    FaithAction->>API: POST /v3/guardrails/ftl-response-faithfulness
    API-->>FaithAction: Response with fdl_faithful_score
    FaithAction->>FaithAction: Compare score <= threshold
    FaithAction-->>Client: Return boolean result

Loading

[greptile-apps]

_{2 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[will-fiddler]

@Pouyanpi Can you take a look?

[Pouyanpi]

Thank you @kevinfiddlerai, lgtm! Just need to run pre-commits and push to resolve the Lint job failure.

poetry run pre-commit run --all-files

[kevinfiddlerai]

Thank you @Pouyanpi, I just re-pushed above