feat: add support for system messages to RunnableRails#1106
Merged
Pouyanpi merged 2 commits intoNVIDIA-NeMo:developfrom May 5, 2025
Merged
feat: add support for system messages to RunnableRails#1106Pouyanpi merged 2 commits intoNVIDIA-NeMo:developfrom
Pouyanpi merged 2 commits intoNVIDIA-NeMo:developfrom
Conversation
Collaborator
|
Thank you @smruthi33 for opening this PR 🚀 Would be great if you can add a test to |
ac471ce to
d0d38ef
Compare
Collaborator
|
@tgasser-nv I added some tests. I think we are good to merge after your review: I am not going to include following minimal integration test but it works. import pytest
import os
from langchain_openai import ChatOpenAI
from langchain_core.messages import AIMessage, HumanMessage, SystemMessage
from langchain_core.prompt_values import ChatPromptValue
from nemoguardrails import RailsConfig
from nemoguardrails.integrations.langchain.runnable_rails import RunnableRails
requires_openai = pytest.mark.skipif(
os.environ.get("OPENAI_API_KEY") is None,
reason="Requires OPENAI_API_KEY environment variable",
)
@requires_openai
def test_system_message_with_openai():
"""Tests SystemMessage handling with a real OpenAI model."""
try:
llm = ChatOpenAI(model="gpt-4o", temperature=0)
except ImportError:
pytest.skip("langchain-openai not installed")
except Exception as e:
pytest.skip(f"Could not initialize ChatOpenAI: {e}")
config = RailsConfig.from_content(config={"models": []})
model_with_rails = RunnableRails(config)
chain = model_with_rails | llm
input_messages = [
SystemMessage(
content="You are an assistant that always replies starting with 'LOLOLOLO!'"
),
HumanMessage(content="what can you do?"),
]
result = chain.invoke(input=ChatPromptValue(messages=input_messages))
assert isinstance(result, AIMessage)
assert result.content.lower().startswith("lolololo!") |
Pouyanpi
approved these changes
May 1, 2025
d0d38ef to
c7707ab
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## develop #1106 +/- ##
========================================
Coverage 68.00% 68.00%
========================================
Files 161 161
Lines 15793 15795 +2
========================================
+ Hits 10740 10742 +2
Misses 5053 5053
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The current implementation does not support messages that have a System Message
In this setup,
RunnableRailsdoes not handle messages containing system roles. Attempting to pass such a structure as a dictionary causes conflicts with the format expected by theRunnableinterface from LangChain, which assumes a flat input structure and does not support multiple message types out of the box. This change addresses that limitation.Related Issue(s)
Checklist