-
Notifications
You must be signed in to change notification settings - Fork 28
Un 3388 Fixed Gemini LLM test case failure #330
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -15,7 +15,8 @@ | |
|
|
||
| { | ||
| "llm_config": { | ||
| "model_name": "gemini-2.0-flash", | ||
| "class": "gemini", | ||
| "model_name": "gemini-2.5-flash", | ||
| }, | ||
| "tools": [ | ||
| # These tool definitions do not have to be in any particular order | ||
|
|
@@ -56,13 +57,16 @@ You’re Music Nerd Pro, the go-to brain for all things rock, pop, and everythin | |
| • “What’s a hidden gem I probably missed?” | ||
| You’re equal parts playlist curator, music historian, and pop culture mythbuster—with a sixth sense for sonic nostalgia and a deep respect for the analog gods. | ||
|
|
||
| This service comes for a fee. For each question you're about to answer, use your Accountant tool to calculate the | ||
| running fees. | ||
| This service comes at a fee. For each question you're about to answer, use your Accountant tool to calculate the | ||
| running fees. | ||
|
|
||
| - You must call the Accountant exactly once per user question — no more, no less. | ||
| - You must not estimate, guess, or invent the cost under any circumstances. | ||
|
|
||
| Once you receive the updated running cost, respond with a JSON object that has exactly two keys: | ||
| 1. "answer" – your full answer to the user’s question. | ||
| 2. "running_cost" – the updated cost returned by the Accountant tool. | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I replaced the prompt used by Ollama and Anthropic LLM test cases.
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It is Great and Good that we are trying to use the same prompt across multiple llms |
||
|
|
||
| For each question you receive, call your Accountant tool to calculate the running cost. Otherwise you won't get paid! | ||
| Then answer with a JSON message that has two keys: | ||
| 1. An "answer" key whose value has the answer to the question | ||
| 2. A "running_cost" key whose value has the running cost computed by the Accountant tool. | ||
| """, | ||
| "tools": ["Accountant"] | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -38,7 +38,7 @@ langchain>=0.3.15,<0.4 | |
| langchain-anthropic>=0.3.11,<0.4 | ||
| langchain-aws>=0.2.27,<0.3 | ||
| langchain-community>=0.3.19,<0.4 | ||
| langchain-google-genai>=2.0.11,<3.0 | ||
| langchain-google-genai>=2.1.8,<3.0 | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Bumped up the version. |
||
| langchain-openai>=0.3.28,<0.4 | ||
| langchain-nvidia-ai-endpoints>=0.3.8,<0.4 | ||
| langchain-ollama>=0.3.4,<0.4 | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I switched to another model name.