test(models/openrouter): add e2e tests

[Stream29]

Related Issues or Context

langgenius/dify#19379

This PR contains Changes to LLM Models Plugin

Only test added.

[gemini-code-assist]

Summary of Changes

Hello @Stream29, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the testing infrastructure for the OpenRouter LLM plugin by introducing comprehensive end-to-end tests. The primary goal is to validate the functionality of various OpenRouter models, ensuring they can be correctly invoked and return expected outputs. This improvement aims to bolster the reliability and stability of the OpenRouter integration within the system.

Highlights

• End-to-End Testing: Introduced new end-to-end tests for the OpenRouter LLM models to ensure their proper invocation and response handling.
• Dependency Updates: Added pyrefly and pytest as development dependencies to support the new testing framework.
• Version Increment: The OpenRouter plugin version has been incremented from 0.0.31 to 0.0.32 in the manifest.yaml.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• models/openrouter/manifest.yaml
  • Incremented the plugin version from 0.0.31 to 0.0.32.
• models/openrouter/pyproject.toml
  • Added pyrefly>=0.50.0 to the dev dependency group.
  • Added pytest>=9.0.2 to the dev dependency group.
• models/openrouter/tests/test_llm_call.py
  • Added a new test file for end-to-end LLM invocation testing for OpenRouter models.
  • Implemented get_all_models function to dynamically discover models from _position.yaml.
  • Created a parameterized test_llm_invoke function using pytest to test each discovered model.
  • The test invokes models with a simple prompt and asserts that a non-empty response is received.
  • Requires the OPENROUTER_API_KEY environment variable to be set for test execution.

Activity

• The pull request was opened by Stream29.
• New test files and development dependencies have been added to introduce end-to-end testing for OpenRouter models.
• The plugin version has been updated.
• No other human activity (comments, reviews) has been recorded yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces end-to-end tests for the OpenRouter model provider, which is a valuable addition for ensuring the integration's reliability. The changes correctly add the necessary testing dependencies and a new test file that covers invoking different models. My review includes a couple of suggestions to improve the test implementation by following common testing best practices and enhancing code readability.

[gemini-code-assist]

For better readability and conciseness, this loop can be replaced with a list comprehension. This is a more Pythonic way to create the list of models.

Suggested change

	models: list[str] = []
	for item in data:
	if isinstance(item, str) and item.strip():
	models.append(item.strip())
	return models
	return [item.strip() for item in data if isinstance(item, str) and item.strip()]

[gemini-code-assist]

Instead of raising a ValueError when the OPENROUTER_API_KEY is not set, it's better practice to skip the test using pytest.skip. This prevents the test suite from failing in environments where the key is not available, and provides a clearer output that the test was skipped due to a missing requirement.

Suggested change

	if not api_key:
	raise ValueError("OPENROUTER_API_KEY environment variable is required")
	if not api_key:
	pytest.skip("OPENROUTER_API_KEY environment variable is required to run this test")