-
-
Notifications
You must be signed in to change notification settings - Fork 334
feat: make LLM provider temperature configurable #3460
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Add configurable temperature parameter to LLM providers, allowing users to control randomness in AI-generated translations. Temperature is stored as a nullable Double and passed to all provider APIs (OpenAI, Anthropic, Google AI, Ollama). Previously, temperature was either hardcoded or null. Closes #3297 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the
✨ Finishing touches🧪 Generate unit tests (beta)
Comment |
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
ee/backend/app/src/main/kotlin/io/tolgee/ee/service/LlmProviderService.kt (1)
78-96:⚠️ Potential issue | 🟠 MajorAdd provider-specific temperature validation before persisting.
Temperature constraints differ significantly across providers—Anthropic requires 0.0–1.0, while OpenAI accepts 0–2, and Gemini ranges by model. Currently, any numeric value (including NaN, Infinity, or negatives) is accepted without validation and will only fail at provider call time. Add server-side validation with provider-specific bounds so invalid configs are rejected early.
🤖 Fix all issues with AI agents
In
`@backend/data/src/main/kotlin/io/tolgee/dtos/request/llmProvider/LlmProviderRequest.kt`:
- Around line 25-27: Add server-side validation to prevent negative temperatures
in LlmProviderRequest by annotating the temperature property with a
DecimalMin("0.0") constraint; update the temperature field declaration in the
LlmProviderRequest data class (where reasoningEffort and temperature are
defined) to apply `@field`:DecimalMin("0.0") so incoming requests with negative
temperature values are rejected, leaving provider-specific max-range checks for
later per-provider validation logic.
In
`@ee/backend/app/src/main/kotlin/io/tolgee/ee/component/llm/AnthropicApiService.kt`:
- Around line 124-126: The `@JsonInclude` on the constructor property temperature
will be ignored at serialization because Kotlin applies it to the parameter by
default; update the annotation to target the backing field so Jackson sees it by
changing the annotation to use the field use-site target (i.e., add
`@field`:JsonInclude(JsonInclude.Include.NON_NULL)) on the temperature property in
the data class (temperature: Double? = null) so null values are omitted during
JSON serialization.
backend/data/src/main/kotlin/io/tolgee/dtos/request/llmProvider/LlmProviderRequest.kt
Show resolved
Hide resolved
| @JsonInclude(JsonInclude.Include.NON_NULL) | ||
| val temperature: Double? = null, | ||
| ) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧩 Analysis chain
🏁 Script executed:
cd /tmp && cat > test_json_include.kt << 'EOF'
// Check the signature of `@JsonInclude` to understand its targets
import com.fasterxml.jackson.annotation.JsonInclude
// Simplified: `@JsonInclude` is a meta-annotation that typically targets fields/methods/properties
// In Jackson, it applies to the serialization of fields/getters
EOFRepository: tolgee/tolgee-platform
Length of output: 48
🌐 Web query:
Jackson @JsonInclude annotation targets Kotlin property what targets does it have
💡 Result:
@JsonInclude is a Java annotation declared with:
@Target({ ANNOTATION_TYPE, METHOD, FIELD, TYPE, PARAMETER })[1]
So, on a Kotlin property, the meaningful/use-site targets you can use are:
@field:JsonInclude(...)→ applies to the backing field (FIELD)@get:JsonInclude(...)→ applies to the getter (METHOD)@param:JsonInclude(...)→ applies to the constructor parameter (PARAMETER, for primary-constructor properties)
Kotlin use-site targets (field, get, param, etc.) are described in the Kotlin annotations docs [2]. Also note: if you omit a use-site target, Kotlin chooses a default based on what targets the annotation supports (and for constructor properties it tends to pick param first) [2].
Example:
data class A(
`@field`:JsonInclude(JsonInclude.Include.NON_NULL)
val x: String?
)Sources: [1] [2]
🏁 Script executed:
# Search the repository for how `@JsonInclude` is used elsewhere in Kotlin files
rg "@JsonInclude" --type kt --context 3Repository: tolgee/tolgee-platform
Length of output: 92
🏁 Script executed:
# Specifically look at the file in question to understand the data class structure
cat -n ee/backend/app/src/main/kotlin/io/tolgee/ee/component/llm/AnthropicApiService.kt | sed -n '110,140p'Repository: tolgee/tolgee-platform
Length of output: 980
🏁 Script executed:
# Search for other `@JsonInclude` usages in Kotlin files across the repo (without type filter)
rg "@JsonInclude" --context 2 -l | grep -E "\.(kt|kts)$" | head -10Repository: tolgee/tolgee-platform
Length of output: 382
🏁 Script executed:
# Get actual usages to see if `@field`: is used elsewhere
rg "@JsonInclude" --context 3 | head -50Repository: tolgee/tolgee-platform
Length of output: 5185
🏁 Script executed:
# Check if the project uses jackson-module-kotlin
rg "jackson-module-kotlin" --type gradle,xml,toml,propertiesRepository: tolgee/tolgee-platform
Length of output: 116
🏁 Script executed:
# Check the pom.xml or build files for Jackson dependencies
find . -name "pom.xml" -o -name "build.gradle*" | head -5 | xargs grep -l "jackson" | head -3Repository: tolgee/tolgee-platform
Length of output: 147
🌐 Web query:
Jackson Kotlin module @JsonIncludeon constructor property without@field: does it work with jackson-module-kotlin
💡 Result:
No—not reliably for serialization.
In Kotlin, if you write:
data class A(`@JsonInclude`(JsonInclude.Include.NON_NULL) val x: String?)then because @JsonInclude is allowed on PARAMETER, Kotlin will (by default) place it on the constructor parameter (@param:) rather than the backing field/getter. Kotlin’s default target selection prefers param first when multiple targets are possible. [1] And @JsonInclude explicitly supports PARAMETER (among others). [2]
jackson-module-kotlin helps Jackson use constructor parameters for creator-based deserialization, but @JsonInclude is a serialization inclusion rule, so you typically need it on what Jackson uses as the “property” during serialization: the getter or field.
Use one of these:
data class A(`@get`:JsonInclude(JsonInclude.Include.NON_NULL) val x: String?)or
data class A(`@field`:JsonInclude(JsonInclude.Include.NON_NULL) val x: String?)(Optionally, Kotlin also has an @all: meta-target to propagate to multiple relevant JVM elements, if enabled in your compiler settings. [1])
Sources:
[1] Kotlin annotation use-site targets and defaulting rules https://kotlinlang.org/docs/annotations.html
[2] @JsonInclude targets include PARAMETER, FIELD, METHOD, TYPE https://fasterxml.github.io/jackson-annotations/javadoc/2.14/com/fasterxml/jackson/annotation/JsonInclude.html
Add @field: prefix to ensure @JsonInclude applies during serialization.
Without a use-site target, Kotlin defaults to @param: for constructor properties since @JsonInclude supports PARAMETER as a target. Jackson's serializer reads inclusion rules from the field or getter, not the constructor parameter, so the annotation will be ignored and temperature: null will serialize despite NON_NULL configuration.
Fix
- `@JsonInclude`(JsonInclude.Include.NON_NULL)
- val temperature: Double? = null,
+ `@field`:JsonInclude(JsonInclude.Include.NON_NULL)
+ val temperature: Double? = null,📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| @JsonInclude(JsonInclude.Include.NON_NULL) | |
| val temperature: Double? = null, | |
| ) | |
| `@field`:JsonInclude(JsonInclude.Include.NON_NULL) | |
| val temperature: Double? = null, | |
| ) |
🤖 Prompt for AI Agents
In
`@ee/backend/app/src/main/kotlin/io/tolgee/ee/component/llm/AnthropicApiService.kt`
around lines 124 - 126, The `@JsonInclude` on the constructor property temperature
will be ignored at serialization because Kotlin applies it to the parameter by
default; update the annotation to target the backing field so Jackson sees it by
changing the annotation to use the field use-site target (i.e., add
`@field`:JsonInclude(JsonInclude.Include.NON_NULL)) on the temperature property in
the data class (temperature: Double? = null) so null values are omitted during
JSON serialization.
Summary
temperaturefield (Double?) to LLM provider configurationCloses #3297
Summary by CodeRabbit