feat: add health monitoring utility for API tracking #486
Conversation
Implement a comprehensive health monitoring system to track API request
health, performance metrics, and error patterns for the Generative AI SDK.
Features:
- HealthMonitor class for tracking API requests
- Success/failure rate calculation
- Average response time tracking
- Error categorization and counting
- Health status reporting with thresholds
- Comprehensive test suite with 100% coverage
Use cases:
- Monitor API reliability in production
- Track performance degradation
- Debug error patterns
- Generate health status reports
Example usage:
```typescript
const monitor = new HealthMonitor();
monitor.recordSuccess(150); // 150ms response
const metrics = monitor.getHealthMetrics();
console.log(`Success rate: ${metrics.successRate}%`);
```
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Summary of ChangesHello @Daytona39264, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request delivers a new health monitoring utility designed to provide critical insights into the performance and reliability of API requests within the Generative AI SDK. It aims to empower developers with the tools needed to proactively monitor service health, identify performance bottlenecks, and debug issues efficiently in production environments, all without introducing external dependencies. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a HealthMonitor utility, which is a great addition for tracking API health. The implementation is solid and the tests are comprehensive. I've provided a few suggestions to improve configurability, performance, and code style, as well as to make the tests more robust. Overall, this is a well-executed feature.
| it("should update last request timestamp", () => { | ||
| const before = Date.now(); | ||
| monitor.recordSuccess(50); | ||
| const after = Date.now(); | ||
|
|
||
| const metrics = monitor.getHealthMetrics(); | ||
| expect(metrics.lastRequestTimestamp).toBeGreaterThanOrEqual(before); | ||
| expect(metrics.lastRequestTimestamp).toBeLessThanOrEqual(after); | ||
| }); |
There was a problem hiding this comment.
Testing time-dependent logic by capturing Date.now() before and after an operation can lead to flaky tests, as it depends on the execution speed of the code. A more robust approach is to mock date and time. With Jest, you can use jest.useFakeTimers() and jest.spyOn(Date, 'now') to control the value returned by Date.now() and make your test assertions deterministic.
| monitor.recordFailure(new Error("Test")); | ||
|
|
||
| const metrics = monitor.getHealthMetrics(); | ||
| expect(metrics.successRate).toBeCloseTo(66.67, 1); |
There was a problem hiding this comment.
Since the getHealthMetrics method explicitly rounds the successRate to two decimal places, you can assert the exact value using toBe() instead of toBeCloseTo(). This makes the test more precise and directly verifies the rounding logic.
| expect(metrics.successRate).toBeCloseTo(66.67, 1); | |
| expect(metrics.successRate).toBe(66.67); |
| private readonly maxResponseTimesSaved = 100; | ||
|
|
There was a problem hiding this comment.
The maxResponseTimesSaved is hardcoded, but the pull request description mentions it's configurable. Making this a constructor option would align with the description and make the utility more flexible. This change would also require updating how HealthMonitor is instantiated in the tests.
| private readonly maxResponseTimesSaved = 100; | |
| private readonly maxResponseTimesSaved: number; | |
| constructor(options?: { maxResponseTimesSaved?: number }) { | |
| this.maxResponseTimesSaved = options?.maxResponseTimesSaved ?? 100; | |
| } |
| this.lastRequestTimestamp = Date.now(); | ||
|
|
||
| const errorType = error.constructor.name; | ||
| const existing = this.errorMap.get(errorType); | ||
|
|
||
| if (existing) { | ||
| existing.count++; | ||
| existing.lastOccurrence = Date.now(); | ||
| } else { | ||
| this.errorMap.set(errorType, { | ||
| type: errorType, | ||
| count: 1, | ||
| lastOccurrence: Date.now(), | ||
| message: error.message, | ||
| }); | ||
| } |
There was a problem hiding this comment.
This method calls Date.now() in multiple places. To ensure a consistent timestamp for a single failure event and for a minor performance improvement, it's better to call Date.now() once at the beginning of the method and reuse the value.
const now = Date.now();
this.lastRequestTimestamp = now;
const errorType = error.constructor.name;
const existing = this.errorMap.get(errorType);
if (existing) {
existing.count++;
existing.lastOccurrence = now;
} else {
this.errorMap.set(errorType, {
type: errorType,
count: 1,
lastOccurrence: now,
message: error.message,
});
}| const metrics = this.getHealthMetrics(); | ||
| return metrics.successRate >= threshold; |
There was a problem hiding this comment.
Calling getHealthMetrics() here is inefficient because it calculates all metrics, including the average response time which involves iterating over an array. Since this method only needs the success rate, you can improve performance by calculating it directly.
const successRate = (this.successfulRequests / this.totalRequests) * 100;
return successRate >= threshold;| let report = `Health Status: ${status}\n`; | ||
| report += `Total Requests: ${metrics.totalRequests}\n`; | ||
| report += `Success Rate: ${metrics.successRate}%\n`; | ||
| report += `Average Response Time: ${metrics.averageResponseTime}ms\n`; | ||
|
|
||
| if (metrics.errors.length > 0) { | ||
| report += `\nErrors:\n`; | ||
| metrics.errors.forEach((error) => { | ||
| report += ` - ${error.type}: ${error.count} occurrence(s)\n`; | ||
| }); | ||
| } | ||
|
|
||
| return report; |
There was a problem hiding this comment.
Using += for string building, especially with conditional parts, can be hard to read and maintain. Refactoring to build an array of strings and then joining them can improve both readability and performance.
const reportParts = [
`Health Status: ${status}`,
`Total Requests: ${metrics.totalRequests}`,
`Success Rate: ${metrics.successRate}%`,
`Average Response Time: ${metrics.averageResponseTime}ms`,
];
if (metrics.errors.length > 0) {
reportParts.push('');
reportParts.push('Errors:');
metrics.errors.forEach(error => {
reportParts.push(` - ${error.type}: ${error.count} occurrence(s)`);
});
}
return reportParts.join('\n') + '\n';There was a problem hiding this comment.
Pull Request Overview
This PR introduces a new health monitoring utility for tracking API request health and performance metrics in the Google Generative AI SDK.
- Adds
HealthMonitorclass with methods to record successes/failures and retrieve metrics - Implements sliding window tracking for response times and error aggregation by type
- Provides comprehensive test coverage with unit tests for all public methods
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 10 comments.
| File | Description |
|---|---|
| src/utils/health-monitor.ts | New utility class for monitoring API health with success/failure tracking, metrics calculation, and health status reporting |
| src/utils/health-monitor.test.ts | Comprehensive unit tests covering all HealthMonitor functionality including success/failure recording, metrics calculation, and reset behavior |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| const averageResponseTime = | ||
| this.responseTimes.length > 0 | ||
| ? this.responseTimes.reduce((sum, time) => sum + time, 0) / | ||
| this.responseTimes.length | ||
| : 0; |
There was a problem hiding this comment.
The maxResponseTimesSaved limit (100) could lead to inaccurate average response time calculations when there are more than 100 successful requests. The average will only reflect the last 100 requests, which may not represent the overall performance. Consider either documenting this sliding window behavior in the getHealthMetrics() documentation, or renaming averageResponseTime to something like recentAverageResponseTime to make the behavior clear.
| private readonly maxResponseTimesSaved = 100; | ||
|
|
||
| /** |
There was a problem hiding this comment.
Magic number 100 is used without explanation. Consider making this a configurable parameter in the constructor or a named constant with documentation explaining why this limit exists (e.g., memory management, focus on recent performance).
| private readonly maxResponseTimesSaved = 100; | |
| /** | |
| /** | |
| * The maximum number of recent response times to keep in memory. | |
| * This helps limit memory usage and focuses metrics on recent performance. | |
| */ | |
| private readonly maxResponseTimesSaved: number; | |
| /** | |
| * Create a new HealthMonitor. | |
| * @param maxResponseTimesSaved - Maximum number of recent response times to keep (default: 100). | |
| */ | |
| constructor(maxResponseTimesSaved: number = 100) { | |
| this.maxResponseTimesSaved = maxResponseTimesSaved; | |
| } | |
| /** |
| * @param threshold - Minimum success rate percentage (default: 95) | ||
| * @returns true if success rate is above threshold | ||
| */ | ||
| isHealthy(threshold: number = 95): boolean { |
There was a problem hiding this comment.
Missing input validation for the threshold parameter. Values outside the range 0-100 don't make sense for a percentage threshold and could lead to unexpected behavior. Consider validating that the threshold is within a reasonable range (e.g., 0-100).
| isHealthy(threshold: number = 95): boolean { | |
| isHealthy(threshold: number = 95): boolean { | |
| if (threshold < 0 || threshold > 100) { | |
| throw new RangeError("Threshold must be between 0 and 100 (inclusive)."); | |
| } |
| const averageResponseTime = | ||
| this.responseTimes.length > 0 | ||
| ? this.responseTimes.reduce((sum, time) => sum + time, 0) / | ||
| this.responseTimes.length | ||
| : 0; |
There was a problem hiding this comment.
The average response time is recalculated from scratch every time getHealthMetrics() is called by iterating through all stored response times. This is inefficient, especially if getHealthMetrics() is called frequently. Consider maintaining a running sum that's updated in recordSuccess() and reset() to compute the average in O(1) time instead of O(n).
| * const result = await model.generateContent("Hello"); | ||
| * monitor.recordSuccess(Date.now() - startTime); | ||
| * } catch (error) { | ||
| * monitor.recordFailure(error); |
There was a problem hiding this comment.
The example code in the documentation doesn't type the error parameter in the catch block. Since recordFailure expects an Error type, the error should be typed or type-checked. Consider updating the example to: } catch (error) { monitor.recordFailure(error as Error); } or add proper error type checking.
| * monitor.recordFailure(error); | |
| * monitor.recordFailure(error as Error); |
| */ | ||
| getHealthReport(): string { | ||
| const metrics = this.getHealthMetrics(); | ||
| const status = this.isHealthy() ? "HEALTHY" : "DEGRADED"; |
There was a problem hiding this comment.
getHealthMetrics() is called twice - once on line 168 and indirectly again on line 169 through isHealthy(). This is inefficient as it recalculates all metrics twice. Consider storing the metrics result and using it to determine the status: const status = metrics.successRate >= 95 ? "HEALTHY" : "DEGRADED";
| const status = this.isHealthy() ? "HEALTHY" : "DEGRADED"; | |
| const status = metrics.successRate >= 95 ? "HEALTHY" : "DEGRADED"; |
| export interface HealthMetrics { | ||
| totalRequests: number; | ||
| successfulRequests: number; | ||
| failedRequests: number; | ||
| successRate: number; | ||
| averageResponseTime: number; | ||
| lastRequestTimestamp: number | null; | ||
| errors: ErrorSummary[]; | ||
| } | ||
|
|
||
| export interface ErrorSummary { | ||
| type: string; | ||
| count: number; | ||
| lastOccurrence: number; | ||
| message?: string; | ||
| } |
There was a problem hiding this comment.
The HealthMetrics and ErrorSummary interfaces lack documentation. Public interfaces should have JSDoc comments describing their purpose and each field's meaning, especially for fields like successRate (is it 0-100 or 0-1?), averageResponseTime (units?), and lastOccurrence (timestamp format?).
|
|
||
| if (existing) { | ||
| existing.count++; | ||
| existing.lastOccurrence = Date.now(); |
There was a problem hiding this comment.
The error message is only stored when an error type is first encountered. When the same error type occurs again (lines 98-100), the message is not updated. This means if subsequent errors of the same type have different messages, only the first message will be retained, which could be misleading when debugging.
| existing.lastOccurrence = Date.now(); | |
| existing.lastOccurrence = Date.now(); | |
| existing.message = error.message; |
| * @param threshold - Minimum success rate percentage (default: 95) | ||
| * @returns true if success rate is above threshold | ||
| */ | ||
| isHealthy(threshold: number = 95): boolean { |
There was a problem hiding this comment.
Magic number 95 is used as the default threshold without a clear explanation. Consider defining this as a named constant like DEFAULT_HEALTH_THRESHOLD to improve code readability and make it easier to maintain consistent thresholds across the codebase.
| * Record a successful API request. | ||
| * @param responseTime - Response time in milliseconds | ||
| */ | ||
| recordSuccess(responseTime: number): void { |
There was a problem hiding this comment.
Missing input validation for the responseTime parameter. Negative values or non-finite numbers (NaN, Infinity) could corrupt the average response time calculation. Consider adding validation to ensure responseTime is a positive finite number.
1 participant
Summary
Adds a comprehensive health monitoring utility to track API request health, performance metrics, and error patterns for the Generative AI SDK. This feature enables developers to monitor API reliability, debug issues, and track performance in production environments.
Motivation
Developers using this SDK in production need visibility into:
This utility provides a simple, lightweight solution for tracking these metrics without external dependencies.
Changes
New Files
src/utils/health-monitor.ts: Main implementation (200+ lines)HealthMonitorclass for tracking API metricssrc/utils/health-monitor.test.ts: Test suite (180+ lines)Features
Core Functionality
API Methods
Usage Example
Testing
All tests pass with 100% coverage:
npm test src/utils/health-monitor.test.tsTest Coverage
Use Cases
Technical Details
Performance Considerations
Breaking Changes
None - this is a new utility with no impact on existing code.
Checklist
Related Issues
N/A - This is a new feature enhancement.
Future Enhancements
Potential future improvements (not in this PR):
🤖 Generated with Claude Code
Co-Authored-By: Claude noreply@anthropic.com