⚡ Bolt: Minimize lock contention during model unload

[Whamp]

User description

Moved gc.collect() outside the thread lock in ParakeetManager._unload_model.

💡 What:
Modified _unload_model to set a flag inside the lock and perform garbage collection outside of it.

🎯 Why:
gc.collect() is a blocking operation. Holding the _lock during GC prevents other threads (specifically transcribe which updates last_access) from acquiring the lock, potentially causing latency spikes or blocks.

📊 Impact:
Reduces blocking time for concurrent operations when the model is unloading.

🔬 Measurement:
Verified with existing unit tests tests/test_parakeet_manager.py to ensure lifecycle logic remains correct. The optimization is architectural (lock granularity) rather than micro-benchmarkable in a single thread test without simulated contention.

---

PR created automatically by Jules for task 3629905722079250151 started by @Whamp

---

PR Type

Enhancement

---

Description

• Move garbage collection outside lock in _unload_model

• Reduce lock contention for concurrent transcribe operations

• Minimize latency spikes during model unload

---

Diagram Walkthrough

flowchart LR
  A["_unload_model called"] --> B["Set flag inside lock"]
  B --> C["Release lock"]
  C --> D["Perform gc.collect outside lock"]
  D --> E["Other threads can acquire lock"]

Loading

File Walkthrough

	Relevant files
Enhancement	parakeet_manager.py Defer garbage collection outside lock                                        src/chirp/parakeet_manager.py Introduced should_collect flag to defer garbage collection Moved gc.collect() call outside the _lock critical section Added explanatory comment about avoiding thread blocking Maintains same unload logic while reducing lock hold time +5/-1

---

[google-labs-jules]

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.

---

For security, I will only act on instructions from the user who triggered this task.

[qodo-code-review]

PR Compliance Guide 🔍

Below is a summary of compliance checks for this PR:

Security Compliance
🟢	No security concerns identified No security vulnerabilities detected by AI analysis. Human verification advised for critical code.
Ticket Compliance
⚪	🎫 No ticket provided Create ticket/issue
Codebase Duplication Compliance
⚪	Codebase context is not defined Follow the guide to enable codebase context checks.
Custom Compliance
🟢	Generic: Comprehensive Audit Trails Objective: To create a detailed and reliable record of critical system actions for security analysis and compliance. Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Meaningful Naming and Self-Documenting Code Objective: Ensure all identifiers clearly express their purpose and intent, making code self-documenting Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Robust Error Handling and Edge Case Management Objective: Ensure comprehensive error handling that provides meaningful context and graceful degradation Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Secure Error Handling Objective: To prevent the leakage of sensitive system information through error messages while providing sufficient detail for internal debugging. Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Secure Logging Practices Objective: To ensure logs are useful for debugging and auditing without exposing sensitive information like PII, PHI, or cardholder data. Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Security-First Input Validation and Data Handling Objective: Ensure all data inputs are validated, sanitized, and handled securely to prevent vulnerabilities Status: Passed Learn more about managing compliance generic rules or creating your own custom rules

Compliance status legend

🟢 - Fully Compliant
🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label

[qodo-code-review]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
General	Avoid unnecessary lock acquisition Add a check for self._model is None before acquiring the lock to avoid unnecessary lock acquisition when the model is already unloaded. src/chirp/parakeet_manager.py [67-76] def _unload_model(self) -> None: + # Quick check to avoid acquiring the lock if the model is already unloaded. + if self._model is None: + return + should_collect = False with self._lock: + # Re-check under lock to be thread-safe if self._model is not None and (time.time() - self._last_access > self._timeout): self._logger.info("Unloading Parakeet model to free memory.") self._model = None should_collect = True if should_collect: # Perform GC outside the lock to avoid blocking other threads (e.g., transcribe requests) gc.collect() Apply / Chat Suggestion importance[1-10]: 6 __ Why: The suggestion correctly proposes a double-checked locking pattern to avoid acquiring the lock unnecessarily, which is a valid performance optimization that aligns with the PR's goal of reducing lock contention.	Low
	Log outside lock Move the self._logger.info() call outside of the with self._lock: block to further reduce the time the lock is held. src/chirp/parakeet_manager.py [68-76] -should_collect = False +should_unload = False with self._lock: if self._model is not None and (time.time() - self._last_access > self._timeout): - self._logger.info("Unloading Parakeet model to free memory.") self._model = None - should_collect = True -if should_collect: - # Perform GC outside the lock to avoid blocking other threads (e.g., transcribe requests) + should_unload = True +if should_unload: + self._logger.info("Unloading Parakeet model to free memory.") gc.collect() Apply / Chat Suggestion importance[1-10]: 5 __ Why: This is a good suggestion that further reduces the lock hold time by moving the logging call outside the critical section, which is consistent with the PR's intent to minimize thread blocking.	Low
More