fix: Respect max_open_pages_per_browser limit for PlaywrightBrowserController on concurrent new_page calls#1712
Conversation
There was a problem hiding this comment.
Pull request overview
This pull request attempts to fix a concurrency issue in PlaywrightBrowserController where concurrent calls to new_page() could create more pages than allowed by the max_open_pages_per_browser limit. The fix introduces a counter to track pages currently being opened and includes this in the capacity check.
Changes:
- Added
_opening_pages_countcounter to track concurrent page creation operations - Modified
has_free_capacityproperty to include pages being opened in the capacity calculation - Restructured
new_page()method with try-finally block to properly manage the opening counter - Added test
test_max_open_pages_limit_on_concurrent_creationto verify concurrent creation respects the limit
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| src/crawlee/browsers/_playwright_browser_controller.py | Introduces _opening_pages_count counter and modifies capacity checking logic with try-finally protection |
| tests/unit/browsers/test_playwright_browser_controller.py | Adds test for concurrent page creation with max limit of 1 |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
c7012ee to
e19fcc9
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Description
max_open_pages_per_browserlimit forPlaywrightBrowserControlleron concurrentnew_pagecalls.Testing
test_max_open_pages_limit_error_on_concurrent_creationto verify that concurrent calls do not create several pages exceeding themax_open_pages_per_browserlimit.