NS-661 Refactor usage of temporary agent networks.#742
NS-661 Refactor usage of temporary agent networks.#742andreidenissov-cog wants to merge 7 commits intomainfrom
Conversation
andreidenissov-cog
force-pushed
the
ASD-NS-661-temp-networks01
branch
from
f809c2d to
a383098
Compare
| :return: Whether the reservation is expired or not. | ||
| """ | ||
| return time.time() >= self.expiration_time_in_seconds | ||
|
|
There was a problem hiding this comment.
Convenience function for simpler checks.
|
|
||
|
|
||
| class FixedAgentNetworkProvider(AgentNetworkProvider): | ||
| """ |
There was a problem hiding this comment.
AgentNetworkProvider which returns immutable instance of temp network.
We don't want extra redirection here which would allow dynamic replacement of agent network version.
| "public": AgentNetworkStorage(), | ||
| "temp": ExpiringAgentNetworkStorage() | ||
| "temp": ExpiringCachingAgentNetworkStorage() | ||
| } |
There was a problem hiding this comment.
Type for our temp networks storage class now changes.
| TempNetworkStorageUpdater(server_context.get_network_storage_dict(), watcher_config, | ||
| TempNetworkStorageUpdater(server_context.get_network_storage_dict(), | ||
| server_context.get_queues()) | ||
| ] |
There was a problem hiding this comment.
No "watcher_config" in parameters list - our TempNetworkStorageUpdater is not really a "watcher".
andreidenissov-cog
force-pushed
the
ASD-NS-661-temp-networks01
branch
from
53523c6 to
5b8d296
Compare
| supporting addition Reservations in bulk and retrieval of individual Reservations, | ||
| as well as expiration of Reservations based on their lifetime. | ||
| """ | ||
|
|
There was a problem hiding this comment.
New interface for classes storing Reservations, with methods to add and extract reservations,
set them to expired state AND set another ExpiringReservationsStorage instance
to serve as a persistent backup, so our instance will work as a front cache for Reservations requests.
| # Another component to start is the temporary networks updater | ||
| temp_networks_updater: TempNetworkStorageUpdater =\ | ||
| TempNetworkStorageUpdater(self.server_context.get_network_storage_dict(), self.server_context.get_queues()) | ||
| components_to_start.append(temp_networks_updater) |
There was a problem hiding this comment.
TempNetworkStorageUpdater is not a "watcher" anymore, so create it separately
and add to the list of components to be started after our HTTP server starts running.
| :param base_storage: An ExpiringReservationsStorage instance to use as a source of truth | ||
| """ | ||
| raise NotImplementedError | ||
|
|
There was a problem hiding this comment.
For S3 based reservations storage, we don't have a backup place.
| stored in its associated agent spec as metadata. | ||
| """ | ||
|
|
||
| def __init__(self, bucket_name: str = "", prefix: str = "reservations/", |
There was a problem hiding this comment.
Implementation especially for S3 operations is mostly copied from S3ReservationsStorage class.
Difference is that we are running our own thread (owned by AbstractExpiringReservationsStorage)
which actively polls for expired Reservation objects.
| raise ValueError(f"Access denied to S3 bucket '{self.bucket_name}'") from exception | ||
| raise ValueError(f"Error accessing S3 bucket '{self.bucket_name}': {exception}") from exception | ||
| # We are good with S3 connection and bucket access at this point, | ||
| # let's start underlying logic: |
There was a problem hiding this comment.
Start AbstractExpiringReservationsStorage thread.
1 participant
No description provided.