Non Blocking Background Syncer

Note: This is part two (of two) of creating an approach for a background synchronisation task. Part one can be found here

To avoid the procedure of electing a master client amongst all online clients, another approach for syncing the client has to be developed.

Processing File Requests while Syncing

Since the synchronisation process will happen completely independent of other clients, we have to guarantee that no file change event from another client is lost during the sync.

• Incoming CreateEvent: Accept fileOfferRequests and FilePushRequests. Changes are merged into the object store after restarting the event aggregator and propagated to all other clients.

• Incoming ModifyEvent: Accept fileOfferRequests and FilePushRequests. Changes are merged into the object store after restarting the event aggregator and propagated to all other clients.

• Incoming MoveEvent:

  - An OfferRequest is received for a move (from client2)
  - Since the target does not exist, it gets accepted (on client1)
  - Therefore, the file is moved (on client1)
  - Since the event aggregator is not running, no object store update is made (on client1)
  - Concurrently, the merging of the object store does not have the move in its state (client2 
    will have updated his object store only after sending his object store to client1)
  - After merging is complete (client1), the event aggregator is started again and its local disk state is 
    compared. There, since the move is already made, a difference occurs between the merged and the actual 
    object store: There are new files and but also new deleted ones. Since we can not distinguish
    whether a modification is made on the client currently in merge due to the end 
    user or due to a modification request, the create resp. delete event will be propagated to
    all other clients again.
  

• Incoming DeleteEvent:

  - An OfferRequest is received for a delete (from client2)
  - Since the file does exist, it gets accepted (on client1)
  - Therefore, the file is deleted (on client1)
  - Since the event aggregator is not running, no object store update is made (on client1)
  - Concurrently, the merging of the object store does not contain the delete event in its state (on client1)
  - After merging is complete (client1), the event aggregator will be started again and its local disk state 
    is compared. There, an additional file deletion will be detected. The file is removed. Since we can not
    distinguish whether a modification is made on the client currently in merge due to the end user or due 
    to a modification request, the delete event will be propagated to all other clients again.
  

Due to the propagation to all clients, a modification to the file offering has to be made: If a file with the same version does already exist on the client which received the offering, he may deny the offer request using a dedicated flag. The following up requests should then made only to the clients which have not denied this flag.

Step by Step Procedure

1. Stop Event Aggregator
2. Fetch Object Stores from all other online clients
3. Merge Object Stores
4. Delete removed paths resp. fetch missing ones
5. Start Event Aggregator
6. Compare disk with merged Object Store

1. Stop Event Aggregator

To avoid generating events while modifying the local disk due to changes in the merged object store, the event aggregator is stopped

2. Fetch Object Stores

After stopping, the client will fetch the object stores from all other clients.

3. Merge Object Stores

Once, all object stores have been received, the client will merge them.

Side Note: If the event aggregator would still be running here, merging the object store would cause the path event listener to work despite that these changes are only made in the object store's folder. Since such events are ignored anyway, we just save here some work.

4. Delete Removed Paths resp. Fetch Missing Ones

Apply the changes from the merged object store to the storage adapter (e.g. the local disk).

• Version Conflict on Fetch: It could happen, that a version conflict arises, while fetching changes from other clients:

           Client 1              Client 2
              |                     |
              a                     a
              |                     |
              b                     b
              |                     |
              c1        -X-         c2
  
  

  In such a case, the merging client has to create a conflict file

Side Note: If the event aggregator would still be running here, we would cause event modifying the merged object store (e.g. trying to remove the object for an already deleted path)

5. Start Event Aggregator

Once, we have made all changes to the disk, we can safely restart the event aggregator.

6. Compare disk with merged Object Store

After restarting the event aggregator, we have to check, that no events are lost during the time between stopping and restarting the event aggregator. To make this happen, we will finally compare the local state of the disk and the state stored in the merged object store. For any file, which has a different hash or was removed, we propagate the event to all other clients.