Redis Support for FedScale

[xuyehe]

Why are these changes needed?

To integrate Redis into aggregator for saving aggregation data.

Related issue number

N/A

Checks

• I've included any doc changes needed for https://fedscale.readthedocs.io/en/latest/
• I've made sure the following tests are passing.
• Testing Configurations
  • Dry Run (20 training rounds & 1 evaluation round)
  • Cifar 10 (20 training rounds & 1 evaluation round)
  • Femnist (20 training rounds & 1 evaluation round)

Note:

1. All tests are run on cpu.
2. Met with the following KeyError bug on the same line in Femnist once with Redis, and also once without Redis (i.e. original code). Sample error output:

File "/users/xuyehe/FedScale-rd/fedscale/core/aggregation/aggregator.py", line 386, in client_completion_handler
    duration=self.virtual_client_clock[results['clientId']]['computation'] +
KeyError: 665

[fanlai0990]

Can we remove the hardcoded path? Instead, use some like os.environ["REDIS_PATH"]

[fanlai0990]

I remember you mentioned that some tf version is not compatible. Can you please update tf version here too?

[fanlai0990]

Please remove any hard code in the future, using args.xxx instead.

[fanlai0990]

Thanks a lot for your contribution! Let's try to run an e2e evaluation to validate the learning curve and Redis overhead first.

[fanlai0990]

Thanks for your contribution. Can you please

• remove all prints or replace them with logging.info
• summarize the results of the overhead analysis

We really appreciate your help.