Beginners Guide to Federated Learning.

What is Federated Learning?

Federated learning is a machine learning technique that collaboratively trains an algorithm across multiple decentralized edge devices (desktops, mobiles, tablets, IOT devices, etc.) or servers (data silos) holding local data samples, without the need for sharing the data or centrally aggregating them.

This approach is in contrast to classical Centralized ML techniques where all the datasets residing on edge devices are uploaded to one central server. Federated Learning also differs from Distributed ML, where the expectation is that the local datasets will be identically distributed with more or less constant availability of the nodes.

Some of the popular applications that use federated learning are Siri and GBoard.

Steps in federated learning

Now lets look into the steps involved in federated learning in detail:

  1. A model instance is running in each of the device.
  2. Once the training is complete on the local data, the device connects to the server and the model weights/parameters are sent to the central server.
  3. The central server aggregates the weight parameters and sends the updated weights back to each device. (Thus ensuring that data never leaves the device and we get the benefit of training on the data from all the devices.)
  4. Once the devices receive the updated parameters, they can now use the updated model parameters for their individual tasks.

Some of the popular federated Learning frameworks available which you can use to build your own federated learning framework are -

PySyft by OpenMined

Limitations of Federated Learning

Federated Learning enables us to keep the data at source during model training, thereby resolving concerns of sharing private data to a great extent and preserving privacy at the outset. In essence, the Federated Averaging algorithm involves the following steps,

There have been many advances in the area of federated learning, but there are some challenges that need to be addressed.

Work Hard. Be Kind. Take Pride.