Pysyft_API_Tutorial.ipynb

Kernel: Python 3

Basic API details

Install pysyft using pip

!pip install syft

Now we import PyTorch and PySyft, however we hook torch with syft with TorchHook function. The TorchHook does the wrapping by adding all the additional functionality to PyTorch for doing Federated Learning and other Private AI techniques.

import torch
import syft as sy
hook = sy.TorchHook(torch) # add extra functionality to PyTorch

First, let's create a virtual machine owned by some health clinic, say harmony_clinic. We will be using this to simulate a machine present at a remote location. A thing to note is that syft calls these machines as VirtualWorker.

# create a machine owned by harmony clinic
harmony_clinic = sy.VirtualWorker(hook=hook,id='clinic')

harmony_clinic is at a remote location but it doesn't have any data we can use. Let's create some data so that we can send it to harmony_clinic

# we create a Tensor, maybe this is some gene sequence
dna = torch.tensor([0,1,2,1,2])

# and now I send it, and in turn we get a pointer back that
# points to that Tensor
dna_ptr = dna.send(harmony_clinic)

print(dna_ptr)

(Wrapper)>[PointerTensor | me:37078163247 -> clinic:73951623209]

We see that the PointerTensor points from me (which is us, PySyft creates this me worker automatically) to harmony_clinic. We also see some random numbers, these are actually object IDs that PySyft assigns to every object.

Notice the object ID for the tensor dna. It is same as above in dna_ptr.

Now harmony_clinic has the tensor that we sent. We can use harmony_clinic._objects to see objects that harmony_cliniccurrently has.

print(harmony_clinic._objects)

{73951623209: tensor([0, 1, 2, 1, 2])}

And in the same way, we can get a tensor back from a remote location by using the .get() function.

# get back dna
dna = dna_ptr.get()
print(dna)

# And as you can see... clinic no longer has the tensor dna anymore!!! It has moved back to our machine!
print(harmony_clinic._objects)

tensor([0, 1, 2, 1, 2])
{}

Deep Learning with Pointers

a = torch.tensor([3.14, 6.28]).send(harmony_clinic)
b = torch.tensor([6.14, 3.28]).send(harmony_clinic)
c = a + b
print(c)

(Wrapper)>[PointerTensor | me:36395876027 -> clinic:33335259535]

Something very interesting happened behind the scenes, i.e. when did c = a + b on our machine, a command was sent to the remote machine that did that exact calculation, created a new tensor on its machine and then sent back a pointer to us which we now call c.

The amazing thing is this API has been extended to all the PyTorch operations including Back propogation. Hurray !!

This means that we can use the same PyTorch code that we usually do when doing Machine Learning.

# we create two tensors and send it to bob
train = torch.tensor([2.4, 6.2], requires_grad=True).send(harmony_clinic)
label = torch.tensor([2, 6.]).send(harmony_clinic)

# we apply some function, in this case a rather simple one, just to show the idea, we use L1 loss
loss = (train-label).abs().sum()

# Yes, even .backward() works when working with Pointers
loss.backward()

# now we retreive back the train tensor
train = train.get()

print(train)

# If everything went well, we will see gradients accumulated 
# in .grad attribute of train
print(train.grad)

tensor([2.4000, 6.2000], requires_grad=True)
tensor([1., 1.])