You leave grandma's house with two cakes. Each time you cross a bridge, you give the troll one cake, and then he gives you back a number of cakes equal to how many you have left.
It's easy to see that every time you leave a bridge, and thus when you return home, you still have two cakes.