Observation of the food exchange behaviour of honey bees, trophallaxis, has been laborious in the past. In this work  I created a classifier to detect trophallaxis as a part of the BeesBook system that aims to automate detection of bee behavior in general. Based on labeled data I created a dataset of images showing trophallaxis and trained a convolutional neural network to classify images. I show that using more than one frame to classify trophallaxis yields a better score than using a single image. The network reaches an F_1 score of 0.89 for detecting trophallaxis which is an improvement over existing methods.