Storage and Parallel Processing of Image Data Gathered for the Analysis of Social Structures in a Bee Colony
The Beesbook project's goal is to gather information of unprecedented detail about the processes in a beehive. During the planned experiments, due to a high temporal and image resolution, a big amount of image data will be acquired (up to 300 Terabyte). This thesis, firstly, presents an approach to live transfer the recorded image data to a specialized data storage facility in a stable and efficient way. For that purpose, it has been elaborated how to use a previously established Gigabit connection to capacity. Secondly, due to the image analysis taking an estimated time of 700 years on one processor, a parallelization of the anaylsis has been developed. The presented approach uses the newly built supercomputer's batch queue and submits processing jobs continuously. The resulting program is applicable to all perfectly parallel problems. In fact, it can be compared to the Map-step of the well-known MapReduce programming model, but in a supercomputer environment instead of a compute cluster.