The purpose of this Technical Report is analyze, debug & benchmark the BigData platform installed at CESGA by running Spark-based ML algorithms over public datasets with different resource allocation (number of executors, number of cores, memory per executor, memory for driver, etc.).
This internal technical document reports the hyper-parametric search for the proposed Neural Network model using Tensorflow and Finis Terrae II. The NN model helps to manage the process of Laser Metal Deposition based on the thermal images from a thermal camera attached to the laser header that record a complete process. This case is an [...]
This technical report shows the behaviour of Lustre storage subsystem for different applications. Lustre is a scalable file subsystem and let the final user to control the way her files are stored. CESGA has 694,5TB usable in Finis Terrae II supercomputer, and the Infiniband network achieves better results than DAS storage. This report is intended [...]
Este informe recoge los principales resultados extraídos del estudio del rendimiento de diferentes herramientas de paralelización en el superordenador Finis Terrae II instalado en el CESGA. Se incluyen resultados tanto de los rendimientos de comunicaciones punto a punto como de comunicaciones colectivas, en las que se analiza como afecta el aumento del número de procesos [...]
En el CESGA gran parte de servicios hacen uso de BBDD de tipo MySQL. Se pretende la mejora de la disponibilidad de dichos servicios analizando soluciones de clúster.
Este documento ofrece un análisis de los requisitos hardware generales para soluciones Big Data como por ejemplo Apache Hadoop, Apache Spark o Apache Hbase.
This technical report analyzes the state of the art of technology for musical real-time interaction over computer/data networks (NMP, Networked Music Performances). It describes the challenges that it introduces and the software implementations that make NMP possible at present. It also presents a practical experience based on NMP where a group of musicians, located at [...]
This technical report introduces a review of the new network technology OpenFlow and its role within the Software Define Networking scenario. The report includes a summary of the OpenFlow standard, a comparison of the different OpenFlow standard versions, some relevant OpenFlow use cases that are of interest to datacenters and also information about the support [...]
This document is intended to provide a comparison of three very popular batch systems (SGE, SLURM and Torque/Maui) to manage the allocation of compute cluster resources.
This document is intended to provide an overview of the technical aspects related to running GAIA Big Data calculations at CESGA's new Hadoop on-demand platform.