Cesga

..

Título: Integrating Neural Network Parallel Training using Tensorflow with SLURM

Tipo do ficheiro: pdf
Data da subida: 26/09/2018
Autor / procedencia: Gonzalo Ferro Costas
Copyright: CESGA

Descrición: One of the most powerful characteristics of Google API for Machine Learning TensorFlow is its capability for distributed computation that allow users to automatically distribute the training process in different computing machines. Despite the fact that the implementation of these characteristics is relatively straightforward, their deployment in a typical High Performance Infrastructure based on queue management systems presents several issues. This report describes a complete python package that solves these issues in the Finis Terrae II supercomputer, which uses Slurm as queue system. In order to show the performance of the distributed TensorFlow in Finis Terrae II, an industrial case based on the experiment 707 (CyPLAM) from FORTISSIMO II was trained using the developed python package

Tags: Machine Learning,TensorFlow,supercomputer,HPC

Título: Evaluation of Machine Learning Fameworks on Finis Terrae II

Tipo do ficheiro: pdf
Data da subida: 27/12/2017
Autor / procedencia: Ándres Gómez Tato
Copyright: CESGA

Descrición: Machine Learning (ML) and Deep Learning (DL) are two technologies used to extract representations of the data for a specific purpose. ML algorithms take a set of data as input to generate one or several predictions. To define the final version of one model, usually there is an initial step devoted to train the algorithm (get the right final values of the parameters of the model). There are several techniques, from supervised learning to reinforcement learning, which have different requirements. On the market, there are some frameworks or APIs that reduce the effort for designing a new ML model. In this report, using the benchmark DLBENCH, we will analyse the performance and the execution modes of some well-known ML frameworks on the Finis Terrae II supercomputer when supervised learning is used. The report will show that placement of data and allocated hardware can have a large influence on the final timeto-solution.

Tags: DLBENCH, MachineLearning, computing, cesga

Título: Overview of different profiling tools

Tipo do ficheiro: pdf
Data da subida: 19/12/2017
Autor / procedencia: Diego Mairena Díaz and José Carlos Mouriño
Copyright: CESGA

Descrición: This technical report will try to give an overview of different profiling tools, both open-source and commercial. Some of them are currently available in FinisTerraeII supercomputer, so a brief usage of this tools will be shown too. This report will give a point of reference to start to use this tools, and at the same time, a little information about when it is recommended the use of each one, languages and parallel standards supported and the different results we can get using them.

Tags: computing, parallel tools, OpenMO, HPC, cesga

Título: Evaluation of the CESGA Big Data Infrastructure

Tipo do ficheiro: pdf
Data da subida: 03/10/2017
Autor / procedencia: José Carlos Mouriño
Copyright: CESGA

Descrición: The purpose of this Technical Report is analyze, debug & benchmark the BigData platform installed at CESGA by running Spark-based ML algorithms over public datasets with different resource allocation (number of executors, number of cores, memory per executor, memory for driver, etc.).

Tags: Big Data, HPC, cesga, supercomputing, innovation

Título: Hyper-parametric search using HPC infrastructures for Tensorflow

Tipo do ficheiro: pdf
Data da subida: 07/09/2017
Autor / procedencia: Gonzalo Ferro, Carmen Cotelo
Copyright: CESGA

Descrición: This internal technical document reports the hyper-parametric search for the proposed Neural Network model using Tensorflow and Finis Terrae II. The NN model helps to manage the process of Laser Metal Deposition based on the thermal images from a thermal camera attached to the laser header that record a complete process. This case is an example of using HPC infrastructures for designing Neural Networks models and finding the best parameters for training them.

Tags: HPC , CESGA, Tensorflow, parametric

Título: Hyper-parametric search using HPC infrastructures for Tensorflow.

Tipo do ficheiro: pdf
Data da subida: 07/09/2017
Autor / procedencia: Gonzalo Ferro, Carmen Cotelo
Copyright: CESGA

Descrición: This internal technical document reports the hyper-parametric search for the proposed Neural Network model using Tensorflow and Finis Terrae II. The NN model helps to manage the process of Laser Metal Deposition based on the thermal images from a thermal camera attached to the laser header that record a complete process. This case is an example of using HPC infrastructures for designing Neural Networks models and finding the best parameters for training them.

Tags: Tensorflow, cesga, supercomputing, HPC, parametric

Título: Evaluation of Lustre subsytem

Tipo do ficheiro: pdf
Data da subida: 18/01/2017
Autor / procedencia: Diego Mairena Díaz and Aurelio Rodríguez López
Copyright: CESGA

Descrición: Abstract: This technical report shows the behaviour of Lustre storage subsystem for different applications. Lustre is a scalable file subsystem and let the final user to control the way her files are stored. CESGA has 694,5TB usable in Finis Terrae II supercomputer, and the Infiniband network achieves better results than DAS storage. This report is intended to be a study of performance with different storage subsystems in a real supercomputer with real typical applications. The file size, the number of OSTs used or the pieces in which file is divided are a few of the parameters used to measure the performance of Lustre and try to obtain the best parameters to use efficiently this storage subsystem.

Tags: Lustre, supercomputer, cesga, storage

Título: Estudio del rendimiento de diferentes herramientas MPI en el sistema FinisTerrae II

Tipo do ficheiro: pdf
Data da subida: 27/12/2016
Autor / procedencia: Ignacio Santos Díaz Sandra González Rodríguez Jose Carlos Mouriño Gallego
Copyright: CESGA

Descrición: Resumen: Este informe recoge los principales resultados extraídos del estudio del rendimiento de diferentes herramientas de paralelización en el superordenador Finis Terrae II instalado en el CESGA. Se incluyen resultados tanto de los rendimientos de comunicaciones punto a punto como de comunicaciones colectivas, en las que se analiza como afecta el aumento del número de procesos por nodo a las eficiencias de las diferentes herramientas MPI estudiadas.

Tags: openMPI, MPI, computation, cesga

Título: Informe despliegue piloto BBDD en clúster

Tipo do ficheiro: pdf
Data da subida: 24/08/2016
Autor / procedencia: David Fernández García, Natalia Costas Lago
Copyright: CESGA

Descrición: En el CESGA gran parte de servicios hacen uso de BBDD de tipo MySQL. Se pretende la mejora de la disponibilidad de dichos servicios analizando soluciones de clúster

Tags: MySQL, MariaDB, cluster, cesga

Título: Requisitos Hardware Big Data, Technical Report.

Tipo do ficheiro: pdf
Data da subida: 11/03/2015
Autor / procedencia: Javier Cacheiro
Copyright: CESGA

Descrición: Este documento ofrece un análisis de los requisitos hardware generales para soluciones Big Data como por ejemplo Apache Hadoop, Apache Spark o Apache Hbase.

Tags: Big Data, apache, Hadoop, computing, HBase, Spark

Título: Technical Report CESGA-2014-003. Networked Music Performance

Tipo do ficheiro: pdf
Data da subida: 09/06/2014
Autor / procedencia: Ivelina Atanasova Karagyulieva and Natalia Costas Lago
Copyright: CESGA

Descrición: This technical report analyzes the state of the art of technology for musical real-time interaction over computer/data networks (NMP, Networked Music Performances). It describes the challenges that it introduces and the software implementations that make NMP possible at present. It also presents a practical experience based on NMP where a group of musicians, located at geographical distant places, interact over a network to perform as they would if they were located in the same room.

Tags: Networked Music Performance, Latency, High performance data network applications, RECETGA (Galicia Science and Technology Network)

Título: OpenFlow and SDN Technical Report

Tipo do ficheiro: pdf
Data da subida: 14/02/2014
Autor / procedencia: Author: Ivelina Atanasova Karagyulieva Revision: Natalia Costas Lago
Copyright: CESGA

Descrición: This technical report introduces a review of the new network technology OpenFlow and its role within the Software Define Networking scenario. The report includes a summary of the OpenFlow standard, a comparison of the different OpenFlow standard versions, some relevant OpenFlow use cases that are of interest to datacenters and also information about the support that is provided in different platforms (both hardware and software). Also we reviewed in a wider scope the SDN scenario and some projects that use OpenFlow and SDN as a basis. Finally, at the end of the document it is shown a description of the Openflow testbed deployed at CESGA and the results of the different tests performed.

Tags: OpenFlow, Software Defined Netrorks, technical report, network technology assessment, CESGA

Título: Analysis of Batch Systems SGE, SLURM, and TORQUE/Maui

Tipo do ficheiro: pdf
Data da subida: 14/02/2014
Autor / procedencia: Javier Cacheiro
Copyright: CESGA

Descrición: This document is intended to provide a comparison of three very popular batch systems (SGE, SLURM and Torque/Maui) to manage the allocation of compute cluster resources.

Tags: batch, SGE, SLURM, Torque/Maui, supercomputing, cesga

Título: GAIA Running Big Data at CESGA

Tipo do ficheiro: pdf
Data da subida: 19/12/2013
Autor / procedencia: Javier Cacheiro
Copyright: CESGA

Descrición: This document is intended to provide an overview of the technical aspects related to running GAIA Big Data calculations at CESGA's new Hadoop on-demand platform.

Tags: big data, gaia, hadoop, supercomputing

Título: Hadoop y MapReduce

Tipo do ficheiro: pdf
Data da subida: 19/12/2013
Autor / procedencia: Diego Nieto
Copyright: CESGA

Descrición: Apache Hadoop es un framework basado en JAVA que soporta aplicaciones distribuidas. Permite a las aplicaciones trabajar con miles de nodos y petabytes de datos. Hadoop se inspiró en los documentos Google para MapReduce y Google File System (GFS). Hadoop es un proyecto de alto nivel Apache y con una gran comunidad base.

Tags: Hadoop, MapReduce, JAVA, CESGA, Supercomputing

Título: Technical Report CESGA-2013-002: Evaluation of Intel Xeon Phi to execute easily scientific applications

Tipo do ficheiro: pdf
Data da subida: 30/07/2013
Autor / procedencia: José Carlos Mouriño Gallego; Carmen Cotelo Queijo;Andrés Gómez Tato; Aurelio Rodríguez López
Copyright: CESGA 2013

Descrición: This technical report presents CESGA experience of porting three applications to the new Intel® Xeon Phi™ coprocessor. The objective of these experiments was to evaluate the complexity of such a work, to check the new capabilities of this new technology, and to know its limitations for executing scientific and technical software. The results show that the migration of these applications to Xeon Phi can be done easily, but getting real improved performance demands a more deeply refactoring.

Tags: Xeon Phi;Parallel; OpenMP;BLAS;MPI;ROMS;Evaluation;

Título: Despliegue de servicios HPC en entornos cloud multipropósito

Tipo do ficheiro: pdf
Data da subida: 03/01/2013
Autor / procedencia: David Rodríguez Penas

Descrición: Despliegue de servicios HPC en entornos cloud multipropósito. Autor : David Rodríguez Penas Director : Natalia Costas Lago Director : Ignacio López Cabido Tutor : Tomás Fernández Pena

Tags: cloud, cesga, hpc

Título: Análisis de tecnologías de virtualización de redes y su aplicación en NRENs

Tipo do ficheiro: pdf
Data da subida: 03/01/2013
Autor / procedencia: David Rodríguez Penas, Natalia Costas Lago

Descrición: El proposito del presente documento es hacer un breve análisis sobre diferentes tecnologías utilizadas por las NREN (Redes Nacionales de Investigación y Educación) para la creación de redes bajo demanda.

Tags: virtualización de redes, circuitos bajo demanda, multidominio

Título: Monitorización de redes y servicios en RECETGA

Tipo do ficheiro: pdf
Data da subida: 14/12/2012
Autor / procedencia: Frco. Javier Rial Rodríguez - CESGA

Descrición: Caso práctico de implementación de Zabbix como servidor de monitorización de red en RECETGA.

Título: MANUAL PARA LA CARACTERIZACIÓN MONTE CARLO DE UN ACELERADOR LINEAL DE RADIOTERAPIA PARA LA PLATAFORMA e-IMRT

Tipo do ficheiro: pdf
Data da subida: 27/01/2012
Autor / procedencia: Zahara MArtín Rodríguez y Araceli Gago Arias
Copyright: Cesga y USC

Descrición: InformeTecnicoComisionado.pdf

Tags: eIMRT, Radioterapia Monte Carlos, Comisionado, Caracterización

Título: Data assimilation with the ROMS model: A ﬁrst approach.

Tipo do ficheiro: pdf
Data da subida: 27/12/2011
Autor / procedencia: Marcos Cobas García, Andrés Gómez Tato and Carmen Cotelo Queijo.

Descrición: This technical report is developed within the framework of the "Oceano" project (INCITE - 09MDS009CT). Part of the objectives of the project are the operational execution of a regional conﬁguration of the ROMS model (http://www.myroms.org), including data assimilation.

Tags: Data assimilation, ROMS, ocean model

Título: Comparativa de software de gestión cloud