…
comment: The repository administrator has not yet configured an RDF license.
primary topic: Parallel Asynchronous Matrix Multiplication for a Distributed Pipelined Neural Network

Parallel Asynchronous Matrix Multiplication for a Distributed Pipelined Neural Network
type: Article
abstract: Machine learning is an approach to devise algorithms that compute an output without a given rule set but based on a self-learning concept. This approach is of great importance for several fields of applications in science and industry where traditional programming methods are not sufficient. In neural networks, a popular subclass of machine learning algorithms, commonly previous experience is used to train the network and produce good outputs for newly introduced inputs. By increasing the size of the network more complex problems can be solved which again rely on a huge amount of training data. Increasing the complexity also leads to higher computational demand and storage requirements and to the need for parallelization. Several parallelization approaches of neural networks have already been considered. Most approaches use special purpose hardware whilst other work focuses on using standard hardware. Often these approaches target the problem by parallelizing the training data. In this work a new parallelization method named poadSGD is proposed for the parallelization of fully-connected, largescale feedforward networks on a compute cluster with standard hardware. poadSGD is based on the stochastic gradient descent algorithm. A block-wise distribution of the network's layers to groups of processes and a pipelining scheme for batches of the training samples are used. The network is updated asynchronously without interrupting ongoing computations of subsequent batches. For this task a one-sided communication scheme is used. A main algorithmic part of the batch-wise pipelined version consists of matrix multiplications which occur for a special distributed setup, where each matrix is held by a different process group. GASPI, a parallel programming model from the field of "Partitioned Global Address Spaces" (PGAS) models is introduced and compared to other models from this class. As it mainly relies on one-sided and asynchronous communication it is a perfect candidate for…
date: 2017
author list: authors
type: EPrint
type: DoctoralThesisEPrint
is part of: repository
has document: Parallel Asynchronous Matrix Multiplication for a Distributed Pipelined Neural Network (PDF)
has document: Parallel Asynchronous Matrix Multiplication for a Distributed Pipelined Neural Network (Other)
subject: 004 Informatik
subject: 510 Mathematik

Parallel Asynchronous Matrix Multiplication for a Distributed Pipelined Neural Network (PDF)
type: Document
has file: 2017-11-20_thesis-Schmidtobreick_publication.pdf
has part: 2017-11-20_thesis-Schmidtobreick_publication.pdf

Anke Mareike Schmidtobreick
type: Person
given name: Anke Mareike
family name: Schmidtobreick

Parallel Asynchronous Matrix Multiplication for a Distributed Pipelined Neural Network (Other)
type: Document
is version of: Parallel Asynchronous Matrix Multiplication for a Distributed Pipelined Neural Network (PDF)
is volatile version of: Parallel Asynchronous Matrix Multiplication for a Distributed Pipelined Neural Network (PDF)
islightbox thumbnail version of: Parallel Asynchronous Matrix Multiplication for a Distributed Pipelined Neural Network (PDF)
has file: lightbox.jpg
has part: lightbox.jpg

Parallel Asynchronous Matrix Multiplication for a Distributed Pipelined Neural Network (Other)
type: Document
is version of: Parallel Asynchronous Matrix Multiplication for a Distributed Pipelined Neural Network (PDF)
is volatile version of: Parallel Asynchronous Matrix Multiplication for a Distributed Pipelined Neural Network (PDF)
ispreview thumbnail version of: Parallel Asynchronous Matrix Multiplication for a Distributed Pipelined Neural Network (PDF)
has file: preview.jpg
has part: preview.jpg

Parallel Asynchronous Matrix Multiplication for a Distributed Pipelined Neural Network (Other)
type: Document
is version of: Parallel Asynchronous Matrix Multiplication for a Distributed Pipelined Neural Network (PDF)
is volatile version of: Parallel Asynchronous Matrix Multiplication for a Distributed Pipelined Neural Network (PDF)
ismedium thumbnail version of: Parallel Asynchronous Matrix Multiplication for a Distributed Pipelined Neural Network (PDF)
has file: medium.jpg
has part: medium.jpg

Parallel Asynchronous Matrix Multiplication for a Distributed Pipelined Neural Network (Other)
type: Document
is version of: Parallel Asynchronous Matrix Multiplication for a Distributed Pipelined Neural Network (PDF)
is volatile version of: Parallel Asynchronous Matrix Multiplication for a Distributed Pipelined Neural Network (PDF)
issmall thumbnail version of: Parallel Asynchronous Matrix Multiplication for a Distributed Pipelined Neural Network (PDF)
has file: small.jpg
has part: small.jpg

Parallel Asynchronous Matrix Multiplication for a Distributed Pipelined Neural Network (Other)
type: Document
is version of: Parallel Asynchronous Matrix Multiplication for a Distributed Pipelined Neural Network (PDF)
is volatile version of: Parallel Asynchronous Matrix Multiplication for a Distributed Pipelined Neural Network (PDF)
is index codes version of: Parallel Asynchronous Matrix Multiplication for a Distributed Pipelined Neural Network (PDF)
has file: indexcodes.txt
has part: indexcodes.txt

HTML Summary of #23737 Parallel Asynchronous Matrix Multiplication for a Distributed Pipelined Neural Network
format: text/html
primary topic: Parallel Asynchronous Matrix Multiplication for a Distributed Pipelined Neural Network

004 Informatik
type: Concept
preferred label: 004 Data processing Computer science @en

510 Mathematik
type: Concept
preferred label: 510 Mathematics @en