Network theorists define patterns in complex networks in various ways to make them accessible to human beholders. Prominent definitions are thereby based on the partition of the network's nodes into groups such that underlying patterns in the link structure become apparent. Clustering and blockmodeling are two well-known approaches of this kind.
In this thesis, we treat pattern search problems as discrete mathematical optimization problems. From this viewpoint, we develop a new mathematical classification of clustering and blockmodeling approaches, which unifies these two fields and replaces several NP-hardness proofs by a single one. We furthermore use this classification to develop integer mathematical programming formulations for pattern search problems and discuss new linearization techniques for polynomial functions therein. We apply these results to a model for a new pattern search problem. Even though it is the most basic problem in combinatorial terms, we can prove its NP-hardness. In fact, we show that it is a generalization of well-known problems including the Traveling Salesman and the Quadratic Assignment Problem. Our derived exact pattern search procedure is up to 10,000 times faster than comparable methods from the literature. To demonstrate its practicability, we finally apply the procedure to the world trade network from the United Nations' database and show that the network deviates by less than 0.14% from the patterns we found.
The goal of this thesis is the development of a novel and efficient algorithm to determine the global optimum of an optimal control problem. In contrast to previous methods, the approach presented here is based on the direct multiple shooting method for discretizing the optimal control problem, which results in a significant increase of efficiency. To relax the discretized optimal control problems, the so-called alpha-branch-and-bound method in combination with validated integration is used.
For the direct comparison of the direct single-shooting-based relaxations with the direct multipleshooting-based algorithm, several theoretical results are proven that build the basis for the efficiency increase of the novel method. A specialized branching strategy takes care that the additionally introduced variables due to the multiple shooting approach do not increase the size of the resulting branch-and-bound tree. An adaptive scaling technique of the commonly used Gershgorin method to estimate the eigenvalues of interval matrices leads to optimal relaxations and therefore leads to a general improvement of the alpha-branch-and-bound relaxations in a single shooting and a multiple shooting framework, as well as for the corresponding relaxations of non-dynamic nonlinear problems. To further improve the computational time, suggestions regarding the necessary second-order interval sensitivities are presented in this thesis, as well as a heuristic that relaxes on a subspace only.
The novel algorithm, as well as the single-shooting-based alternative for a direct comparison, are implemented in a newly developed software package called GloOptCon. The new method is used to globally solve both a number of benchmark problems from the literature, and so far in the context of global optimal control still little considered applications. The additional problems pose new challenges either due to their size or due to having its origin in mixed integer optimal control with an integer-valued time-dependent control variable. The theoretically proven increase of efficiency is validated by the numerical results. Compared to the previous approach from the literature, the number of iterations for typical problems is more than halved, meanwhile the computation time is reduced by almost an order of magnitude. This in turn allows the global solution of significantly larger optimal control problems.
The aim of this thesis is the development of new concepts for environmental 3D reconstruction in automotive surround-view systems where information of the surroundings of a vehicle is displayed to a driver for assistance in parking and low-speed manouvering.
The proposed driving assistance system represents a multi-disciplinary challenge combining techniques from both computer vision and computer graphics. This work comprises all necessary steps, namely sensor setup and image acquisition up to 3D rendering in order to provide a comprehensive visualization for the driver.
Visual information is acquired by means of standard surround-view cameras with fish eye optics covering large fields of view around the ego vehicle. Stereo vision techniques are applied to these cameras in order to recover 3D information that is finally used as input for the image-based rendering. New camera setups are proposed that improve the 3D reconstruction around the whole vehicle, attending to different criteria. Prototypic realization was carried out that shows a qualitative measure of the results achieved and prove the feasibility of the proposed concept.
Thanks to revolutionary developments in microscopy techniques such as robotic high-throughput setups or light sheet microscopy, vast amounts of data can be acquired at unprecedented temporal and spatial resolution. The mass of data naturally prohibits manual analysis, though, and life scientists thus have to rely more and more on automated cell tracking methods. However, automated cell tracking involves intricacies that are not commonly found in traditional tracking applications. For instance, cells may undergo mitosis, which results in variable numbers of tracking targets across successive frames. These difficulties have been addressed by tracking-by-assignment models in the past, which dissect the task into two stages, detection and tracking. However, as with every two-stage framework, the approach hinges on the quality of the first stage, and errors propagate partially irrevocably from the detection to the tracking phase.
The research in this thesis thus focuses on methods to advance tracking-by-assignment models in order to avoid these errors by exploiting synergy effects between the two (previously) separate stages. We propose two approaches, both in terms of probabilistic graphical models, which allow for information exchange between the detection and the tracking step to different degrees. The first algorithm, termed Conservation tracking, models both possible over- and undersegmentation errors and implements global consistency constraints in order to reidentify target identities even across occlusion or erroneous detections. Wrong detections from the first step can hence be corrected in the second stage. The second method goes one step further and optimizes the two stages completely jointly in one holistic model. In this way, the detection and tracking step can maximally benefit from each other and reach the overall most likely interpretation of the data. Both algorithms yield notable results which are state-of-the-art.
In spite of the distinguished results achieved with these methods, automated cell tracking methods are still error-prone and manual proof-reading is often unavoidable for life scientists. To avoid the time-consuming manual identification of errors on very large datasets, most ambiguous predictions ought to be detected automatically so that these can be corrected by a human expert with minimal effort. In response, we propose two easy-to-use methods to sample multiple solutions from a tracking-by-assignment graphical model and derive uncertainty measures from the variations across the samples. We showcase the usefulness for guided proof-reading on the cell tracking model proposed in this work.
Finally, the successful application of structured output learning algorithms to cell tracking in previous work inspired us to advance the state-of-the-art by an algorithm called Coulomb Structured Support Vector Machine (CSSVM). The CSSVM improves the expected generalization error for unseen test data by the training of multiple concurrent graphical models. Through the novel diversity encouraging term, motivated from experimental design, the ensemble of graphical models is learned to yield diverse predictions for test data. The best prediction amongst these models may then be selected by an oracle or with respect to a more complex loss. Experimental evaluation shows significantly better results than using only one model and achieves state-of-the-art performance on challenging computer vision tasks.
Today, GPUs and other parallel accelerators are widely used in high performance computing, due to their high computational power and high performance per watt. Still, one of the main bottlenecks of GPU-accelerated cluster computing is the data transfer between distributed GPUs. This not only affects performance, but also power consumption. Often, a data transfer between two distributed GPUs even requires intermediate copies in host memory. This overhead penalizes small data movements and synchronization operations.
In this work, different communication methods for distributed GPUs are implemented and evaluated. First, a new technique, called GPUDirect RDMA, is implemented for the Extoll device and evaluated. The performance results show that this technique brings performance benefits for small- and mediums-sized data transfers, but for larger transfer sizes, a staged protocol is preferable since the PCIe-bus does not well support peer-to-peer data transfers.
In the next step, GPUs are integrated to the one-sided communication library GPI-2. Since this interface was designed for heterogeneous memory structures, it allows an easy integration of GPUs. The performance results show that using one-sided communication for GPUs brings some performance benefits compared to two-sided communication which is the current state-of-the-art. However, using GPI-2 for communication still requires a host thread to control GPU-related communication, although the data is transferred directly between the GPUs without any host copies. Therefore, the subsequent part of the work analyze GPU-controlled communication.
First, a put/get communication interface, based on Infiniband verbs, for the GPU is implemented. This interface enables the GPU to independently source and synchronize communication requests without any involvements of the CPU. However, the Infiniband verbs protocol adds a lot of sequential overhead to the communication, so the performance of GPU-controlled put/get communication is far behind the performance of CPU-controlled put/get communication.
Another problem is intra-GPU synchronization, since GPU blocks are non-preemptive. The use of communication requests within a GPU can easily result in a deadlock. Dynamic parallelism solves this problem. Although the performance of applications using GPU-controlled communication is still slightly worse than the performance of hybrid applications, the performance per watt increases, since the CPU can be relieved from the communication work.
As a communication model that is more in line with the massive parallelism of GPUs, the performance of a hardware-supported global address space for GPUs is evaluated. This global address space allows communication with simple load and store instructions which can be performed by multiple threads in parallel. With this method, the latency for a GPU-to-GPU data transfer can be reduced to 3us, using an FPGA. The results show that a global address space is best for applications that require small, non-blocking, and irregular data transfers. However, the main bottleneck of this method is that is does not allow overlapping of communication and computation which is the case for put/get communication. However, by using GPU optimized communication models, depending on the application, between 10 and 50% better energy efficiency can be reached than by using a hybrid model with CPU-controlled communication.
The hydraulic conductivity of a confined aquifer is estimated using the quasi-linear geostatistical approach (QLGA), based on measurements of dependent quantities such as the hydraulic head or the arrival time of a tracer. This requires the solution of the steady-state groundwater flow and solute transport equations, which are coupled by Darcy's law. The standard Galerkin finite element method (FEM) for the flow equation combined with the streamline diffusion method (SDFEM) for the transport equation is widely used in the hydrogeologists' community. This work suggests to replace the first by the two-point flux cell-centered finite volume scheme (CCFV) and the latter by the Discontinuous Galerkin (DG) method. The convection-dominant case of solute (contaminant) transport in groundwater has always posed a special challenge to numerical schemes due to non-physical oscillations at steep fronts. The performance of the DG method is experimentally compared to the SDFEM approach with respect to numerical stability, accuracy and efficient solvability of the occurring linear systems. A novel method for the reduction of numerical under- and overshoots is presented as a very efficient alternative to local mesh refinement. The applicability and software-technical integration of the CCFV/DG combination into the large-scale inversion scheme mentioned above is realized. The high-resolution estimation of a synthetic hydraulic conductivity field in a 3-D real-world setting is simulated as a showcase on Linux high performance computing clusters. The setup in this showcase provides examples of realistic flow fields for which the solution of the convection-dominant transport problem by the streamline diffusion method fails.
Dust storms emerging in the Earth's major desert regions significantly influence weather processes, the CO2-cycle and the climate on a global scale. Their effects on organisms range from providing nutrition to vegetation and microbes to direct impact on human settlements, transportation and health. The detection of dust storms, the prediction of their development, and the estimation of sources are therefore of immediate interest to a wide range of scientific disciplines. Recent spatio-temporal resolution increases of remote sensing instruments have created new opportunities to understand these phenomena. The scale of the data and their inherent stochasticity, however, pose significant challenges. This thesis develops a combination of methods from statistics, image processing, and physics that paves the way for efficient probabilistic dust assessment using satellite imagery. As a first step, we propose a BHM that maps SEVIRI measurements to a predictor of the dust density. Case studies demonstrate that, as compared to linear methods, our LSM approach mitigates effects of signal intrinsic noise on further processing steps. Furthermore, an extensive cross-validation study is employed to show that LSM successfully adapts to intra-daily changes of the infrared data and yields outstanding dust detection accuracy. Physically, the dust density and its transport process are tied together by the continuity equation. A traditional approach to determine the flow field for a given density is the variational method of Horn and Schunck (HS), which simplifies the equation to compression free motion. We characterize the equation's solution as a GMRF and introduce compressible dynamics. This link between probabilistic and variational perspectives leads to applied and theoretical advances. It enables us to employ the INLA technique for computationally efficient inference and integration over hyper-parameters. The importance of allowing for compressible motion and treating the problem in a statistical manner is emphasized by simulation and case studies showing a significant reduction in errors of the estimated flow field. In addition, we demonstrate how our methodology provides uncertainty quantification, dust storm forecasts and estimation of emission sources. The thesis is concluded by examining the analytical properties of our approach. It is shown that, under mild restrictions on an underlying Sobolev space, existence and uniqueness of the compressible flow can be guaranteed on a continuous domain and a well-posed discretization exists. Lastly, our variational calculations point to an interpretation of the density as a solution to flow-parameterized SPDE naturally extending Matern fields to non-isotropy, which provides a further step towards a joint model of dust density and flow field.
Recent data acquisition techniques permit an improved analysis of living organisms. These techniques produce 3D+t information of cell developments in unprecedentedly high resolution. Biologists have a strong desire to analyze these cell evolutions in order to find similarities in their migration and division behaviors. The exploration of such patterns helps them in understanding how cells and hence organisms are able to ensure a regular shape development. However, the enormous size of the time-dependent data with several tens of thousands of cells and the need to analyze it in 3D hinder an interactive analysis. Visualizing the data to identify and extract relevant features provides a solution to this problem. For this, new visualization approaches are required that reduce the complexity of the data to detect important features in the visual analysis.
In this thesis, novel visual similarity analysis methods are presented to interactively process very large 3D+t data of cell developments. Three main methods are developed that allow different visual analysis strategies. The usefulness of them is demonstrated by applications to cells from zebrafish embryos and Arabidopsis thaliana plants. Both data sets feature a high regularity in the shape formation of the organs and domain experts seek to research similar cell behaviors that are responsible for this development. For example, the identification of 3D division behaviors in plants is still an unresolved issue. The first method is a novel visualization approach that can automatically classify cell division types in plant data sets with high memory and time efficiency. The visualization is based on the generation of newly introduced cell isosurfaces that allow a quantitative and spatial comparison of cell division behaviors among individual plants. The method is applied to cells of the lateral root of Arabidopsis plants and reveals similar division schemes with respect to their temporal order. The second method enables a new visual similarity analysis for arbitrary 3D trajectory data in order to extract similar movement behaviors. The algorithm performs a grouping of thousands of trajectories with an optional level of detail modification. The clustering is based on a newly weighted combination of geometry and migratory features for which the weights are used to emphasize feature combinations. As a result, similar collective cell movements in zebrafish as well as a hitherto unknown correlation between division types and subsequent nuclei migrations in the Arabidopsis plants are detected. The third method is a novel visualization technique called the structure map. It permits a compact and interactive similarity analysis of thousands of binary tree structures. Unique trees are pre-ordered in the map based on spectral similarities and substructures are highlighted according to user-selected tree descriptors. Applied to cell developments from zebrafish depicted as trees, the map achieves compression rates up to 95% according to spectral analysis and facilitates an immediate identification of biologically implausible events and outliers. Additionally, similar quantities of feature appearances are detected in the center of the lateral root of several Arabidopsis plants.
Being able to provide accurate forecasts of future quantities has always been a great human desire and is essential in numerous situations in daily life. Meanwhile, it has become routine to work with probabilistic forecasts in the form of full predictive distributions rather than with single deterministic point forecasts in many disciplines, with weather prediction acting as a key example.
Nowadays, probabilistic weather forecasts are usually constructed from ensemble prediction systems, which consist of multiple runs of numerical weather prediction models differing in the initial conditions and/or the parameterized numerical representation of the atmosphere. The raw ensemble forecasts typically reveal biases and dispersion errors and thus call for statistical postprocessing to realize their full potential. Several ensemble postprocessing methods have been developed and are partly recapitulated in this thesis, yet many of them only apply to a single weather quantity at a single location and for a single prediction horizon. In many applications, however, there is a critical need to account for spatial, temporal and inter-variable dependencies.
To address this, a tool called ensemble copula coupling (ECC) is introduced and examined. Essentially, ECC uses the empirical copula induced by the raw ensemble to aggregate samples from predictive distributions for each location, variable and look-ahead time separately, which are obtained via existing univariate postprocessing methods. The ECC ensemble inherits the multivariate rank dependence pattern from the raw ensemble, thereby capturing the flow dependence.
Several variants and modifications of ECC are studied, and it is demonstrated that the ECC concept provides an overarching frame for existing techniques scattered in the literature.
From a mathematical point of view, it is shown that ECC can be considered a copula approach by pointing out relationships to multivariate discrete copulas, which are introduced in this thesis and for which relevant mathematical properties are derived.
A generalization of standard ECC is introduced, which aggregates samples from not necessarily univariate, but general predictive distributions obtained by low-dimensional postprocessing in an ECC-like manner.
Finally, the SimSchaake approach, which combines the notion of similarity-based ensemble methods with that of the so-called Schaake shuffle, is presented as an alternative to ECC. In this technique, the dependence patterns are based on verifying observations rather than on raw ensemble forecasts as in ECC.
The methods and concepts are illustrated and evaluated based on case studies, using real ensemble forecast data of the European Centre for Medium-Range Weather Forecasts. Essentially, the new multivariate approaches developed in this thesis reveal good predictive performances, thus contributing to improved probabilistic forecasts.