title: Active Learning: New Approaches, and Industrial Applications
creator: Röder, Jens
subject: 004
subject: 004 Data processing Computer science
subject: 500
subject: 500 Natural sciences and mathematics
description: Active learning is one form of supervised machine learning. In supervised learning, a set of labeled samples is passed to a learning algorithm for training a classifier. However,  labeling large amounts of training samples can be costly and error-prone. Active learning deals with the development of algorithms that interactively select a subset of  the available unlabeled samples for labeling, and aims at minimizing the labeling effort while maintaining classification performance.  The key challenge for the development of so-called active learning strategies is the balance between exploitation and exploration: On the one hand, the estimated decision  boundary needs to be refined in feature space regions where it has already been established, while, on the other hand, the feature space needs to be scanned carefully for unexpected class distributions. In this thesis, two approaches to active learning are presented that consider these two aspects in a novel way.  In order to lay the foundations for the first one, it is proposed to express the uncertainty in class prediction of a classifier at a test point in terms of a second-order distribution. The mean of this distribution corresponds to the common estimate of the posterior class probabilities and thus is related to the distance of the test point to the decision boundary, whereas the spread of the distribution indicates the degree of exploration in  the corresponding region of feature space. This allows for the evaluation of the utility of labeling a yet unlabeled point with respect to classifier improvement in a principled  way and leads to a completely novel approach to active learning. The proposed strategy is then implemented and evaluated based on kernel density classification.  The generic active learning strategy can be combined with any other classifier, but it performs best if the derived second-order distributions are sufficiently good approximations to the sampling distribution. Although second-order distributions for random forests are derived in this thesis, they do not approximate sufficiently well the sampling distribution and mainly allow only for the relative comparison of prediction uncertainty between test points. In order to combine the state of the art classification performance of random forests with the principal ideas of the first active learning approach, a related second approach for random forests is derived. It is, in addition, tailored to the demands in industrial optical inspection: bag-wise labeling with weak labels and strongly imbalanced classes. Moreover, an outlier detection scheme based on random forests is derived that is used by the proposed active learning algorithm.  Finally, a new computational scheme for Gaussian process classification is presented. It is compared to two standard methods in geostatistics, both with respect to theoretical consistency and practical performance. The method evolved as a by-product when considering using Gaussian process models for active learning.
date: 2013-01-24
type: Dissertation
type: info:eu-repo/semantics/doctoralThesis
type: NonPeerReviewed
format: application/pdf
identifier: https://archiv.ub.uni-heidelberg.de/volltextserverhttps://archiv.ub.uni-heidelberg.de/volltextserver/14379/1/Dissertation_JensRoeder.pdf
identifier: DOI:10.11588/heidok.00014379
identifier: urn:nbn:de:bsz:16-heidok-143791
identifier:   Röder, Jens  (2013) Active Learning: New Approaches, and Industrial Applications.  [Dissertation]     
relation: https://archiv.ub.uni-heidelberg.de/volltextserver/14379/
rights: info:eu-repo/semantics/openAccess
rights: http://archiv.ub.uni-heidelberg.de/volltextserver/help/license_urhg.html
language: eng