title: Can oracle-based imitation learning improveneural machine translation with dataaggregation?
creator: Hormann, Luca
subject: ddc-004
subject: 004 Data processing Computer science
description: Through  globalization  of  the  industry  and  the  collaboration  of  nations  world  wide  theimportance  of  machine  translated  documents  is  rising.   Many  new  ideas  and  conceptswere proposed in the recent years to improve the overall quality of machine translation(MT).  A  lot  of  focus  in  machine  learning  is  going  towards  the  research  of  alternativelearning techniques, as the basic existing paradigms such as supervised, unsupervised andreinforcement learning are not a perfect fit for every task.  Imitation learning is a techniquewhich combines the exploratory aspect of reinforcement with the efficiency of supervisedlearning.   Through  the  usage  of  an  interactive  expert,  the  learning  model,  also  calledstudent, is able to obtain intermediate feedback for it’s predictions at any given point intime.  Thereby the student is aware of it’s mistakes by considering the difference of it’s andthe expert’s prediction from this point.  Imitation learning can improve all major problemsfor training artificial neural networks given the right expert.  These are:  the time it takesto train a model,  the acquisition of data and most importantly the overall performancegiven some specific metric.The  most  common  quality  evaluation  metric  used  for  MT  is  BLEU.  It  is  based  on  then-gram precision between the generated translation, given some input, and the referencetranslation.  Therefore it is non-differentiable and can not be used as a loss to train a MTmodel directly. For the training usually the maximum likelihood estimation (MLE) is used,such that the likelihood of each token in the output, given the input sequence, is maximized.This creates a discrepancy between training (MLE) and validation objective (BLEU). Thisthesis tries to overcome this issue by directly learning on the differences of the expectedBLEU from the student and the expert in an imitation learning scenario.  The expert isrepresented by a traditional statistical machine translation (SMT) model that should helpthe student in solving the problems mentioned above.  For this a novel data aggregationmethodAggregateData  using  approximatedBLEUexplorations  (ADBLEU)  based  onimitation  learning  was  implemented.   After  conducting  several  experiments,  validatingdifferent approaches of data aggregation, it is shown that it is not possible to significantlyimprove the state-of-the-art student, due to limitations of the SMT expert.
date: 2021
type: Master's thesis
type: info:eu-repo/semantics/masterThesis
type: NonPeerReviewed
format: application/pdf
identifier: https://archiv.ub.uni-heidelberg.de/volltextserverhttps://archiv.ub.uni-heidelberg.de/volltextserver/30516/1/Master_Thesis.pdf
identifier: DOI:10.11588/heidok.00030516
identifier: urn:nbn:de:bsz:16-heidok-305169
identifier:   Hormann, Luca  (2021) Can oracle-based imitation learning improveneural machine translation with dataaggregation?  [Master's thesis]     
relation: https://archiv.ub.uni-heidelberg.de/volltextserver/30516/
rights: info:eu-repo/semantics/openAccess
rights: http://archiv.ub.uni-heidelberg.de/volltextserver/help/license_urhg.html
language: eng