Master's thesis talk. Yi-Han is advised by Felix Dietrich and Severin Reiz.
Previous talks at the SCCS Colloquium
Yi-Han Hsieh: Second Order training for Natural Language Processing using Newton-CG Optimizer
SCCS Colloquium |
We present a Hessian-based second-order optimizer called Newton-CG to solve a Portuguese to English Machine Translation (MT) Task in the Natural Language Processing (NLP) field. We mainly focus on the comparison of the performance between Newton-CG and first-order optimizers Adam and Stochastic gradient descent(SGD) that both are widely used in deep learning tasks. In this task, We use the most dominant NMT model called Transformer.