Please use this identifier to cite or link to this item:
Title: Accelerating Newton optimization for log-linear models through feature redundancy
Keywords: Newton-Raphson Method
Feature Extraction
Mathematical Models
Regression Analysis
Issue Date: 2006
Publisher: IEEE
Citation: Proceedings of the Sixth International Conference on Data Mining, Hong Kong, China, 18-22 December 2006, 1-10
Abstract: Log-linear models are widely used for labeling feature vectors and graphical models, typically to estimate robust conditional distributions in presence of a large number of potentially redundant features. Limited-memory quasi-Newton methods like LBFGS or BLMVM are optimization workhorses for such applications, and most of the training time is spent computing the objective and gradient for the optimizer. We propose a simple technique to speed up the training optimization by clustering features dynamically, and interleaving the standard optimizer with another, coarse-grained, faster optimizer that uses far fewer variables. Experiments with logistic regression training for text classification and conditional random field (CRF) training for information extraction show promising speed-ups between 2× and 9× without any systematic or significant degradation in the quality of the estimated models.
URI: 10.1109/ICDM.2006.11
ISBN: 0-7695-2701-7
Appears in Collections:Proceedings papers

Files in This Item:
File Description SizeFormat 
4053013.pdf1.35 MBAdobe PDFThumbnail

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.