Constructing Composite Likelihoods in General Random Fields
Sebastian Nowozin
16 Apr 2013 2 Comments ICML Workshop on Interactions between Inference and Learning
ICML Workshop on Interactions between Inference and Learning
We propose a simple estimator based on composite likelihoods for parameter learning in random field models. The estimator can be applied to all discrete graphical models such as Markov random fields and conditional random fields, including ones with higher-order energies. It is computationally efficient because it requires only inference over tree-structured subgraphs of the original graph, and it is consistent, that is, it asymptotically gives the optimal parameter estimate in the model class. We verify these conceptual advantages in synthetic experiments and demonstrate the difficulties encountered by popular alternative estimation approaches.
State From To ( Cc) Subject Date Due Action
Completed
Sebastian Nowozin ICML Workshop on Interactions between Inference and Learning
Request for Endorsed for oral presentation: Constructing Composite Likelihoods...

16 Apr 2013
Fulfill
Sebastian Nowozin ICML Workshop on Interactions between Inference and Learning
Fulfilled: Inferning 2013 call for workshop papers

16 Apr 2013
Reveal: document
Sebastian Nowozin
Revealed: document: Constructing Composite Likelihoods in General Random Fields

16 Apr 2013
Completed
Sameer Singh Anonymous c554
Request for review of Constructing Composite Likelihoods in General Random Fields

09 May 2013 01 Mar 2013
Completed
Sameer Singh Anonymous 6f70
Request for review of Constructing Composite Likelihoods in General Random Fields

09 May 2013 01 Mar 2013
Reveal: document
Sameer Singh
Revealed: document: Endorsed for oral presentation: Constructing Composite...

29 May 2013
Fulfill
Sameer Singh Sebastian Nowozin
Fulfilled: Request for Endorsed for oral presentation: Constructing Composite...

29 May 2013

2 Comments

Anonymous 6f70 21 May 2013
State From To ( Cc) Subject Date Due Action
Reveal: document
Anonymous 6f70
Revealed: document: review of Constructing Composite Likelihoods in General...

21 May 2013
Fulfill
Anonymous 6f70 Sameer Singh
Fulfilled: Request for review of Constructing Composite Likelihoods in General...

21 May 2013
This is a very well written and well structured paper (albeit significantly longer than the workshop suggested limits). The paper provides a good overview of conditional Markov random field parameter learning with a focus on composite likelihood methods. It then proposes two specializations of composite likelihood learning, but with interesting properties that are exploited for learning structured models. The experiments are simple but sufficient for a workshop paper of this type. Overall the paper is very good fit to this workshop.
Please log in to comment.
Anonymous c554 29 May 2013
State From To ( Cc) Subject Date Due Action
Reveal: document
Anonymous c554
Revealed: document: review of Constructing Composite Likelihoods in General...

29 May 2013
Fulfill
Anonymous c554 Sameer Singh
Fulfilled: Request for review of Constructing Composite Likelihoods in General...

29 May 2013
This paper presents two new estimators for parameter learning in MRFs/CRFs, based on the general theory of composite likelihood (which is a generalization of pseudo-likelihood). The two estimators maximize Criss-Cross Likelihood (MXXLE) and V-Acyclic Subgraph Likelihood (MCLE), and are both consistent, convex and optimizable in the fully observed case. The experiments suggest that these new estimators work well, in terms of parameter estimation error not prediction performance on held-out data (which is what the practitioners would care about the most, but is not reported in the paper). The estimators are based on the general theory of composite likelihoods, but certainly novel. The document is very well written and does a great job reviewing the background. Overall, the paper is certainly a great fit for inferning, because it does a great job at discussing issues in using approximate inference for parameter learning.
Please log in to comment.
ICML Workshop on Interactions between Inference and Learning 29 May 2013
State From To ( Cc) Subject Date Due Action
Reveal: document
Sameer Singh
Revealed: document: Endorsed for oral presentation: Constructing Composite...

29 May 2013
Fulfill
Sameer Singh Sebastian Nowozin
Fulfilled: Request for Endorsed for oral presentation: Constructing Composite...

29 May 2013
Endorsed for oral presentation: Constructing Composite Likelihoods in General Random Fields
Please log in to comment.

Please log in to comment.