Campus Units
Statistics, Industrial and Manufacturing Systems Engineering
Document Type
Article
Publication Version
Accepted Manuscript
Publication Date
11-5-2019
Journal or Book Title
Information and Inference: A Journal of the IMA
DOI
10.1093/imaiai/iaz022
Abstract
A probability model exhibits instability if small changes in a data outcome result in large and, often unanticipated, changes in probability. This instability is a property of the probability model, given by a distributional form and a given configuration of parameters. For correlated data structures found in several application areas, there is increasing interest in identifying such sensitivity in model probability structure. We consider the problem of quantifying instability for general probability models defined on sequences of observations, where each sequence of length N has a finite number of possible values that can be taken at each point. A sequence of probability models, indexed by N, and an associated parameter sequence result to accommodate data of expanding dimension. Model instability is formally shown to occur when a certain log probability ratio under such models grows faster than N. In this case, a one component change in the data sequence can shift probability by orders of magnitude. Also, as instability becomes more extreme, the resulting probability models are shown to tend to degeneracy, placing all their probability on potentially small portions of the sample space. These results on instability apply to large classes of models commonly used in random graphs, network analysis and machine learning contexts.
Copyright Owner
The Author(s)
Copyright Date
2019
Language
en
File Format
application/pdf
Recommended Citation
Kaplan, Andee; Nordman, Daniel J.; and Vardeman, Stephen B., "On the S-instability and degeneracy of discrete deep learning models" (2019). Statistics Publications. 289.
https://lib.dr.iastate.edu/stat_las_pubs/289
Comments
This is a pre-copyedited, author-produced version of an article accepted for publication in Information and Inference: A Journal of the IMA following peer review. The version of record: Kaplan, Andee, Daniel J. Nordman, and Stephen B. Vardeman. "On the S-instability and degeneracy of discrete deep learning models," Information and Inference: A Journal of the IMA is available online at DOI: 10.1093/imaiai/iaz022. Posted with permission.