Data is as Data Does: The Influence of Computation on Inference

John Patrick Cunningham (Columbia University)
simons.berkeley.edu/talks/joh...
AI≡Science: Strengthening the Bond Between the Sciences and Artificial Intelligence
Probabilistic models remain a hugely popular class of techniques in modern machine learning, and their expressiveness has been extended by modern large-scale compute. While exciting, these generalizations almost always come with approximations, and researchers typically ignore the fundamental influence of computational approximations. Thus, results from modern probabilistic methods become as much about the approximation method as they are about the data and the model, undermining both the Bayesian principle and the practical utility of inference in probabilistic models for real applications in science and industry.
To expose this issue and to demonstrate how to do approximate inference correctly in at least one model class, in this talk I will derive a new type of Gaussian Process approximation that provides consistent estimation of the combined posterior arising from both the finite number of data observed and the finite amount of computation expended. The most common GP approximations map to an instance in this class, such as methods based on the Cholesky factorization, conjugate gradients, and inducing points. I will show the consequences of ignoring computational uncertainty, and prove that implicitly modeling it improves generalization performance. I will show how to do model selection while considering computation, and I will describe an application to neurobiological data.

[@alidogramaci7468]

I am delighted to see such good work is being carried out at Columbia. One question I have as I am midway into your presentation:
What you call the effective data set: is it unique? Can you build a confidence or credible set (region) around it?

[@hyperduality2838]

Comparison, reflection, abstraction -- Immanuel Kant.
Abstraction is the process of creating new concepts or ideas according to Immanuel Kant.
Creating new concepts is a syntropic process -- teleological.
Syntax is dual to semantics -- languages or communication, data.
Large language models are using duality, if mathematics is a language then it is dual.
Sense is dual to nonsense.
Right is dual to wrong.
"Only the Sith think in terms of absolutes" -- Obi Wan Kenobi.
"Sith lords come in pairs" -- Obi Wan Kenobi.
Concepts are dual to percepts" -- the mind duality of Immanuel Kant.
The intellectual mind/soul (concepts) is dual to the sensory mind/soul (percepts) -- the mind duality of Thomas Aquinas.
Your mind/soul converts perceptions or measurements into conceptions or ideas, mathematicians create new concepts all the time from their observations, intuitions or perceptions.
The mind/soul is actually dual.
Mind (syntropy) is dual to matter (entropy) -- Descartes or Plato's divided line.
Your mind converts entropy or average information into syntropy or mutual information -- information (data) is dual.
Concepts or ideas are therefore syntropic in form or structure.
Teleological physics (syntropy) is dual to non teleological physics (entropy) -- physics is dual.
Syntropy (prediction) is dual to increasing entropy -- the 4th law of thermodynamics!
Duality creates reality!
"Always two there are" -- Yoda.
Physics is all about generalization or abstraction -- a syntropic process, teleological.
Truth is dual to falsity -- propositional logic or Bayesian logic.
Absolute truth is dual to relative truth -- Hume's fork.
Truth is dual.

[@pensiveintrovert4318]

Maybe preparing first would help sounding like a lecturer instead of a highschooler spitting out random statements.