- Видео 60
- Просмотров 19 820
RPI Seminars MIDO
США
Добавлен 24 июл 2020
academic talks
Manifold learning for point-cloud data with applications in biology
In this talk, I will introduce LOT Wassmap, a computationally feasible algorithm to uncover low-dimensional structures in the Wasserstein space. The algorithm is motivated by the observation that many datasets are naturally interpreted as point-clouds or probability measures rather than points in $\mathbb{R}^n$, and that finding low-dimensional descriptions of such datasets requires manifold learning algorithms in the Wasserstein space. Most available algorithms are based on computing the pairwise Wasserstein distance matrix, which can be computationally challenging for large datasets in high dimensions. The proposed algorithm leverages approximation schemes such as Sinkhorn distances and...
Просмотров: 208
Видео
New Federated Algorithms for Deep Learning with Unbounded Smooth Landscape
Просмотров 258Год назад
The current analysis of federated optimization algorithms for training deep neural networks typically requires the loss landscape to be smooth. However, there is a class of neural networks which is inherently nonsmooth, with a potentially unbounded smoothness parameter. Examples include recurrent neural networks, long-short-term memory networks, and transformers. It remains unclear how to desig...
Bilevel Optimization: Stochastic Algorithms and Applications in Inverse Reinforcement Learning
Просмотров 3,3 тыс.Год назад
Bilevel Optimization is a class of optimization problem that has two levels of nested optimization subproblems. It can be used to model applications arising in areas such as signal processing, machine learning, game theory, etc. In this talk, we will first discuss several recent works, including a few of our own works, which develop efficient stochastic algorithms for this class of problems. Th...
Sparse coding via a Delaunay triangulation. Abiy Tasissa@Tufts Univ.
Просмотров 118Год назад
Sparse coding is a technique of representing data as a sparse linear combination of a set of vectors. This representation facilitates computation and analysis of high-dimensional data that is prevalent in many applications. We study sparse coding in the setting where the set of vectors define a unique Delaunay triangulation. We propose a weighted l1 regularizer and show that it provably yields ...
A proximal algorithm for sampling, Yongxin Chen@Gtech
Просмотров 446Год назад
The sampling problem to draw random samples from an unnormalized distribution plays an important role in many areas such as Bayesian inference and computational biology. In this work, we present new results on the proximal sampler, a recent method introduced by Lee, Shen, and Tian in 2021. The proximal sampler can be viewed as a sampling analog of the proximal point method in optimization and w...
Scalable, Projection-Free Optimization Methods
Просмотров 187Год назад
We will introduce an approach to constrained optimization replacing orthogonal projections with much cheaper radial ones. This results in new first-order methods that are (i) scalable, sporting minimal per iteration costs, (ii) always yield fully feasible solutions, and (iii) applicable to a wide family of potentially nonconvex constraints. For example, when applied to linear programs, iteratio...
Better Data Ordering for Stochastic Gradient Descent
Просмотров 127Год назад
Training example order in SGD has long been known to affect convergence rate. Recent results show that accelerated rates are possible in a variety of cases for permutation-based sample orders, in which each example from the training set is used once before any example is reused. This talk will cover a line of work in my lab on sample-ordering schemes. We will discuss the limits of the classic r...
Continuous-in-time Limit for Bayesian Bandits, YuhuaZhu@UCSD
Просмотров 152Год назад
This talk revisits the bandit problem in the Bayesian setting. The Bayesian approach formulates the bandit problem as an optimization problem, and the goal is to find the optimal policy which minimizes the Bayesian regret. One of the main challenges facing the Bayesian approach is that computation of the optimal policy is often intractable, especially when the length of the problem horizon or t...
Multiscale inverse problem: Interaction between multiple scales in physical system reconstruction
Просмотров 115Год назад
Inverse problems are ubiquitous. We probe the media with sources and measure the outputs, to infer the media information. At the scale of quantum, classical, statistical and fluid, we face inverse Schroedinger, inverse Newton’s second law, inverse Boltzmann problem, and inverse diffusion respectively. In this talk, we discuss the connection between these problems. In particular, we will provide...
Offline and Online Learning and Decision-Making in the Predict-then-Optimize Setting
Просмотров 5962 года назад
In the predict-then-optimize setting, the parameters of an optimization task are predicted based on contextual features, and it is desirable to leverage the structure of the underlying optimization task when training a machine learning model. A natural loss function in this setting is based on considering the cost of the decisions induced by the predicted parameters, in contrast to standard mea...
Machine learning meets dynamics & variational Stiefel optimization, Molei Tao@Gatech
Просмотров 2212 года назад
The interaction between machine learning and dynamics can lead to both new methodology for dynamics, and deepened understanding and/or efficacious algorithms for machine learning. This talk will focus on the latter. Specifically, in half of the talk, I will describe some of the nontrivial (and pleasant) effects of large learning rates, which are often used in practical training of machine learn...
Implicit biases of optimization algs for NNs: static & dynamic perspectives, Chao Ma@Stanford
Просмотров 1372 года назад
Modern neural networks are usually over-parameterized-the number of parameters exceeds the number of training data. In this case the loss functions tend to have many (or even infinite) global minima, which imposes an additional challenge of minima selection on optimization algorithms besides the convergence. Specifically, when training a neural network, the algorithm not only has to find a glob...
Graph Clustering Dynamics: From Spectral to Mean Shift, Katy Craig@UCSB
Просмотров 2342 года назад
Abstract: Clustering algorithms based on mean shift or spectral methods on graphs are ubiquitous in data analysis. However, in practice, these two types of algorithms are treated as conceptually disjoint: mean shift clusters based on the density of a dataset, while spectral methods allow for clustering based on geometry. In joint work with Nicolás García Trillos and Dejan Slepčev, we define a n...
Shortest Paths in Graphs of Convex Sets, with Applications to Control and Motion Planning
Просмотров 1,9 тыс.2 года назад
Given a graph, the shortest-path problem requires finding a sequence of edges of minimum cost connecting a source vertex to a target vertex. In this talk we introduce a generalization of this classical problem in which the position of each vertex in the graph is a continuous decision variable, constrained to lie in a corresponding convex set, and the cost of an edge is a convex function of the ...
Deep Image Prior (and Its Cousin) for Inverse Problems: the Untold Stories
Просмотров 1 тыс.2 года назад
Deep image prior (DIP) parametrizes visual objects as outputs of deep neural networks (DNNs); its consin neural implicit representation (NIR) directly parametrizes visual objects as DNNs. These stunningly simple ideas, when integrated into natural optimization formulations for visual inverse problems, have matched or even beaten the state-of-the-art methods on numerous visual reconstruction tas...
Learning Nonlocal Operators for Heterogeneous Material Modeling, Yue Yu@Lehigh
Просмотров 2062 года назад
Learning Nonlocal Operators for Heterogeneous Material Modeling, Yue Yu@Lehigh
Global Loss Landscape of Neural Networks: Knowns and Unknowns
Просмотров 4032 года назад
Global Loss Landscape of Neural Networks: Knowns and Unknowns
Universal Conditional Gradient Sliding for Convex Optimization
Просмотров 2232 года назад
Universal Conditional Gradient Sliding for Convex Optimization
Units-equivariant machine learning, Soledad Villar@JHU
Просмотров 1112 года назад
Units-equivariant machine learning, Soledad Villar@JHU
Symmetry-preserving machine learning, Wei Zhu@UMASS
Просмотров 1792 года назад
Symmetry-preserving machine learning, Wei Zhu@UMASS
How Differential Equations Insight Benefit Deep Learning, Bao Wang@Univ. of Utah
Просмотров 1962 года назад
How Differential Equations Insight Benefit Deep Learning, Bao Wang@Univ. of Utah
Algorithms for Deterministically Constrained Stochastic Optimization
Просмотров 3942 года назад
Algorithms for Deterministically Constrained Stochastic Optimization
Active strict saddles in nonsmooth optimization
Просмотров 952 года назад
Active strict saddles in nonsmooth optimization
On the Robustness of Deep Neural Networks
Просмотров 1722 года назад
On the Robustness of Deep Neural Networks
A Stochastic variance-reduced Primal-dual Method for Convex-concave Saddle point Problems
Просмотров 1812 года назад
A Stochastic variance-reduced Primal-dual Method for Convex-concave Saddle point Problems
Equivariant Neural Networks for Learning Spatiotemporal Dynamics, R.Walters, Northeastern University
Просмотров 1842 года назад
Equivariant Neural Networks for Learning Spatiotemporal Dynamics, R.Walters, Northeastern University
Inexact Proximal Gradient Method with Subspace Acceleration
Просмотров 1102 года назад
Inexact Proximal Gradient Method with Subspace Acceleration
Distributed stochastic non-convex optimization: Optimal regimes and tradeoffs
Просмотров 983 года назад
Distributed stochastic non-convex optimization: Optimal regimes and tradeoffs
Data-driven discovery of interacting particle system using Gaussian processes, S. Tang@UCSB
Просмотров 1783 года назад
Data-driven discovery of interacting particle system using Gaussian processes, S. Tang@UCSB
has the project been opensoueced
poor speak , sadly
inspiring talk! This talk makes me understand the advances of neural ode much better than before.
Some segments in the video are stamped not adjacent to each other
Is there working code/demo for this work?