arXiv subjects

Tongzhou Wang

Publications and source records attributed to Tongzhou Wang.

22 records · Page 2Linked to original sources

arXiv2020

Dataset Distillation

Model distillation aims to distill the knowledge of a complex model into a simpler one. In this paper, we consider an alternative formulation called dataset distillation: we keep the model fixed and instead attempt to distill the knowledge from a large training dataset into a small one. The idea is to synthesize a small number of data points that do not need to come from the correct data distribution, but will, when given to the learning algorithm as training data, approximate the model trained on the original data. For example, we show that it is possible to compress 60,000 MNIST training images into just 10 synthetic distilled images (one per class) and achieve close to original performance with only a few gradient descent steps, given a fixed network initialization. We evaluate our method in various initialization settings and with different learning objectives. Experiments on multiple datasets show the advantage of our approach compared to alternative methods.

cs.LG↗

arXiv2019

All-Poly(ionic liquid) Membrane-derived Porous Carbon Membranes: Scalable Synthesis and Application for Photothermal Conversion in Seawater Desalination

Herein we firstly introduce a straightforward, scalable and technologically relevant strategy to manufacture charged porous polymer membranes (CPMs) in a controllable manner. The pore sizes and porous architectures of CPMs are well-controlled by rational choice of anions in poly(ionic liquid)s (PILs). Continuously, heteroatom-doped hierarchically porous carbon membrane (HCMs) can be readily fabricated via morphology-maintaining carbonization of as-prepared CPMs. These HCMs being as photothermal membranes exhibited excellent performance for solar seawater desalination, representing a promising strategy to construct advanced functional nanomaterials for portable water production technologies.

physics.app-ph↗

arXiv2019

Meta-Learning MCMC Proposals

Effective implementations of sampling-based probabilistic inference often require manually constructed, model-specific proposals. Inspired by recent progresses in meta-learning for training learning agents that can generalize to unseen environments, we propose a meta-learning approach to building effective and generalizable MCMC proposals. We parametrize the proposal as a neural network to provide fast approximations to block Gibbs conditionals. The learned neural proposals generalize to occurrences of common structural motifs across different models, allowing for the construction of a library of learned inference primitives that can accelerate inference on unseen models with no model-specific training required. We explore several applications including open-universe Gaussian mixture models, in which our learned proposals outperform a hand-tuned sampler, and a real-world named entity recognition task, in which our sampler yields higher final F1 scores than classical single-site Gibbs sampling.

cs.AI↗

arXiv2017

Learning to Synthesize a 4D RGBD Light Field from a Single Image

We present a machine learning algorithm that takes as input a 2D RGB image and synthesizes a 4D RGBD light field (color and depth of the scene in each ray direction). For training, we introduce the largest public light field dataset, consisting of over 3300 plenoptic camera light fields of scenes containing flowers and plants. Our synthesis pipeline consists of a convolutional neural network (CNN) that estimates scene geometry, a stage that renders a Lambertian light field using that geometry, and a second CNN that predicts occluded rays and non-Lambertian effects. Our algorithm builds on recent view synthesis methods, but is unique in predicting RGBD for each light field ray and improving unsupervised single image depth estimation by enforcing consistency of ray depths that should intersect the same scene point. Please see our supplementary video at https://youtu.be/yLCvWoQLnms

cs.CV↗