Searcharxiv⌕ Search

arXiv subjects

Zhicheng Shi

Publications and source records attributed to Zhicheng Shi.

2 recordsLinked to original sources

ReTaCo: Residual-Target Control for On-Policy Distillation

On-policy distillation (OPD) trains a student on its own generated prefixes with token-level teacher feedback, but transmitting or storing the teacher's full-vocabulary distribution at every token is costly. Entropy-aware OPD (EOPD) adds forward supervision to reverse KL to help the student recover plausible tokens it underestimates, using only the teacher's top-$k$ probabilities to limit cost. Because EOPD renormalizes these probabilities, its target assigns no mass to the omitted vocabulary. We prove that the resulting loss keeps pushing the student's top-$k$ mass toward one even after the student matches the teacher's relative probabilities within the top-$k$ set, so the teacher itself is not a stationary point whenever the omitted tokens have positive teacher probability. We propose ReTaCo (Residual-Target Control), which keeps the top-$k$ tokens individually and groups the remaining tokens into one residual symbol, and pairs this forward target with a single-sample estimator whose expectation equals the full-vocabulary reverse KL. With teacher top-$k$ mass $m$, the residual target is $(1-β)(1-m)$ for $β\in[0,1]$: $β=0$ preserves the teacher's mass, and larger $β$ moves more mass onto the top-$k$ tokens without changing their relative probabilities. At a fixed prefix, we prove that the population objective has a unique optimum whose top-$k$ mass lies between $m$ and $m+β(1-m)$ and increases monotonically with $β$; at $β=0$, underestimated top-$k$ tokens still receive non-vanishing recovery gradients. Numerical optimization confirms these predictions, and across three teacher-student pairs, ReTaCo outperforms EOPD on most mathematics and code benchmarks.

cs.LG↗

Unveiling the secrets of the mid-infrared Moon

The Moon's optical characteristics in visible and long-wavelength infrared (LWIR) have long been observed with our eyes or with instruments. What the mid-infrared (MIR) Moon looks like is still a mystery. For the first time we present detailed appearance of the MIR Moon observed by a high-resolution geostationary satellite and reveal the essence behind its appearance. The appearance of the MIR Moon is opposite to its normal visible appearance. In addition the MIR Moon shows limb darkening. Both the absolute and the relative brightness distribution of the MIR lunar disk changes with the solar incidence angle. The signatures of the MIR Moon are controlled by both the reflection and emission of the lunar surface. We also show first-ever brightness temperature maps of the lunar disk without needing a mosaic, which better show the temperature variation across the lunar disk. They reveal that the relationship between brightness temperature and solar incidence angle i is cos1/bi, and the power parameter is smaller than the Lambertian temperature model of cos1/4i observed for lunar orbit-based measurements. The slower decrease of the brightness temperature when moving away from the sub-solar point than the Lambertian model is due to topographic effects. The brightness temperature is dominated by albedo and the solar incidence angle and influenced by the topography. Our results indicate that the Moon in the MIR exhibits many interesting phenomena which were previously unknown, and contains abundant information about lunar reflection and thermal emission for future study.

astro-ph.EP↗