arXiv subjects

Lin Niu

Publications and source records attributed to Lin Niu.

27 records · Page 2Linked to original sources

arXiv2023

RPTQ: Reorder-based Post-training Quantization for Large Language Models

Large-scale language models (LLMs) have demonstrated impressive performance, but their deployment presents challenges due to their significant memory usage. This issue can be alleviated through quantization. In this paper, we identify that the challenge in quantizing activations in LLMs arises from varying ranges across channels, rather than solely the presence of outliers. To address this challenge, we introduce a quantization method called RPTQ, which utilizes a reorder-based approach. By rearranging the channels and quantizing them in clusters, RPTQ effectively mitigates the impact of range differences between channels. To minimize the overhead of the reorder operation, we fuse it into the layer norm operation and weights in linear layers. In our experiments, RPTQ achieved a significant breakthrough by utilizing 3-bit activation in LLMs for the first time, resulting in a substantial reduction in memory usage. For instance, quantizing OPT-175b can lead to a memory consumption reduction of up to 80%.

cs.CL↗

arXiv2023

Improving Post-Training Quantization on Object Detection with Task Loss-Guided Lp Metric

Efficient inference for object detection networks is a major challenge on edge devices. Post-Training Quantization (PTQ), which transforms a full-precision model into low bit-width directly, is an effective and convenient approach to reduce model inference complexity. But it suffers severe accuracy drop when applied to complex tasks such as object detection. PTQ optimizes the quantization parameters by different metrics to minimize the perturbation of quantization. The p-norm distance of feature maps before and after quantization, Lp, is widely used as the metric to evaluate perturbation. For the specialty of object detection network, we observe that the parameter p in Lp metric will significantly influence its quantization performance. We indicate that using a fixed hyper-parameter p does not achieve optimal quantization performance. To mitigate this problem, we propose a framework, DetPTQ, to assign different p values for quantizing different layers using an Object Detection Output Loss (ODOL), which represents the task loss of object detection. DetPTQ employs the ODOL-based adaptive Lp metric to select the optimal quantization parameters. Experiments show that our DetPTQ outperforms the state-of-the-art PTQ methods by a significant margin on both 2D and 3D object detectors. For example, we achieve 31.1/31.7(quantization/full-precision) mAP on RetinaNet-ResNet18 with 4-bit weight and 4-bit activation.

cs.CV↗

arXiv2023

PD-Quant: Post-Training Quantization based on Prediction Difference Metric

Post-training quantization (PTQ) is a neural network compression technique that converts a full-precision model into a quantized model using lower-precision data types. Although it can help reduce the size and computational cost of deep neural networks, it can also introduce quantization noise and reduce prediction accuracy, especially in extremely low-bit settings. How to determine the appropriate quantization parameters (e.g., scaling factors and rounding of weights) is the main problem facing now. Existing methods attempt to determine these parameters by minimize the distance between features before and after quantization, but such an approach only considers local information and may not result in the most optimal quantization parameters. We analyze this issue and ropose PD-Quant, a method that addresses this limitation by considering global information. It determines the quantization parameters by using the information of differences between network prediction before and after quantization. In addition, PD-Quant can alleviate the overfitting problem in PTQ caused by the small number of calibration sets by adjusting the distribution of activations. Experiments show that PD-Quant leads to better quantization parameters and improves the prediction accuracy of quantized models, especially in low-bit settings. For example, PD-Quant pushes the accuracy of ResNet-18 up to 53.14% and RegNetX-600MF up to 40.67% in weight 2-bit activation 2-bit. The code is released at https://github.com/hustvl/PD-Quant.

cs.CV↗

arXiv2022

Decompositions of graphs of nonnegative characteristic with some forbidden subgraphs

A {\em $(d,h)$-decomposition} of a graph $G$ is an order pair $(D,H)$ such that $H$ is a subgraph of $G$ where $H$ has the maximum degree at most $h$ and $D$ is an acyclic orientation of $G-E(H)$ of maximum out-degree at most $d$. A graph $G$ is {\em $(d, h)$-decomposable} if $G$ has a $(d,h)$-decomposition. Let $G$ be a graph embeddable in a surface of nonnegative characteristic. In this paper, we prove the following results. (1) If $G$ has no chord $5$-cycles or no chord $6$-cycles or no chord $7$-cycles and no adjacent $4$-cycles, then $G$ is $(3,1)$-decomposable, which generalizes the results of Chen, Zhu and Wang [Comput. Math. Appl, 56 (2008) 2073--2078] and the results of Zhang [Comment. Math. Univ. Carolin, 54(3) (2013) 339--344]. (2) If $G$ has no $i$-cycles nor $j$-cycles for any subset $\{i,j\}\subseteq \{3,4,6\}$ is $(2,1)$-decomposable, which generalizes the results of Dong and Xu [Discrete Math. Alg. and Appl., 1(2) (2009), 291--297].

math.CO↗

arXiv2021

Heteroclinic Cycles of A Symmetric May-Leonard Competition Model with Seasonal Succession

In this paper, we are concerned with the stability of heteroclinic cycles of the symmetric May-Leonard competition model with seasonal succession. Sufficient conditions for stability of heteroclinic cycles are obtained. Meanwhile, we present the explicit expression of the carrying simplex in a special case. By taking ϕas the switching strategy parameter for a given example, the bifurcation of stability of the heteroclinic cycle is investigated. We also find that there are rich dynamics (including nontrivial periodic solutions, invariant closed curves and heteroclinic cycles) for such a new system via numerical simulation. Biologically, the result is interesting as a caricature of the complexities that seasonal succession can introduce into the classical May-Leonard competitive model.

math.DS↗

arXiv2021

Generic Poincaré-Bendixson Theorem for singularly perturbed monotone systems with respect to cones of rank-$2$

We investigate the singularly perturbed monotone systems with respect to cones of rank $2$ and obtain the so called Generic Poincaré-Bendixson theorem for such perturbed systems, that is, for a bounded positively invariant set, there exists an open and dense subset $\mathcal{P}$ such that for each $z\in \mathcal{P}$, the $ω$-limit set $ω(z)$ that contains no equilibrium points is a closed orbit.

math.DS↗

arXiv2018

Non-Oscillation Principle for Eventually Competitive and Cooperative Systems

A nonlinear dynamical system is called eventually competitive (or cooperative) provided that it preserves a partial order in backward (or forward) time only after some reasonable initial transient. We presented in this paper the Non-oscillation Principle for eventually competitive or cooperative systems, by which the non-ordering of (both $ω$- and $α$-) limit sets is obtained for such systems; and moreover, we established the Poincaré-Bendixson Theorem and structural stability for three-dimensional eventually competitive and cooperative systems.

math.DS↗

arXiv2016

Large-area and high-quality 2D transition metal telluride

Atomically thin transitional metal ditellurides like WTe2 and MoTe2 have triggered tremendous research interests because of their intrinsic nontrivial band structure. They are also predicted to be 2D topological insulators and type-II Weyl semimetals. However, most of the studies on ditelluride atomic layers so far rely on the low-yield and time-consuming mechanical exfoliation method. Direct synthesis of large-scale monolayer ditellurides has not yet been achieved. Here, using the chemical vapor deposition (CVD) method, we demonstrate controlled synthesis of high-quality and atom-thin tellurides with lateral size over 300 μm. We found that the as-grown WTe2 maintains two different stacking sequences in the bilayer, where the atomic structure of the stacking boundary is revealed by scanning transmission electron microscope (STEM). The low-temperature transport measurements revealed a novel semimetal-to-insulator transition in WTe2 layers and an enhanced superconductivity in few-layer MoTe2. This work paves the way to the synthesis of atom-thin tellurides and also quantum spin Hall devices.

cond-mat.mtrl-sci↗

arXiv2015

Controlled Synthesis of Organic/Inorganic van der Waals Solid for Tunable Light-matter Interactions

Van der Waals (vdW) solids, as a new type of artificial materials that consist of alternating layers bonded by weak interactions, have shed light on fascinating optoelectronic device concepts. As a result, a large variety of vdW devices have been engineered via layer-by-layer stacking of two-dimensional materials, although shadowed by the difficulties of fabrication. Alternatively, direct growth of vdW solids has proven as a scalable and swift way, highlighted by the successful synthesis of graphene/h-BN and transition metal dichalcogenides (TMDs) vertical heterostructures from controlled vapor deposition. Here, we realize high-quality organic and inorganic vdW solids, using methylammonium lead halide (CH3NH3PbI3) as the organic part (organic perovskite) and 2D inorganic monolayers as counterparts. By stacking on various 2D monolayers, the vdW solids behave dramatically different in light emission. Our studies demonstrate that h-BN monolayer is a great complement to organic perovskite for preserving its original optical properties. As a result, organic/h-BN vdW solid arrays are patterned for red light emitting. This work paves the way for designing unprecedented vdW solids with great potential for a wide spectrum of applications in optoelectronics.

physics.chem-ph↗