arXiv subjects

Xun Zou

Publications and source records attributed to Xun Zou.

13 recordsLinked to original sources

arXiv2025

MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention

We introduce MiniMax-M1, the world's first open-weight, large-scale hybrid-attention reasoning model. MiniMax-M1 is powered by a hybrid Mixture-of-Experts (MoE) architecture combined with a lightning attention mechanism. The model is developed based on our previous MiniMax-Text-01 model, which contains a total of 456 billion parameters with 45.9 billion parameters activated per token. The M1 model natively supports a context length of 1 million tokens, 8x the context size of DeepSeek R1. Furthermore, the lightning attention mechanism in MiniMax-M1 enables efficient scaling of test-time compute. These properties make M1 particularly suitable for complex tasks that require processing long inputs and thinking extensively. MiniMax-M1 is trained using large-scale reinforcement learning (RL) on diverse problems including sandbox-based, real-world software engineering environments. In addition to M1's inherent efficiency advantage for RL training, we propose CISPO, a novel RL algorithm to further enhance RL efficiency. CISPO clips importance sampling weights rather than token updates, outperforming other competitive RL variants. Combining hybrid-attention and CISPO enables MiniMax-M1's full RL training on 512 H800 GPUs to complete in only three weeks, with a rental cost of just $534,700. We release two versions of MiniMax-M1 models with 40K and 80K thinking budgets respectively, where the 40K model represents an intermediate phase of the 80K training. Experiments on standard benchmarks show that our models are comparable or superior to strong open-weight models such as the original DeepSeek-R1 and Qwen3-235B, with particular strengths in complex software engineering, tool utilization, and long-context tasks. We publicly release MiniMax-M1 at https://github.com/MiniMax-AI/MiniMax-M1.

cs.CL↗

arXiv2025

MiniMax-01: Scaling Foundation Models with Lightning Attention

We introduce MiniMax-01 series, including MiniMax-Text-01 and MiniMax-VL-01, which are comparable to top-tier models while offering superior capabilities in processing longer contexts. The core lies in lightning attention and its efficient scaling. To maximize computational capacity, we integrate it with Mixture of Experts (MoE), creating a model with 32 experts and 456 billion total parameters, of which 45.9 billion are activated for each token. We develop an optimized parallel strategy and highly efficient computation-communication overlap techniques for MoE and lightning attention. This approach enables us to conduct efficient training and inference on models with hundreds of billions of parameters across contexts spanning millions of tokens. The context window of MiniMax-Text-01 can reach up to 1 million tokens during training and extrapolate to 4 million tokens during inference at an affordable cost. Our vision-language model, MiniMax-VL-01 is built through continued training with 512 billion vision-language tokens. Experiments on both standard and in-house benchmarks show that our models match the performance of state-of-the-art models like GPT-4o and Claude-3.5-Sonnet while offering 20-32 times longer context window. We publicly release MiniMax-01 at https://github.com/MiniMax-AI.

cs.CL↗

arXiv2023

Self-adjusting Population Sizes for the $(1, λ)$-EA on Monotone Functions

We study the $(1,λ)$-EA with mutation rate $c/n$ for $c\le 1$, where the population size is adaptively controlled with the $(1:s+1)$-success rule. Recently, Hevia Fajardo and Sudholt have shown that this setup with $c=1$ is efficient on \onemax for $s<1$, but inefficient if $s \ge 18$. Surprisingly, the hardest part is not close to the optimum, but rather at linear distance. We show that this behavior is not specific to \onemax. If $s$ is small, then the algorithm is efficient on all monotone functions, and if $s$ is large, then it needs superpolynomial time on all monotone functions. In the former case, for $c<1$ we show a $O(n)$ upper bound for the number of generations and $O(n\log n)$ for the number of function evaluations, and for $c=1$ we show $O(n\log n)$ generations and $O(n^2\log\log n)$ evaluations. We also show formally that optimization is always fast, regardless of $s$, if the algorithm starts in proximity of the optimum. All results also hold in a dynamic environment where the fitness function changes in each generation.

cs.NE↗

arXiv2022

OneMax is not the Easiest Function for Fitness Improvements

We study the $(1:s+1)$ success rule for controlling the population size of the $(1,\lambda)$-EA. It was shown by Hevia Fajardo and Sudholt that this parameter control mechanism can run into problems for large $s$ if the fitness landscape is too easy. They conjectured that this problem is worst for the OneMax benchmark, since in some well-established sense OneMax is known to be the easiest fitness landscape. In this paper we disprove this conjecture and show that OneMax is not the easiest fitness landscape with respect to finding improving steps. As a consequence, we show that there exists $s$ and $\varepsilon$ such that the self-adjusting $(1,\lambda)$-EA with $(1:s+1)$-rule optimizes OneMax efficiently when started with $\varepsilon n$ zero-bits, but does not find the optimum in polynomial time on Dynamic BinVal. Hence, we show that there are landscapes where the problem of the $(1:s+1)$-rule for controlling the population size of the $(1, \lambda)$-EA is more severe than for OneMax.

cs.NE↗

arXiv2021

Molecular modeling with machine-learned universal potential functions

Molecular modeling is an important topic in drug discovery. Decades of research have led to the development of high quality scalable molecular force fields. In this paper, we show that neural networks can be used to train a universal approximator for energy potential functions. By incorporating a fully automated training process we have been able to train smooth, differentiable, and predictive potential functions on large-scale crystal structures. A variety of tests have also been performed to show the superiority and versatility of the machine-learned model.

q-bio.QM↗

arXiv2021

Exponential Slowdown for Larger Populations: The $(μ+1)$-EA on Monotone Functions

Pseudo-Boolean monotone functions are unimodal functions which are trivial to optimize for some hillclimbers, but are challenging for a surprising number of evolutionary algorithms (EAs). A general trend is that EAs are efficient if parameters like the mutation rate are set conservatively, but may need exponential time otherwise. In particular, it was known that the $(1+1)$-EA and the $(1+λ)$-EA can optimize every monotone function in pseudolinear time if the mutation rate is $c/n$ for some $c<1$, but they need exponential time for some monotone functions for $c>2.2$. The second part of the statement was also known for the $(μ+1)$-EA. In this paper we show that the first statement does not apply to the $(μ+1)$-EA. More precisely, we prove that for every constant $c>0$ there is a constant integer $μ_0$ such that the $(μ+1)$-EA with mutation rate $c/n$ and population size $μ_0\leμ\le n$ needs superpolynomial time to optimize some monotone functions. Thus, increasing the population size by just a constant has devastating effects on the performance. This is in stark contrast to many other benchmark functions on which increasing the population size either increases the performance significantly, or affects performance mildly. The reason why larger populations are harmful lies in the fact that larger populations may temporarily decrease selective pressure on parts of the population. This allows unfavorable mutations to accumulate in single individuals and their descendants. If the population moves sufficiently fast through the search space, such unfavorable descendants can become ancestors of future generations, and the bad mutations are preserved. Remarkably, this effect only occurs if the population renews itself sufficiently fast, which can only happen far away from the optimum. This is counter-intuitive since usually optimization gets harder as we approach the optimum.

cs.NE↗

arXiv2020

Downlink Asynchronous Non-Orthogonal Multiple Access with Quantizer Optimization

In this letter, we study a two-user downlink asynchronous non-orthogonal multiple access (ANOMA) with limited feedback. We employ the max-min criterion for the power allocation and derive the closed-form expressions for the upper and lower bounds of the max-min rate. It is demonstrated that ANOMA can achieve the same or even higher average maxmin rate with a lower feedback rate compared with NOMA. Moreover, we propose a quantizer optimization algorithm which applies to both NOMA and ANOMA. Simulation results show that the optimized quantizer significantly improves the average max-min rate compared with the conventional uniform quantizer, especially in the scenario with a low feedback rate.

cs.IT↗

arXiv2019

Trellis-Coded Non-Orthogonal Multiple Access

In this letter, we propose a trellis-coded nonorthogonal multiple access (NOMA) scheme. The signals for different users are produced by trellis coded modulation (TCM) and then superimposed on different power levels. By interpreting the encoding process via the tensor product of trellises, we introduce a joint detection method based on the Viterbi algorithm. Then, we determine the optimal power allocation between the two users by maximizing the free distance of the tensor product trellis. Finally, we manifest that the trellis-coded NOMA outperforms the uncoded NOMA at high signal-to-noise ratio (SNR).

cs.IT↗

arXiv2019

Cooperative Asynchronous Non-Orthogonal Multiple Access with Power Minimization Under QoS Constraints

Recent studies have demonstrated the superiority of non-orthogonal multiple access (NOMA) over orthogonal multiple access (OMA) in cooperative communication networks. In this paper, we propose a novel half-duplex cooperative asynchronous NOMA (C-ANOMA) framework with user relaying, where a timing mismatch is intentionally added in the broadcast signal. We derive the expressions for the individual throughputs of the strong user (acts as relay) which employs the block-wise successive interference cancellation (SIC) and the weak user which combines the symbol-asynchronous signal with the interference-free signal. We analytically prove that in the C-ANOMA systems with a sufficiently large frame length, the strong user attains the same throughput to decode its own message while both users can achieve a higher throughput to decode the weak user's message compared with those in the cooperative NOMA (C-NOMA) systems. Besides, we obtain the optimal timing mismatch when the frame length goes to infinity. Furthermore, to exploit the trade-off between the power consumption of base station and that of the relay user, we solve a weighted sum power minimization problem under quality of services (QoS) constraints. Numerical results show that the C-ANOMA system can consume less power compared with the C-NOMA system to satisfy the same QoS requirements.

cs.IT↗

arXiv2019

An Analysis of Two-User Uplink Asynchronous Non-Orthogonal Multiple Access Systems

Recent studies have numerically demonstrated the possible advantages of the asynchronous non-orthogonal multiple access (ANOMA) over the conventional synchronous non-orthogonal multiple access (NOMA). The ANOMA makes use of the oversampling technique by intentionally introducing a timing mismatch between symbols of different users. Focusing on a two-user uplink system, for the first time, we analytically prove that the ANOMA with a sufficiently large frame length can always outperform the NOMA in terms of the sum throughput. To this end, we derive the expression for the sum throughput of the ANOMA as a function of signal-to-noise ratio (SNR), frame length, and normalized timing mismatch. Based on the derived expression, we find that users should transmit at full powers to maximize the sum throughput. In addition, we obtain the optimal timing mismatch as the frame length goes to infinity. Moreover, we comprehensively study the impact of timing error on the ANOMA throughput performance. Two types of timing error, i.e., the synchronization timing error and the coordination timing error, are considered. We derive the throughput loss incurred by both types of timing error and find that the synchronization timing error has a greater impact on the throughput performance compared to the coordination timing error.

cs.IT↗

arXiv2018

Interleaving Channel Estimation and Limited Feedback for Point-to-Point Systems with a Large Number of Transmit Antennas

We introduce and investigate the opportunities of multi-antenna communication schemes whose training and feedback stages are interleaved and mutually interacting. Specifically, unlike the traditional schemes where the transmitter first trains all of its antennas at once and then receives a single feedback message, we consider a scenario where the transmitter instead trains its antennas one by one and receives feedback information immediately after training each one of its antennas. The feedback message may ask the transmitter to train another antenna; or, it may terminate the feedback/training phase and provide the quantized codeword (e.g., a beamforming vector) to be utilized for data transmission. As a specific application, we consider a multiple-input single-output system with $t$ transmit antennas, a short-term power constraint $P$, and target data rate $ρ$. We show that for any $t$, the same outage probability as a system with perfect transmitter and receiver channel state information can be achieved with a feedback rate of $R_1$ bits per channel state and via training $R_2$ transmit antennas on average, where $R_1$ and $R_2$ are independent of $t$, and depend only on $ρ$ and $P$. In addition, we design variable-rate quantizers for channel coefficients to further minimize the feedback rate of our scheme.

cs.IT↗

arXiv2017

Asynchronous Channel Training in Multi-Cell Massive MIMO

Pilot contamination has been regarded as the main bottleneck in time division duplexing (TDD) multi-cell massive multiple-input multiple-output (MIMO) systems. The pilot contamination problem cannot be addressed with large-scale antenna arrays. We provide a novel asynchronous channel training scheme to obtain precise channel matrices without the cooperation of base stations. The scheme takes advantage of sampling diversity by inducing intentional timing mismatch. Then, the linear minimum mean square error (LMMSE) estimator and the zero-forcing (ZF) estimator are designed. Moreover, we derive the minimum square error (MSE) upper bound of the ZF estimator. In addition, we propose the equally-divided delay scheme which under certain conditions is the optimal solution to minimize the MSE of the ZF estimator employing the identity matrix as pilot matrix. We calculate the uplink achievable rate using maximum ratio combining (MRC) to compare asynchronous and synchronous channel training schemes. Finally, simulation results demonstrate that the asynchronous channel estimation scheme can greatly reduce the harmful effect of pilot contamination.

eess.SP↗

arXiv2017

Debugging Transactions and Tracking their Provenance with Reenactment

Debugging transactions and understanding their execution are of immense importance for developing OLAP applications, to trace causes of errors in production systems, and to audit the operations of a database. However, debugging transactions is hard for several reasons: 1) after the execution of a transaction, its input is no longer available for debugging, 2) internal states of a transaction are typically not accessible, and 3) the execution of a transaction may be affected by concurrently running transactions. We present a debugger for transactions that enables non-invasive, post-mortem debugging of transactions with provenance tracking and supports what-if scenarios (changes to transaction code or data). Using reenactment, a declarative replay technique we have developed, a transaction is replayed over the state of the DB seen by its original execution including all its interactions with concurrently executed transactions from the history. Importantly, our approach uses the temporal database and audit logging capabilities available in many DBMS and does not require any modifications to the underlying database system nor transactional workload.

cs.DB↗