arXiv subjects

Guo Chen

Publications and source records attributed to Guo Chen.

At least 109 records · Page 6Linked to original sources

arXiv2024

EgoVideo: Exploring Egocentric Foundation Model and Downstream Adaptation

In this report, we present our solutions to the EgoVis Challenges in CVPR 2024, including five tracks in the Ego4D challenge and three tracks in the EPIC-Kitchens challenge. Building upon the video-language two-tower model and leveraging our meticulously organized egocentric video data, we introduce a novel foundation model called EgoVideo. This model is specifically designed to cater to the unique characteristics of egocentric videos and provides strong support for our competition submissions. In the Ego4D challenges, we tackle various tasks including Natural Language Queries, Step Grounding, Moment Queries, Short-term Object Interaction Anticipation, and Long-term Action Anticipation. In addition, we also participate in the EPIC-Kitchens challenge, where we engage in the Action Recognition, Multiple Instance Retrieval, and Domain Adaptation for Action Recognition tracks. By adapting EgoVideo to these diverse tasks, we showcase its versatility and effectiveness in different egocentric video analysis scenarios, demonstrating the powerful representation ability of EgoVideo as an egocentric foundation model. Our codebase and pretrained models are publicly available at https://github.com/OpenGVLab/EgoVideo.

cs.CV↗

arXiv2024

Realized Stable BP-N at Ambient Pressure by Phosphorus Doping

Black phosphorus nitrogen (BP-N) is an attractive high-energy-density material. However, high-pressure synthesized BP-N will decompose at low-pressure and cannot be quenched to ambient conditions. Finding a method to stabilize it at 0 GPa is of great significance for its practical applications. However, unlike cg-N, LP-N, and HLP-N, it is always a metastable phase at high-pressure up to 260 GPa, and decomposes into chains at 23 GPa. Here, based on the first-principles simulations, we find that P atom doping can effectively reduce the synthesis pressure of BP-N and maintain its stability at 0 GPa. Uniform distribution of P atom dopants within the layer helps maintain the structural stability of BP-N layer at 0 GPa, while interlayer electrostatic interaction induced by N-P dipoles enhances its dynamic stability by eliminating interlayer slipping. Furthermore, pressure is conducive to enhancing the stability of BP-N and its doped forms by suppressing N-chain dissociation. For the configuration with 12.5% doping concentration, a gravimetric energy density of 8.07 kJ/g can be realized, which is nearly two times higher than TNT.

cond-mat.mtrl-sci↗

arXiv2024

Retrieval-Augmented Egocentric Video Captioning

Understanding human actions from videos of first-person view poses significant challenges. Most prior approaches explore representation learning on egocentric videos only, while overlooking the potential benefit of exploiting existing large-scale third-person videos. In this paper, (1) we develop EgoInstructor, a retrieval-augmented multimodal captioning model that automatically retrieves semantically relevant third-person instructional videos to enhance the video captioning of egocentric videos. (2) For training the cross-view retrieval module, we devise an automatic pipeline to discover ego-exo video pairs from distinct large-scale egocentric and exocentric datasets. (3) We train the cross-view retrieval module with a novel EgoExoNCE loss that pulls egocentric and exocentric video features closer by aligning them to shared text features that describe similar actions. (4) Through extensive experiments, our cross-view retrieval module demonstrates superior performance across seven benchmarks. Regarding egocentric video captioning, EgoInstructor exhibits significant improvements by leveraging third-person videos as references. Project page is available at: https://jazzcharles.github.io/Egoinstructor/

cs.CV↗

arXiv2024

RH-SQL: Refined Schema and Hardness Prompt for Text-to-SQL

Text-to-SQL is a technology that converts natural language queries into the structured query language SQL. A novel research approach that has recently gained attention focuses on methods based on the complexity of SQL queries, achieving notable performance improvements. However, existing methods entail significant storage and training costs, which hampers their practical application. To address this issue, this paper introduces a method for Text-to-SQL based on Refined Schema and Hardness Prompt. By filtering out low-relevance schema information with a refined schema and identifying query hardness through a Language Model (LM) to form prompts, this method reduces storage and training costs while maintaining performance. It's worth mentioning that this method is applicable to any sequence-to-sequence (seq2seq) LM. Our experiments on the Spider dataset, specifically with large-scale LMs, achieved an exceptional Execution accuracy (EX) of 82.6%, demonstrating the effectiveness and greater suitability of our method for real-world applications.

cs.CL↗

arXiv2024

Acceleration of the Universe without the Hubble tension with Kaniadakis holographic dark energy using the Hubble horizon as the IR cut-off

We introduce a holographic dark energy model that incorporates the first-order approximate Kaniadaski entropy, utilizing the Hubble horizon, $1/H$, as the infrared cutoff. We investigate the cosmological evolution within this framework. The model introduces an extra parameter relative to the $Λ$CDM model. It posits a Universe that is initially dominated by dark matter, which then evolves to a phase where dark energy becomes the predominant component, with this transition occurring at a redshift of approximately $z \sim 0.419$. The energy density of dark energy is ultimately expected to become constant, thereby circumventing the potential issue of a "big rip". Employing the most recent Type Ia supernova and Hubble parameter data, we constrain the model's parameters and find a Hubble constant of $H_0=72.8$ km/s/Mpc, thereby resolving the Hubble tension issue. The estimated age of the Universe, based on the best-fit parameter values, is $14.2$ Gyr. Furthermore, we predict the number of strong gravitational lenses and conduct statefinder and $Om$ diagnostic analyses to validate and characterize the model.

astro-ph.CO↗

arXiv2024

Large Language Model Assisted Optimal Bidding of BESS in FCAS Market: An AI-agent based Approach

To incentivize flexible resources such as Battery Energy Storage Systems (BESSs) to offer Frequency Control Ancillary Services (FCAS), Australia's National Electricity Market (NEM) has implemented changes in recent years towards shorter-term bidding rules and faster service requirements. However, firstly, existing bidding optimization methods often overlook or oversimplify the key aspects of FCAS market procedures, resulting in an inaccurate depiction of the market bidding process. Thus, the BESS bidding problem is modeled based on the actual bidding records and the latest market specifications and then formulated as a deep reinforcement learning (DRL) problem. Secondly, the erratic decisions of the DRL agent caused by imperfectly predicted market information increases the risk of profit loss. Hence, a Conditional Value at Risk (CVaR)-based DRL algorithm is developed to enhance the risk resilience of bidding strategies. Thirdly, well-trained DRL models still face performance decline in uncommon scenarios during online operations. Therefore, a Large Language Models (LLMs)-assisted artificial intelligence (AI)-agent interactive decision-making framework is proposed to improve the strategy timeliness, reliability and interpretability in uncertain new scenarios, where conditional hybrid decision and self-reflection mechanisms are designed to address LLMs' hallucination challenge. The experiment results demonstrate that our proposed framework has higher bidding profitability compared to the baseline methods by effectively mitigating the profit loss caused by various uncertainties.

eess.SY↗

arXiv2024

Simultaneous multicolour transit photometry of hot Jupiters HAT-P-19b, HAT-P-51b, HAT-P-55b, and HAT-P-65b

Accurate physical parameters of exoplanet systems are essential for further exploration of planetary internal structure, atmospheres, and formation history. We aim to use simultaneous multicolour transit photometry to improve the estimation of transit parameters, to search for transit timing variations (TTVs), and to establish which of our targets should be prioritised for follow-up transmission spectroscopy. We performed time series photometric observations of 12 transits for the hot Jupiters HAT-P-19b, HAT-P-51b, HAT-P-55b, and HAT-P-65b using the simultaneous four-colour camera MuSCAT2 on the Telescopio Carlos Sánchez. We collected 56 additional transit light curves from TESS photometry. To derive transit parameters, we modelled the MuSCAT2 light curves with Gaussian processes to account for correlated noise. To derive physical parameters, we performed EXOFASTv2 global fits to the available transit and radial velocity data sets, together with the Gaia DR3 parallax, isochrones, and spectral energy distributions. To assess the potential for atmospheric characterisation, we compared the multicolour transit depths with a flat line and a clear atmosphere model. We consistently refined the transit and physical parameters. We improved the orbital period and ephemeris estimates, and found no evidence for TTVs or orbital decay. The MuSCAT2 broadband transmission spectra of HAT-P-19b and HAT-P-65b are consistent with previously published low-resolution transmission spectra. We also found that, except for HAT-P-65b, the assumption of a planetary atmosphere can improve the fit to the MuSCAT2 data. In particular, we identified HAT-P-55b as a priority target among these four planets for further atmospheric studies using transmission spectroscopy.

astro-ph.EP↗

arXiv2024

MVBench: A Comprehensive Multi-modal Video Understanding Benchmark

With the rapid development of Multi-modal Large Language Models (MLLMs), a number of diagnostic benchmarks have recently emerged to evaluate the comprehension capabilities of these models. However, most benchmarks predominantly assess spatial understanding in the static image tasks, while overlooking temporal understanding in the dynamic video tasks. To alleviate this issue, we introduce a comprehensive Multi-modal Video understanding Benchmark, namely MVBench, which covers 20 challenging video tasks that cannot be effectively solved with a single frame. Specifically, we first introduce a novel static-to-dynamic method to define these temporal-related tasks. By transforming various static tasks into dynamic ones, we enable the systematic generation of video tasks that require a broad spectrum of temporal skills, ranging from perception to cognition. Then, guided by the task definition, we automatically convert public video annotations into multiple-choice QA to evaluate each task. On one hand, such a distinct paradigm allows us to build MVBench efficiently, without much manual intervention. On the other hand, it guarantees evaluation fairness with ground-truth video annotations, avoiding the biased scoring of LLMs. Moreover, we further develop a robust video MLLM baseline, i.e., VideoChat2, by progressive multi-modal training with diverse instruction-tuning data. The extensive results on our MVBench reveal that, the existing MLLMs are far from satisfactory in temporal understanding, while our VideoChat2 largely surpasses these leading models by over 15% on MVBench. All models and data are available at https://github.com/OpenGVLab/Ask-Anything.

cs.CV↗

arXiv2024

Detection of Na in the atmosphere of the hot Jupiter HAT-P-55b

The spectral signatures of optical absorbers, when combined with those of infrared molecules, play a critical role in constraining the cloud properties of exoplanet atmospheres. We aim to use optical transmission spectroscopy to confirm the tentative color signature previously observed by multiband photometry in the atmosphere of hot Jupiter HAT-P-55b. We observed a transit of HAT-P-55b with the OSIRIS spectrograph on the Gran Telescopio Canarias (GTC). We created two sets of spectroscopic light curves using the conventional band-integrated method and the newly proposed pixel-based method to derive the transmission spectrum. We performed Bayesian spectral retrieval analyses on the transmission spectrum to interpret the observed atmospheric properties. The transmission spectra derived from the two methods are consistent, both spectrally resolving the tentative color signature observed by MuSCAT2. The retrievals on the combined OSIRIS and MuSCAT2 transmission spectrum yield the detection of Na at 5.5$σ$ and the tentative detection of MgH at 3.4$σ$. The current optical-only wavelength coverage cannot constrain the absolute abundances of the atmospheric species. Space-based observations covering the molecular infrared bands or ground-based high-resolution spectroscopy are needed to further constrain the atmospheric properties of HAT-P-55b.

astro-ph.EP↗

arXiv2024

Closeby Habitable Exoplanet Survey (CHES). I. Astrometric Noise and Planetary Detection Efficiency due to Stellar Spots and Faculae

The Closeby Habitable Exoplanet Survey (CHES) is dedicated to the astrometric exploration for habitable-zone Earth-like planets orbiting solar-type stars in close proximity, achieving unprecedented micro-arcsecond precision. Given the elevated precision, thorough consideration of photocenter jitters induced by stellar activity becomes imperative. This study endeavors to model the stellar activity of solar-type stars, compute astrometric noise, and delineate the detection limits of habitable planets within the astrometric domain. Simulations were conducted for identified primary targets of CHES, involving the generation of simulated observed data for astrometry and photometry, accounting for the impact of stellar activity. Estimation of activity levels in our samples was achieved through chromospheric activity indices, revealing that over 90% of stars exhibited photocenter jitters below 1 $μ\mathrm{as}$. Notably, certain proximate stars, such as $α$ Cen A and B, displayed more discernible noise arising from stellar activity. Subsequent tests were performed to evaluate detection performance, unveiling that stellar activity tends to have a less pronounced impact on planetary detectability for the majority of stars. Approximately 95% of targets demonstrated a detection efficiency exceeding 80%. However, for several cold stars, e.g., HD 32450 and HD 21531, with the habitable zones close to the stars, a reduction in detection efficiency was observed. These findings offer invaluable insights into the intricate interplay between stellar activity and astrometric precision, significantly advancing our understanding in the search for habitable planets.

astro-ph.EP↗

arXiv2024

Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understanding

Understanding videos is one of the fundamental directions in computer vision research, with extensive efforts dedicated to exploring various architectures such as RNN, 3D CNN, and Transformers. The newly proposed architecture of state space model, e.g., Mamba, shows promising traits to extend its success in long sequence modeling to video modeling. To assess whether Mamba can be a viable alternative to Transformers in the video understanding domain, in this work, we conduct a comprehensive set of studies, probing different roles Mamba can play in modeling videos, while investigating diverse tasks where Mamba could exhibit superiority. We categorize Mamba into four roles for modeling videos, deriving a Video Mamba Suite composed of 14 models/modules, and evaluating them on 12 video understanding tasks. Our extensive experiments reveal the strong potential of Mamba on both video-only and video-language tasks while showing promising efficiency-performance trade-offs. We hope this work could provide valuable data points and insights for future research on video understanding. Code is public: https://github.com/OpenGVLab/video-mamba-suite.

cs.CV↗

arXiv2024

Free-standing cubic gauche nitrogen stable at 760 K under ambient pressure

Cubic gauche nitrogen (cg-N) has received wide attention due to its high energy density and environmental friendliness. However, existing synthesis methods for cg-N predominantly rely on the high-pressure techniques, or the utilization of nanoconfined effects using highly toxic and sensitive sodium azide as precursor, which significantly restrict the practical application of cg-N as high energy density materials (HDEM). Here, based on the first-principles simulations, we find that the adsorption of potassium on the cg-N surface exhibits superior stabilization compared to sodium. Then, we chose the safer potassium azide as raw material for synthesizing cg-N. Through plasma-enhanced chemical vapor deposition treatment, the free-standing cg-N was successfully synthesized without the need of high-pressure and nanoconfined effects. Importantly, it demonstrates excellent thermal stability up to 760 K, and then a rapid and intense thermal decomposition occurs, exhibiting typical behaviors of HDEM thermal decomposition. Our work has significantly promoted the practical application of cg-N as HDEM.

cond-mat.mtrl-sci↗

arXiv2024

TOI-2266 b: a keystone super-Earth at the edge of the M dwarf radius valley

We validate the Transiting Exoplanet Survey Satellite (TESS) object of interest TOI-2266.01 (TIC 348911) as a small transiting planet (most likely a super-Earth) orbiting a faint M5 dwarf ($V=16.54$) on a 2.33~d orbit. The validation is based on an approach where multicolour transit light curves are used to robustly estimate the upper limit of the transiting object's radius. Our analysis uses SPOC-pipeline TESS light curves from Sectors 24, 25, 51, and 52, simultaneous multicolour transit photometry observed with MuSCAT2, MuSCAT3, and HiPERCAM, and additional transit photometry observed with the LCOGT telescopes. TOI-2266 b is found to be a planet with a radius of $1.54\pm\0.09\,R_\oplus$, which locates it at the edge of the transition zone between rocky planets, water-rich planets, and sub-Neptunes (the so-called M~dwarf radius valley). The planet is amenable to ground-based radial velocity mass measurement with red-sensitive spectrographs installed in large telescopes, such as MAROON-X and Keck Planet Finder (KPF), which makes it a valuable addition to a relatively small population of planets that can be used to probe the physics of the transition zone. Further, the planet's orbital period of 2.33 days places it inside a `keystone planet' wedge in the period-radius plane where competing planet formation scenarios make conflicting predictions on how the radius valley depends on the orbital period. This makes the planet also a welcome addition to the small population of planets that can be used to test small-planet formation scenarios around M~dwarfs.

astro-ph.EP↗

arXiv2024

InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks

The exponential growth of large language models (LLMs) has opened up numerous possibilities for multimodal AGI systems. However, the progress in vision and vision-language foundation models, which are also critical elements of multi-modal AGI, has not kept pace with LLMs. In this work, we design a large-scale vision-language foundation model (InternVL), which scales up the vision foundation model to 6 billion parameters and progressively aligns it with the LLM, using web-scale image-text data from various sources. This model can be broadly applied to and achieve state-of-the-art performance on 32 generic visual-linguistic benchmarks including visual perception tasks such as image-level or pixel-level recognition, vision-language tasks such as zero-shot image/video classification, zero-shot image/video-text retrieval, and link with LLMs to create multi-modal dialogue systems. It has powerful visual capabilities and can be a good alternative to the ViT-22B. We hope that our research could contribute to the development of multi-modal large models. Code and models are available at https://github.com/OpenGVLab/InternVL.

cs.CV↗

arXiv2024

InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation

This paper introduces InternVid, a large-scale video-centric multimodal dataset that enables learning powerful and transferable video-text representations for multimodal understanding and generation. The InternVid dataset contains over 7 million videos lasting nearly 760K hours, yielding 234M video clips accompanied by detailed descriptions of total 4.1B words. Our core contribution is to develop a scalable approach to autonomously build a high-quality video-text dataset with large language models (LLM), thereby showcasing its efficacy in learning video-language representation at scale. Specifically, we utilize a multi-scale approach to generate video-related descriptions. Furthermore, we introduce ViCLIP, a video-text representation learning model based on ViT-L. Learned on InternVid via contrastive learning, this model demonstrates leading zero-shot action recognition and competitive video retrieval performance. Beyond basic video understanding tasks like recognition and retrieval, our dataset and model have broad applications. They are particularly beneficial for generating interleaved video-text data for learning a video-centric dialogue system, advancing video-to-text and text-to-video generation research. These proposed resources provide a tool for researchers and practitioners interested in multimodal video understanding and generation.

cs.CV↗

arXiv2023

Decoupling SQL Query Hardness Parsing for Text-to-SQL

The fundamental goal of the Text-to-SQL task is to translate natural language question into SQL query. Current research primarily emphasizes the information coupling between natural language questions and schemas, and significant progress has been made in this area. The natural language questions as the primary task requirements source determines the hardness of correspond SQL queries, the correlation between the two always be ignored. However, when the correlation between questions and queries was decoupled, it may simplify the task. In this paper, we introduce an innovative framework for Text-to-SQL based on decoupling SQL query hardness parsing. This framework decouples the Text-to-SQL task based on query hardness by analyzing questions and schemas, simplifying the multi-hardness task into a single-hardness challenge. This greatly reduces the parsing pressure on the language model. We evaluate our proposed framework and achieve a new state-of-the-art performance of fine-turning methods on Spider dev.

cs.CL↗

arXiv2023

AVSegFormer: Audio-Visual Segmentation with Transformer

The combination of audio and vision has long been a topic of interest in the multi-modal community. Recently, a new audio-visual segmentation (AVS) task has been introduced, aiming to locate and segment the sounding objects in a given video. This task demands audio-driven pixel-level scene understanding for the first time, posing significant challenges. In this paper, we propose AVSegFormer, a novel framework for AVS tasks that leverages the transformer architecture. Specifically, we introduce audio queries and learnable queries into the transformer decoder, enabling the network to selectively attend to interested visual features. Besides, we present an audio-visual mixer, which can dynamically adjust visual features by amplifying relevant and suppressing irrelevant spatial channels. Additionally, we devise an intermediate mask loss to enhance the supervision of the decoder, encouraging the network to produce more accurate intermediate predictions. Extensive experiments demonstrate that AVSegFormer achieves state-of-the-art results on the AVS benchmark. The code is available at https://github.com/vvvb-github/AVSegFormer.

cs.CV↗

arXiv2023

High-resolution transmission spectroscopy of ultra-hot Jupiter WASP-33b with NEID

We report an attempt to detect molecular and atomic species in the atmosphere of the ultra-hot Jupiter WASP-33b using the high-resolution echelle spectrograph NEID with a wavelength coverage of 380$-$930 nm. By analyzing the transmission spectrum of WASP-33b using the line-by-line technique and the cross-correlation technique, we confirm previous detection of H$α$, H$β$, H$γ$, and Ca II infrared triplets. We find no evidence for a significant day-to-night wind in WASP-33b, taking into account the effects of stellar pulsations using a relatively novel GP method and poorly constrained systemic velocity measurements. We also detect the previously reported pre-transit absorption signal, which may be a pulsation mode induced by the planet. Combined with previous CARMENES and HARPS-N observations, we report the non-detection of TiO, Ti I, and V I in the transmission spectrum, while they were already detected in the dayside atmosphere of WASP-33b. This implies a difference in the chemical compositions and abundances between the dayside and terminator atmospheres of WASP-33b, and certainly requires further improvements in the sensitivity of the detection methods.

astro-ph.EP↗