arXiv subjects

Omar Florez

Publications and source records attributed to Omar Florez.

6 recordsLinked to original sources

arXiv2024

Alt-Text with Context: Improving Accessibility for Images on Twitter

In this work we present an approach for generating alternative text (or alt-text) descriptions for images shared on social media, specifically Twitter. More than just a special case of image captioning, alt-text is both more literally descriptive and context-specific. Also critically, images posted to Twitter are often accompanied by user-written text that despite not necessarily describing the image may provide useful context that if properly leveraged can be informative. We address this task with a multimodal model that conditions on both textual information from the associated social media post as well as visual signal from the image, and demonstrate that the utility of these two information sources stacks. We put forward a new dataset of 371k images paired with alt-text and tweets scraped from Twitter and evaluate on it across a variety of automated metrics as well as human evaluation. We show that our approach of conditioning on both tweet text and visual information significantly outperforms prior work, by more than 2x on BLEU@4.

cs.CV↗

arXiv2023

TwHIN-BERT: A Socially-Enriched Pre-trained Language Model for Multilingual Tweet Representations at Twitter

Pre-trained language models (PLMs) are fundamental for natural language processing applications. Most existing PLMs are not tailored to the noisy user-generated text on social media, and the pre-training does not factor in the valuable social engagement logs available in a social network. We present TwHIN-BERT, a multilingual language model productionized at Twitter, trained on in-domain data from the popular social network. TwHIN-BERT differs from prior pre-trained language models as it is trained with not only text-based self-supervision, but also with a social objective based on the rich social engagements within a Twitter heterogeneous information network (TwHIN). Our model is trained on 7 billion tweets covering over 100 distinct languages, providing a valuable representation to model short, noisy, user-generated text. We evaluate our model on various multilingual social recommendation and semantic understanding tasks and demonstrate significant metric improvement over established pre-trained language models. We open-source TwHIN-BERT and our curated hashtag prediction and social engagement benchmark datasets to the research community.

cs.CL↗

arXiv2023

Non-Parametric Temporal Adaptation for Social Media Topic Classification

User-generated social media data is constantly changing as new trends influence online discussion and personal information is deleted due to privacy concerns. However, most current NLP models are static and rely on fixed training data, which means they are unable to adapt to temporal change -- both test distribution shift and deleted training data -- without frequent, costly re-training. In this paper, we study temporal adaptation through the task of longitudinal hashtag prediction and propose a non-parametric dense retrieval technique, which does not require re-training, as a simple but effective solution. In experiments on a newly collected, publicly available, year-long Twitter dataset exhibiting temporal distribution shift, our method improves by 64.12% over the best parametric baseline without any of its costly gradient-based updating. Our dense retrieval approach is also particularly well-suited to dynamically deleted user data in line with data privacy laws, with negligible computational cost and performance loss.

cs.CL↗

arXiv2022

Optomechanical generation of coherent GHz vibrations in a phononic waveguide

Nanophononics has the potential for information transfer, in an analogous manner to its photonic and electronic counterparts. The adoption of phononic systems has been limited, due to difficulties associated with the generation, manipulation, and detection of phonons, especially at GHz frequencies. Existing techniques often require piezoelectric materials with an external radiofrequency excitation that are not readily integrated into existing CMOS infrastructures, while non-piezoelectric demonstrations have been inefficient. In this work, we explore the optomechanical generation of coherent phonons in a suspended 2D silicon phononic crystal cavity with a guided mode around 6.8 GHz. By incorporating an air-slot into this cavity, we turn the phononic waveguide into an optomechanical platform that exploits localized photonic modes resulting from inherent fabrication imperfections for the transduction of mechanics. Such a platform exhibits very fine control of phonons using light, and is capable of coherent self-sustained phonon generation via mechanical lasing around 6.8 GHz. The ability to generate high frequency coherent mechanical vibrations within such a simple 2D CMOS-compatible system could be a first step towards the development of sources in phononic circuitry and the coherent manipulation of other solid-state properties.

physics.optics↗

arXiv2022

Excitation and detection of acoustic phonons in nanoscale systems

Phonons play a key role in the physical properties of materials, and have long been a topic of study in physics. While the effects of phonons had historically been considered to be a hindrance, modern research has shown that phonons can be exploited due to their ability to couple to other excitations and consequently affect the thermal, dielectric, and electronic properties of solid state systems, greatly motivating the engineering of phononic structures. Advances in nanofabrication have allowed for structuring and phonon confinement even down to the nanoscale, drastically changing material properties. Despite developments in fabricating such nanoscale devices, the proper manipulation and characterization of phonons continues to be challenging. However, a fundamental understanding of these processes could enable the realization of key applications in diverse fields such as topological phononics, information technologies, sensing, and quantum electrodynamics, especially when integrated with existing electronic and photonic devices. Here, we highlight seven of the available methods for the excitation and detection of acoustic phonons and vibrations in solid materials, as well as advantages, disadvantages, and additional considerations related to their application. We then provide perspectives towards open challenges in nanophononics and how the additional understanding granted by these techniques could serve to enable the next generation of phononic technological applications.

cond-mat.mtrl-sci↗

arXiv2016

Brillouin Scattering Self-Cancellation

The interaction between light and acoustic phonons is strongly modified in sub-wavelength confinement, and has led to the demonstration and control of Brillouin scattering in photonic structures such as nano-scale optical waveguides and cavities. Besides the small optical mode volume, two physical mechanisms come into play simultaneously: a volume effect caused by the strain induced refractive index perturbation (known as photo-elasticity), and a surface effect caused by the shift of the optical boundaries due to mechanical vibrations. As a result proper material and structure engineering allows one to control each contribution individually. In this paper, we experimentally demonstrate the perfect cancellation of Brillouin scattering by engineering a silica nanowire with exactly opposing photo-elastic and moving-boundary effects. This demonstration provides clear experimental evidence that the interplay between the two mechanisms is a promising tool to precisely control the photon-phonon interaction, enhancing or suppressing it.

physics.optics↗