When the stars don't align: Investigating inconsistencies in binary black hole formation across population synthesis codes
Binary population synthesis (BPS) codes are extremely useful tools for investigating both the end-to-end lives of binary stars as well as a myriad of astrophysical phenomena observed in the Universe. Given the widespread use of BPS in modern astrophysical research, it is imperative to systematically compare BPS codes across the spectrum of computational efficiency, flexibility, and physical realism to gauge their consistency and robustness. In this work, we perform BPS using three modern codes---the rapid code $\texttt{COSMIC}$, the hybrid code $\texttt{METISSE}$ integrated into $\texttt{COSMIC}$, and the detailed code $\texttt{POSYDON}$---on three single-metallicity populations of identical initial binaries, ensuring consistent choices in physical parameterizations where possible. Investigating the final population of merging binary black holes (BBHs) as a test case, we find stark differences in the properties, formation pathways, and progenitors across the three codes. In an initial population of one million binary stars at $0.01 Z_\odot$, each code results in $\sim 5,000-9,000$ BBHs that merge within a Hubble time. However, only one initial binary becomes a BBH merger in all three codes, and $\lesssim 14 \%$ of BBH progenitors consistently merge in two codes. Binaries that become BBH mergers in two codes often go through different evolutionary pathways and result in different final properties. In short, the codes are inconsistent in predicting BBH merger properties, even for identical initial binary systems. Our results highlight the need for systematic comparisons of BPS techniques, for a deeper understanding of physical and computational differences between BPS codes, and for caution in over-interpreting the results from any BPS code. (Abridged)