A faster horse on a safer trail: generalized inference for the efficient reconstruction of weighted networks
Due to the interconnectedness of financial entities, estimating certain key properties of a complex financial system (e.g. the implied level of systemic risk) requires detailed information about the structure of the underlying network. However, since data about financial linkages are typically subject to confidentiality, network reconstruction techniques become necessary to infer both the presence of connections and their intensity. Recently, several "horse races" have been conducted to compare the performance of the available financial network reconstruction methods. These comparisons, however, were based on arbitrarily-chosen similarity metrics between the real and the reconstructed network. Here we establish a generalised maximum-likelihood approach to rigorously define and compare weighted reconstruction methods. Our generalization maximizes the conditional entropy to solve the problem represented by the fact that the density-dependent constraints required to reliably reconstruct the network are typically unobserved. The resulting approach admits as input any reconstruction method for the purely binary topology and, conditionally on the latter, exploits the available partial information to infer link weights. We find that the most reliable method is obtained by "dressing" the best-performing binary method with an exponential distribution of link weights having a properly density-corrected and link-specific mean value and propose two unbiased (in the sense of maximum conditional entropy) variants of it. While the one named CReMA is perfectly general (as a particular case, it can place optimal weights on a network whose topology is known), the one named CReMB is recommended both in case of full uncertainty about the network topology and if the existence of some links is certain. In these cases, the CReMB is faster and reproduces empirical networks with highest generalised likelihood.