2026 EOSBF
mle_fit: A High-Performance Computational Framework for Robust Power-Law Analysis in Complex Systems¶
Power-law distributions appear everywhere in the science of complex systems as a central characteristic. However, their accurate identification and characterization is a non-trivial task, since graphical analysis and linear fitting in log-log scale are unreliable for testing the power-law hypothesis. A rigorous statistical approach is therefore necessary to estimate their scaling exponent and assess their goodness-of-fit.
To address this challenge, we developed mle_fit, a novel library for fitting and testing heavy-tailed distributions, integrating a user-friendly Python interface with a high-performance back-end in Modern Fortran. The library implements the Maximum Likelihood Estimation (MLE) procedure to estimate the scaling exponent and uses the Kolmogorov-Smirnov (KS) statistics to determine the optimal lower bound of the power-law behavior. It also computes a p-value to assess the goodness-of-fit, allowing for hypothesis testing.
The library’s key feature is its high-performance Fortran core, tightly integrated with the Python interface, which provides a significant performance increase over existing pure Python and R packages for fitting heavy-tailed distributions, making it an ideal choice for big data. In addition, it offers more refined control by allowing the user to choose a local minimum for the KS statistic instead of the global one. This enables a nuanced trade-off between the fit’s quality and the amount of data in the tail, with the corresponding p-value serving as the ultimate criterion for hypothesis testing.
We applied mle_fit to two key problems in quantitative linguistics and network science, using a corpus of 12 literary works. First, we investigated Zipf’s law and found no evidence for the classical rank-frequency distribution as a power law; in contrast, we confirmed that the corresponding word-frequency distribution (also known as the lexical spectrum) robustly follows a power law. Second, we analyzed the degree distribution of word adjacency networks, finding strong evidence that these are scale-free networks, with exponents consistently falling within the range, which reinforces the small-world properties of natural language.
In conclusion, our work introduces a powerful and efficient tool for the complex systems community. The mle_fit library enables rigorous and fast analysis of power-law phenomena. Future work will focus on extending its capabilities, including support for binned data and fitting other relevant distributions, such as the truncated power-law. Furthermore, we plan to implement additional fine-control parameters, like a weighted KS statistic, to better handle finite-size effects and deviations in the tail.
Acknowledgments: FAPEMIG, CAPES, and CNPq.