Skip to content

Latest commit

 

History

History
648 lines (611 loc) · 105 KB

README.md

File metadata and controls

648 lines (611 loc) · 105 KB
Raw

PyPop7: Pure-PYthon library of POPulation-based (evolutionary / swarm-based / pattern search) OPtimization in black-box cases

GNU General Public License v3.0 PyPI for PyPop7 Documentation Status arxiv JMLR-2024 Downloads Downloads visitors WeChat-Group

PyPop7 is a Pure-PYthon library of POPulation-based OPtimization for single-objective, real-parameter, black-box problems. Its goal is to provide a unified interface and a set of elegant algorithmic implementations (e.g., evolutionary algorithms, swarm-based optimizers, and pattern search) for Black-Box Optimization (BBO), particularly population-based optimizers, in order to facilitate research repeatability, wide benchmarking of BBO, and especially their real-world applications.

More specifically, for alleviating their curse of dimensionality, the focus of PyPop7 is to cover their State Of The Art for Large-Scale Optimization (LSO), though many of their small/medium-scaled versions and variants are also included here (mainly for theoretical or benchmarking or educational purposes). For a growing list of public use cases of PyPop7, please refer to this online website for details. Although we have chosen GPL-3.0 license, anyone could use, modify, and improve this open-source library entirely freely for any (no matter open-source or closed-source) purposes.

How to Quickly Use

The following three steps are enough to utilize the black-box optimization power of this open-source library PyPop7:

  1. Use pip to install pypop7 on the Python3-based virtual environment via venv or conda:
$ pip install pypop7

For PyPop7, the number 7 was added just because pypop has been registered by other in PyPI. The icon butterfly for PyPop7 is used to respect/allude to the book (butterflies in its cover) of Fisher ("the greatest of Darwin's successors"): The Genetical Theory of Natural Selection, which directly inspired Prof. Holland's proposal of Genetic Algorithms (GA).

  1. Define the objective/cost/error/fitness function to be minimized for the optimization problem at hand (in this library, the term fitness function is used, following the well-established tradition of evolutionary computation):
import numpy as np  # for numerical computation, also the computing engine of pypop7

# the example is Rosenbrock, one notorious test function from the optimization community
def rosenbrock(x):
    return 100.0 * np.sum(np.square(x[1:] - np.square(x[:-1]))) + np.sum(np.square(x[:-1] - 1.0))

# to define the fitness function to be minimized and also its settings
ndim_problem = 1000
problem = {'fitness_function': rosenbrock,
           'ndim_problem': ndim_problem,  # dimension
           'lower_boundary': -5.0 * np.ones((ndim_problem,)),  # lower search boundary
           'upper_boundary': 5.0 * np.ones((ndim_problem,))}  # upper search boundary

Note that without loss of generality, only the minimization process is considered in this library, since maximization can be easily transferred to minimization just by negating it.

  1. Run one black-box optimizer or more on the above optimization problem (rosenbrock):
# here we choose LM-MA-ES owing to its low complexity and metric-learning ability for LSO:
#   please refer to https://pypop.readthedocs.io/en/latest/es/lmmaes.html for details
from pypop7.optimizers.es.lmmaes import LMMAES  # Limited-Memory Matrix Adaptation Evolution Strategy
# to define all the necessary algorithm options (which may differ among different optimizers)
options = {'fitness_threshold': 1e-10,  # to terminate when best-so-far fitness is lower than it
           'max_runtime': 3600.0,  # to terminate when the actual runtime exceeds 1 hours
           'seed_rng': 0,  # seed of random number generation (explicitly set for repeatability)
           'x': 4.0 * np.ones((ndim_problem,)),  # initial mean of search (mutation) distribution
           'sigma': 3.0,  # initial global step-size of search distribution (not necessarily optimal)
           'verbose': 500}
lmmaes = LMMAES(problem, options)  # to initialize the optimizer
results = lmmaes.optimize()  # to run its (often time-consuming) randomized search/evolution process
print(results)

Please refer to https://pypop.rtfd.io/ for online documentations of this well-designed ("self-boasted") Python library for black-box optimization (several praises from others).

A (Still Growing) Number of Black-Box Optimizers (BBO)

  • lso: indicates the specific BBO version for LSO (e.g., dimension > 100, but not an absolutely deterministic number),
  • c: indicates the competitive or de facto BBO version for small- or medium-dimensional problems (though it may also work well under some certain LSO circumstances),
  • b: indicates the baseline BBO version mainly for theoretical and/or educational interest, owing to its algorithmic simplicity (usually relative ease to mathematical analysis).

Note that this above classification based on only the dimension of objective function is just a very rough estimation for algorithm selection. In practice, perhaps the simplest way to algorithm selection is trial-and-error. Or you can try more advanced Automated Algorithm Selection techniques.

Clearly, this is an algorithm-centric rather than benchmarking-centric Python library only for black-box optimization, though benchmarking is also very crucial for black-box optimization.

For new/missed BBO, we have provided a unified API to freely add them if they can well satisfy the design philosophy widely recognized in the scientific research community. Note that currently both Ant Colony Optimization (ACO) and Tabu Search (TS) are not covered in this library, since they work well mainly in discrete or combinatorial search spaces in many cases. Furthermore, both brute-force (exhaustive) search and grid search are also excluded here, since it works only for very low (typically < 10) dimensions. In the near-future version, we will consider to add others (e.g., Simultaneous Perturbation Stochastic Approximation (SPSA)) into this open-source library. Please refer to development guide for more details.

Computational Efficiency

For large-scale optimization (LSO), computational efficiency is an indispensable performance criterion of BBO/DFO/ZOO in the post-Moore era. To obtain high-performance computation as much as possible, NumPy is heavily used in this library as the base of numerical computation along with SciPy and scikit-learn. Sometimes Numba is also utilized, in order to further accelerate the wall-clock time.

Folder Structure

The main folder structure of this open-source library PyPop7 is presented below:

  • .circleci: for automatic testing based on pytest.
    • config.yml: configuration file in CircleCI.
  • .github: all configuration files for GitHub.
  • docs: for online documentations.
  • pypop7: all Python source code of BBO.
  • tutorials: a set of tutorials.
  • .gitignore: for GitHub.
  • .readthedocs.yaml: for readthedocs.
  • CODE_OF_CONDUCT.md: code of conduct.
  • LICENSE: open-source license.
  • README.md: basic information of this library.
  • coverage-badge.svg: coverage rate of testing, calculated via Coverage.py and generated via https://smarie.github.io/python-genbadge/.
  • pyproject.toml: for PyPI.
  • requirements.txt: for development.
  • setup.cfg: for PyPI (used via pyproject.toml).

References

For each population-based algorithm family, we are providing several representative applications published on some (rather all) top-tier journals/conferences (such as, Nature, Science, PNAS, PRL, JACS, JACM, PIEEE, JMLR, ICML, NeurIPS, ICLR, CVPR, ICCV, RSS, just to name a few), reported in the (now still actively-updated) paper list called DistributedEvolutionaryComputation.

Sponsor

Now it is supported by National Natural Science Foundation of China under Grant No. 72401122, Guangdong Basic and Applied Basic Research Foundation under Grant No. 2024A1515012241 and 2021A1515110024. From 2021 to 2023, this open-source pure-Python library PyPop7 was supported by Shenzhen Fundamental Research Program under Grant No. JCYJ20200109141235597 (a total of 2,000,000 Yuan).

Citation

If this open-source pure-Python library PyPop7 is used in your paper or project, it is highly welcomed but NOT mandatory to cite the following arXiv preprint paper: Duan, Q., Zhou, G., Shao, C., Wang, Z., Feng, M., Huang, Y., Tan, Y., Yang, Y., Zhao, Q. and Shi, Y., 2024. PyPop7: A pure-Python library for population-based black-box optimization. arXiv preprint arXiv:2212.05652. (Now it has been submitted to JMLR, after 3 reviews from Tue, 28 Mar 2023 to Wed, 01 Nov 2023 to Fri, 05 Jul 2024, and accepted in Fri, 11 Oct 2024.)

Star History

Hits

PyPop7-Star-Data