Eindhoven University of Technology (TU/e)

Imagination is more important than knowledge

*Albert Einstein (1879—1955, German physicist)*

This webpage provides a collection of research activities and publications on

Large amounts of data in high dimensions are becoming progressively available and accessible from various sources such as business enterprises, transaction-based information, social media, remote sensing technologies, wireless sensor networks, the Internet of Things as well as the Internet search and web analytics, and will continue to fuel exponential growth in data for the unforeseeable future. Therefore, the ability to analyse such large datasets, so-called big data, has become a key basis of competition, underpinning new waves of productivity growth, technological innovation, and consumer surplus. Achievements in these areas involve tasks requiring statistical manipulation and optimization over large sets of parameters, such as training artificial neural networks. Hence, it has become increasingly important, not only to be able to navigate high-dimensional landscapes but also to do so in a reasonable time.

For large-scale optimization problems, such as the ones originated by high dimensional machine learning problems, **metaheuristics** inspired by biological, social or opinion dynamics, have emerged as the go-to method. While metaheuristics offer high-level robust procedures that coordinate local and global search strategies to ‘quickly’ find sufficiently good-quality solutions, many lack any proof or guarantee of the quality of solutions. Well-known metaheuristics include simulated annealing, genetic algorithms, particle swarm optimization, ant colony optimization, artificial bee colony optimization, and hybrids thereof. On the other hand, **interacting particle systems** (IPS) are frequently used to investigate the emergence of collective behaviour in various applications such as mathematical biology, swarming, crowd dynamics, and opinion formation on social networks. In search of macroscopic patterns from microscopic interactions among agents, techniques from kinetic and mean-field theory are often employed. **Purpose-driven Interacting Particle Systems** aims in bringing the two fields together to develop a mathematical framework in which metaheuristics for optimization or sampling can be theoretically analyzed.

- Lorentz Center Workshop on Purpose-driven Particle Systems, March 13 - 17, 2023.

- Pinnau, R., Totzeck, C., Tse, O., & Martin, S. (2017). A consensus-based model for global optimization and its mean-field limit.

Mathematical Models and Methods in Applied Sciences, 27(01), 183-204. - Totzeck, C. (2017). Consensus-based global optimization.

In Progress in Industrial Mathematics at ECMI 2016 19 (pp. 409-416). - Carrillo, J. A., Choi, Y. P., Totzeck, C., & Tse, O. (2018). An analytical framework for consensus-based global optimization method.

Mathematical Models and Methods in Applied Sciences, 28(06), 1037-1066. - Ha, S. Y., Jin, S., & Kim, D. (2020). Convergence of a first-order consensus-based global optimization algorithm.

Mathematical Models and Methods in Applied Sciences, 30(12), 2417-2444. - Fornasier, M., Huang, H., Pareschi, L., & Sünnen, P. (2020). Consensus-based optimization on hypersurfaces: Well-posedness and mean-field limit.

Mathematical Models and Methods in Applied Sciences, 30(14), 2725-2751. - Brünnette, T. (2021) Consensus Based Optimization with Finite Range Interaction.

Master thesis, Eindhoven University of Technology. - Carrillo, J. A., Jin, S., Li, L., & Zhu, Y. (2021). A consensus-based global optimization method for high dimensional machine learning problems.

ESAIM: Control, Optimisation and Calculus of Variations, 27, S5. - Fornasier, M., Huang, H., Pareschi, L., & Sünnen, P. (2021). Consensus-based optimization on the sphere: Convergence to global minimizers and machine learning.

The Journal of Machine Learning Research, 22(1), 10722-10776. - Ha, S. Y., Jin, S., & Kim, D. (2021). Convergence and error estimates for time-discrete consensus-based optimization algorithms.

Numerische Mathematik, 147, 255-282. - Totzeck, C. (2021). Trends in consensus-based optimization.

In Active Particles, Volume 3: Advances in Theory, Models, and Applications (pp. 201-226). - Bae, H. O., Ha, S. Y., Kang, M., Lim, H., Min, C., & Yoo, J. (2022). A constrained consensus-based optimization algorithm and its application to finance.

Applied Mathematics and Computation, 416, 126726. - Borghi, G., Herty, M., & Pareschi, L. (2022). A consensus-based algorithm for multi-objective optimization and its mean-field description.

In 2022 IEEE 61st Conference on Decision and Control (CDC) (pp. 4131-4136). IEEE. - Choi, Y. P., & Koo, D. (2022). One-dimensional consensus-based algorithm for non-convex optimization.

Applied Mathematics Letters, 124, 107658. - Fornasier, M., Huang, H., Pareschi, L., & Sünnen, P. (2022). Anisotropic diffusion in consensus-based optimization on the sphere.

SIAM Journal on Optimization, 32(3), 1984-2012. - Fornasier, M., Klock, T., & Riedl, K. (2022). Convergence of anisotropic consensus-based optimization in mean-field law.

In Applications of Evolutionary Computation: 25th European Conference, EvoApplications 2022, Proceedings (pp. 738-754). - Göttlich, S., & Totzeck, C. (2022). Parameter Calibration with Consensus-Based Optimization for Interaction Dynamics Driven by Neural Networks.

In Progress in Industrial Mathematics at ECMI 2021 (pp. 17-22). - Huang, H., & Qiu, J. (2022). On the mean‐field limit for the consensus‐based optimization.

Mathematical Methods in the Applied Sciences, 45(12), 7814-7831. - Borghi, G., Herty, M., & Pareschi, L. (2023). Constrained consensus-based optimization.

SIAM Journal on Optimization, 33(1), 211-236. - Sünnen, P. (2023). Analysis of a Consensus-based Optimization Method on Hypersurfaces and Applications.

Doctoral dissertation, Technische Universität München.

- Totzeck, C., Pinnau, R., Blauth, S., & Schotthöfer, S. (2018). A numerical comparison of consensus‐based global optimization to other particle‐based global optimization schemes.

PAMM, 18(1), e201800291. - Grassi, S., & Pareschi, L. (2021). From particle swarm optimization to consensus-based optimization: stochastic modeling and mean-field limit.

Mathematical Models and Methods in Applied Sciences, 31(08), 1625-1657. - Huang, H. (2021). A note on the mean-field limit for the particle swarm optimization.

Applied Mathematics Letters, 117, 107133. - Cipriani, C., Huang, H., & Qiu, J. (2022). Zero-inertia limit: from particle swarm optimization to consensus-based optimization.

SIAM Journal on Mathematical Analysis, 54(3), 3091-3121. - Choi, Y. P., Ju, H., & Koo, D. (2023). Convergence analysis of Particle Swarm Optimization in one dimension.

Applied Mathematics Letters, 137, 108481.

- Carrillo, J. A., Hoffmann, F., Stuart, A. M., & Vaes, U. (2022). Consensus‐based sampling.

Studies in Applied Mathematics, 148(3), 1069-1140.

- Fornasier, M., Huang, H., Pareschi, L., & Sünnen, P. (2020). Consensus-based optimization on the sphere I: Well-posedness and mean-field limit.

arXiv preprint arXiv:2001.11994. - Totzeck, C., & Wolfram, M. T. (2020). Consensus-based global optimization with personal best.

arXiv preprint arXiv:2005.07084. - Carrillo, J. A., Totzeck, C., & Vaes, U. (2021). Consensus-based optimization and ensemble Kalman inversion for global optimization problems with constraints.

arXiv preprint arXiv:2111.02970. - Fornasier, M., Klock, T., & Riedl, K. (2021). Consensus-based optimization methods converge globally.

arXiv preprint arXiv:2103.15130. - Göttlich, S., & Totzeck, C. (2021). Parameter calibration with Consensus-based Optimization for interaction dynamics driven by neural networks.

arXiv preprint arXiv:2109.04690. - Grassi, S., Huang, H., Pareschi, L., & Qiu, J. (2021). Mean-field particle swarm optimization.

arXiv preprint arXiv:2108.00393. - Ko, D., Ha, S. Y., Jin, S., & Kim, D. (2021). Convergence analysis of the discrete consensus-based optimization algorithm with random batch interactions and heterogeneous noises.

arXiv preprint arXiv:2107.14383. - Borghi, G., Herty, M., & Pareschi, L. (2022). An adaptive consensus-based method for multi-objective optimization with uniform Pareto front approximation.

arXiv preprint arXiv:2208.01362. - Bungert, L., Wacker, P., & Roith, T. (2022). Polarized consensus-based dynamics for optimization and sampling.

arXiv preprint arXiv:2211.05238. - Huang, H., Qiu, J., & Riedl, K. (2022). On the global convergence of particle swarm optimization methods.

arXiv preprint arXiv:2201.12460. - Huang, H., Qiu, J., & Riedl, K. (2022). Consensus-Based Optimization for Saddle Point Problems.

arXiv preprint arXiv:2212.12334. - Klamroth, K., Stiglmayr, M., & Totzeck, C. (2022). Consensus-Based Optimization for Multi-Objective Problems: A Multi-Swarm Approach.

arXiv preprint arXiv:2211.15737. - Riedl, K. (2022). Leveraging Memory Effects and Gradient Information in Consensus-Based Optimization: On Global Convergence in Mean-Field Law.

arXiv preprint arXiv:2211.12184. - Borghi, G., Grassi, S., & Pareschi, L. (2023). Consensus-based optimization with memory effects: random selection and applications.

arXiv preprint arXiv:2301.13242. - Carrillo, J. A., Trillos, N. G., Li, S., & Zhu, Y. (2023). FedCBO: Reaching Group Consensus in Clustered Federated Learning through Consensus-based Optimization.

arXiv preprint arXiv:2305.02894.