Code Is BrokenSymbolic and Evolutionary Artificial Intelligence
Markov Decision Processes and the Foundations of RL
These notes introduce the foundational concepts of Reinforcement Learning (RL), a subfield of machine learning where agents learn to interact with an environment to maximize cumulative rewards. The content is based on the teaching material of Lorenzo Fiaschi for the Symbolic and Evolutionary Artificial Intelligence course at the University of Pisa (a.
Many-Objective Optimization and the NSGA-III algorithm
In engineering, optimization problems often involve multiple conflicting objectives. While classic Multi-Objective Optimization typically refers to problems with 2 or 3 objectives (see Families of Multiobjective Genetic Algorithms), there exist scenarios with a considerably higher number of objectives. When the number of objectives increases beyond a certain threshold, problems are categorized differently:
Families of Multiobjective Genetic Algorithms
In real-world optimization problems, it is common to encounter scenarios where multiple conflicting objectives must be optimized simultaneously. This is the essence of multiobjective optimization, where instead of seeking a single best solution, the goal is to approximate a set of trade-off solutions that form what is known as the Pareto front.
Function Approximators in Reinforcement Learning
These notes explore the principles and methods of function approximation in Reinforcement Learning (RL), focusing on differentiable approaches for value and policy estimation. The content is based on the teaching material of Lorenzo Fiaschi for the Symbolic and Evolutionary Artificial Intelligence course at the University of Pisa (a.
Lexicographic Multi-Objective Optimization and the I-Big-M Method
This section introduces Lexicographic Multi-Objective Optimization Problems (LMOPs), explores standard and non-standard approaches to solving them, and illustrates how Non-Archimedean methods naturally extend classical algorithms. We will gradually connect definitions, algorithms, and examples to form a coherent understanding — culminating with the Infinitely Big-M method.
Mixed Pareto-Lexicographic Optimization
In many real-world scenarios, not all objectives are equally important. Some goals must be achieved with higher priority, while others may be considered only if the most important ones are satisfied or optimized. This intuition leads to the study of Mixed Pareto-Lexicographic (MPL) optimization problems, which aim to combine the strengths of lexicographic and Pareto-based approaches within multi-objective optimization.
Model-Based Reinforcement Learning and Exploration Strategies
Model-Based Reinforcement Learning (MBRL) introduces a paradigm in which the agent first learns an internal model of the environment from experience, and then uses this model to derive value functions, optimal policies, or action-value functions through planning. Model learning itself is formulated as a supervised learning problem, and the estimation of model uncertainty provides an additional perspective for reasoning about decision-making.
Non-Archimedean Artificial Intelligence – Alpha Theory
Non-Archimedean mathematics enlarges the familiar real number system into a richer numerical universe where infinite and infinitesimal quantities stand side by side with ordinary finite ones. This expansion matters for Artificial Intelligence because the way we represent numbers shapes the way we reason with them.
Parallelization of Multi-Objective Evolutionary Algorithms
In the context of Multi-Objective Evolutionary Algorithms (MOEAs), parallelization becomes a natural and effective strategy when multiple computing cores or distributed computing nodes are available. This is due to the intrinsic characteristics of evolutionary algorithms, which make them well-suited for concurrent execution.
Proximal Policy Optimization and Advanced Actor–Critic Variants
The concept of trust policy regions aims to limit excessive variation in the policy between updates, thus improving stability and avoiding performance collapse. While using the KL divergence as a strict constraint achieves this goal, it can be computationally demanding, with the notable exception of the Natural Policy Gradient method.
Reinforcement Learning - Model Free Methods
In many real-world scenarios, it is impractical or impossible to assume full knowledge of the environment’s dynamics, such as the transition probabilities or the reward function. For this reason, model-free reinforcement learning techniques have been developed. These approaches allow an agent to learn from direct interaction with the environment, without requiring an explicit model.
The Problem of Visualizing Data in High Dimensions
Visualizing data in spaces with more than three dimensions is a challenging task, particularly in the context of many-objective optimization. The goal of such visualizations may vary depending on the context: In some cases, the aim is to display the distribution of the dataset in the input space; In others, it is useful to visualize the Efficient Set (or Pareto front) in the output space.
Why Optimization Is the Engine Behind AI
Optimization is at the heart of modern artificial intelligence. Even though it might seem redundant to revisit this topic after having studied it in courses like “Optimization Methods and Game Theory,” its centrality in AI justifies another look. In fact, optimization is the underlying mechanism that enables the vast majority of learning algorithms.