This video presents a proof-of-concept for a "universal function approximator" that operates differently from traditional neural networks. Motivated by the opaque nature of neural networks, which produce millions of uninterpretable parameters, the creator developed a system that uses a genetic algorithm to evolve human-readable mathematical functions.
Functions are represented as trees of operators and constants. The system evolves a population of these function trees over generations using a set of mutation rules, including altering constants, replacing subtrees, and simplifying nodes. The fitness of each function is determined by a two-part loss function that considers both its accuracy on a given dataset and its complexity (the size of the tree), thereby favoring simpler solutions. While the system successfully approximates smooth, continuous functions, it fails on more complex, discontinuous tasks like identifying prime numbers. This failure is attributed to the evolution of chaotic, high-frequency functions where small mutations lead to unpredictable results, rendering the evolutionary process ineffective.
Creating a Understandable AI: An Evolutionary Approach to Function Approximation
0:00:05 The Problem with Neural Networks: The creator expresses frustration that neural networks, while powerful, are "black boxes" that produce a mess of matrix multiplications rather than an understandable mathematical formula that approximates a function.
0:01:37 A New Approach: Evolving Functions: The project's core idea is to represent functions as trees of mathematical operators (e.g., +, *, x, constants) and use an evolutionary algorithm to mutate and select the best-fitting functions for a given dataset.
0:02:51 Four Rules of Mutation: To evolve the functions, the algorithm uses four key mutation methods:
Slightly increasing or decreasing a random constant.
Replacing a random node (and its subtree) with a new, randomly generated tree.
Deleting a node by replacing it with one of its children.
Simplifying a subtree into a single constant if it contains no variables.
0:03:49 Evaluating Functions with a Complexity Penalty: A function's "fitness" is calculated using a loss function that combines two factors: how accurately it predicts the data points and a "complexity penalty" based on the number of nodes in its tree. This prioritizes simpler, more elegant solutions.
0:04:36 Demonstration 1: Approximating x²+1: In a live demo, the system is given a few data points for the function y = x² + 1. It almost instantly evolves a function that is an extremely close approximation of the correct answer.
0:06:13 Demonstration 2: Approximating a Complex Function: The system is then challenged with a more complex, non-obvious set of points. By increasing the "complexity" parameter (reducing the penalty for larger functions), it successfully finds a near-perfect function to fit the data.
0:08:33 Current Limitations: The current implementation is a proof-of-concept and has notable limitations, such as only supporting a single input and a single output, making it unusable for multi-variable problems or data like images.
0:09:04 The Deeper Problem of High-Entropy Functions: The algorithm fails when tasked with a difficult, non-smooth problem like predicting prime numbers. It generates overly chaotic and sensitive functions with extreme derivatives, where tiny changes cause massive output swings. This makes the evolutionary process ineffective, as mutations become random and unproductive.
0:10:40 A Potential Solution: The creator suggests this deeper problem could be solved by adding a penalty for "high entropy" functions—those with regions of high frequency or extreme derivatives—to guide the evolution toward smoother, more well-behaved solutions.