Đề tài Evolving neural network through a reverse encoding tree

Evolving Neural Network Through A Reverse Encoding Tree Giáo viên hướng dẫn: PGS.TS Huỳnh Thị Thanh Bình Sinh Viên : Đào Minh Khánh 20183562 Lê Thế Tài 20174180 Evolving Neural Networks through a Reverse Encoding Tree : Haoling Zhang;  Chao-Han Huck Yang, Hec tor Zenil, Narsis A.Kiani, Yue Shen, Jesper N.Tegner, IEEE 2020  Reference Evolving neural networks through augmenting topologies: Kenneth O. Stanley,Risto Miikkulainen, The MIT Press Journals 2002 x Problem √ 01 Neur o evolution through Augmenting Topologies 02 Encoding Network 03 Neural Evolution of Reverse Encoding Tree 04 CartPole-v0 1) NEAT NeuralEvolution of Augmenting Topologies(NEAT) is a genetic algorithm for generation of evolving neural network developed by Ken Stanley 2002 Target: Modify the weighting parameters and structure of networks Keys techniques: - Genetic Encoding - Historical Marking Crossover - Speciation - Explicit Fitness Sharing - Minimal Structure of Initialization NEAT Is there a genetic representation that allows disparate topologies to cross over in a meaningful way? How can topological innovation that needs a few generations to be optimized be protected so that it does not disappear from the population prematurely? How can topologies be minimized throughout evolution without the need for a specially contrived fitness function that measures complexity? NEAT Encoding of Network Architecture NEAT Mutation Crossover in Network Topologies “Competing Conventions” NEAT Crossover “Historical Markings” Protecting Innovation with Speciation E: number of excess genes D: number of disjoint genes W: the average weight difference of matching genes, include disabled genes N: the number of genes in the larger genome, normalized for genome size (N can be set to 1 if both genomes are small) Protecting Innovation with Speciation Threshold δt, in each generation, genomes are sequentially placed into species Each existing species is represented by a random genome inside the species from the previous generation Fitness Computation Speciation “explicit fitness sharing” The adjusted fitness f’i for organism i is calculated according to its distance δ from every other organism j in the population The sharing function sh( ) is set to 0 when distance δ(i, j) is above the threshold δt; otherwise, sh(δ(i, j)) is set to 1 Minimal Initialization Random is unlikely to reduce the complexity of the structure via mutation want network to be as simple as possible Inital structure with no hidden nodes. 2) Network Encoding NeuralEvolution of Reverse Encoding Tree Definition Each neural network is encoded by edge encoding Limit the maximum numbers of nodes (m) in the neural network Each neural network is defined a individual seeding(I) from the population Algorithms Detail Initialize Population Review Evolution Progress m is the max node in the network p is the number of individual when initializing population Initialize Population The distance between two individuals is encoded as theEuclidean distance of the corresponding feature matrix  Initialize Population Evolution Progress The correlation coefficient of distance from the optimal position of the individual and fitness for all the individuals in each cluster is calculated, to describe the situation of each cluster: p=-0.78 W hen 2) Create a glocal individual from the specified parent individuals b) Golden section search a) Binary search RET Search 1) Create a nearby individual from the specified parent Random probability Add new node, delete node, delete connection,.. b Golden section search Cart Pole hyper-parameter value iteration 1000 fitness_threshold 0.9999 evoluation_size 6 activation relu episode_steps 500 episode generation 20 Method Avg.gen Paper 147.33 0 Hidden Node 141.13 2 Hidden Node 144.79 3 Hidden Node 142.11 4 Hidden Node 140.29 6 Hidden Node 114.89 8 Hidden Node 142.81 10 Hidden Node 92.05 15 Hidden Node 138.27 Conclusion Advantage Work with a few problem in reinforcement learning( Cart-Pole, Lunar Lander,...) Disadvantage: The number maximum node must be chosen by people Expensive memory when the number max node is large