CPSC 636-600 Homework 4 (Total of 100 points) See the course web page for the due date and submission info. Instructor: Yoonsuck Choe April 6, 2015 1 SOM Problem 1 (Program: 10 pts): Fill in the two lines in the som.m skeleton code (marked ”......”) from the course web page. http://courses.cs.tamu.edu/choe/15spring/636/src/som.m Problem 2 (Written: 10 pts): With som.m, run a space-filling curve experiment. 1D lattice, 2D input (uniformly randomly distributed within a unit square). Visualize the map in input space at 0, 20, 100, 1000, 10000, and 25000 iterations. Problem 3 (Written: 10 pts): (1) Generate 20 random (x, y) points, and run a 1D-lattice SOM with 200 neurons to run a traveling salesman experiment. Adjust the learning rate and neighborhood radius update schedule appropriately. (2) Write a simple function to calculated the tour distance based on the learned weights. Hint: Simply go through the weight matrix w from top row to the bottom row and calculate the sum of Euclidian distance between w(n,:) and w(n+1,:). (3) Run SOM with different initial weights (repeat this 5 times), and run it on the same 20 inputs. Calculate the tour distance for each run and compare the results. Discuss how optimal SOM can get, based on these results. 2 Neurodynamics Problem 4 (Written: 10 pts): Consider the following dynamical system: where the matrix A = [-4. dx = Ax dt -2; 5, 1] and x is the state vector (column vector, as usual). 1. Calculate the eigenvalues of the matrix A and guess what would be the behavior of the dynamical system (diverge? converge?). A = [-4. -2; 5, 1]; eig(A) in Octave or Matlab will give you the eigenvalues. See slide08.pdf, page 12. 1 2. Given an initial condition x(0) of (-2,1)’, compute the trajectory and plot it. What is the equilibrium point? You can use a simple numerical integration method based on Taylor series expansion: dx dt x=x(t) = x(t) + ∆tAx(t) x(t + ∆t) = x(t) + ∆t where ∆t is a scalar value that indicates the integration step size. Try ∆t = 0.01 and iterate for 1000 times. Collect the data points and plot it. 3. Try various other initial conditions and see how the system works. Try plotting all the trajectories in one plot. 4. Optional: experiment with different A matrices. Try to guess what kind of matrices will give different categories of eigenvalues. 5. Optional: experiment with different ∆t (0.001, 0.1, 0.5) and compare the accuracy (the smaller the more accurate). You may have to increase the number of iterations for smaller ∆t. Problem 5 (Program: 10 pts): Fill in the two lines in the hopfield.m skeleton code from the course web page, run the three examples (noisy input and partial input), and report your results. http://courses.cs.tamu.edu/choe/15spring/636/src/hopfield.m There are several other files that you may need. Important: For matlab, use a dogfilter of size 7 instead of 17. Or, better yet, 1. Fix the last line in dogfilter.m to this M = real(conv2(X,g,’valid’));. 2. When calling dogfilter, do dogfilter(rand(36,36),17), instead of dogfilter(rand(20,20),17). This is when you construct the input matrix. A typical patten you should get is like this: Problem 6 (Written: 10 pts): Test your implementation and see how much noise the Hopfield network can tolerate. Change the noise level and see beyond which point the network does not settle to the correct attractor. Adjusting the inequality part (0.6) will allow you to control the noise level: (rand(1,20*20)<0.6). Problem 7 (Written: 10 pts): Test how much omitted information the Hopfield network can tolerate. This part inp(1:10,:)=zeros(10,20); controls how much input to eliminate. Problem 8 (Written: 10 pts): Try increasing the number of memory patterns and check if you get the αc value as determined in slide09.pdf page 25–26. 2 3 PCA Problem 9 (Written: 20 pts): Write a simple function to do PCA, using cov(...), eig(...) and other standard Octave/Matlab functions. Implement Oja’s rule and compare the results (first principal component vector) against the basic implementation of PCA above. Use the following data set. inp=[randn(800,2)/9+0.5;randn(1000,2)/6+ones(1000,2)]; 3