We wanted to be able to store into text files the number of altControls a random brain used, the number of processes the brain used, the constants that each Above or Below Threshold was using, and all the variables (the left turn, right turn, and amount of time), for each MotorControl each random brain used.

Now, there is a savetofile() process which allows us to write a file line by line including all of the above methods.  The BufferedWriter() and FileWriter() methods were called to create a new file for each individual random brain. This way we can keep track of what a robot did and run the robot again if we were happy with the brain it used.

In order to run the robot’s brain again, we needed to create a loadfromfile() process. This process takes in a file already created, and breaks it down line by line. The Scanner() method was called in order to read in a file that is given. Again, like above, each line has a different meaning regarding the robots brain (such as the constants it used). This allows for an easy way to see what the robot’s brain created, along with running the brain again to see if it was a worthwhile brain!

The process of coding the crossover method was complicated. We decided to crossover our robots starting at choosing a sensor. For a chosen sensor, the robots would switch their information pathways attached to that sensor. I’ll be calling a connection between a sensor, threshold process, AltController process, MotorController process, and MotorFinal process a ‘branch’. This crossover implementation is much like two of our mutation options: adding and removing a random branch. In crossover, I needed to remove one or more branches (depending on how many branches are connected to the chosen sensor) from the two robots and add them to the opposite robot.

My first attempt at writing this method did not turn out so well. As I started writing, I realized I needed to worry about more and more channel connections and processes. The main problem was remembering to delete the old information signal channels from each process and add new channels that were created. Each test of the method would leave channels talking to empty space or process floating by themselves. Eventually, we were able to create a working method after writing some pseudo-code. Learning to write the pseudo-code first has become very handy.

Before the weekend, we were able to run one test of our new crossover method.
This graph shows the scores for 5 trials using a population size of 12 robots running for 20 generations (That’s 240 simulations). Trial 1 was able to get its score to get much lower (which is better) than the rest of the trials. This trial created robots which were able to navigate into the island room of the arena, a room which gives the robot a very good score boost because it’s a difficult room to access. So this trial started selecting robots that got into that room, however, as I watched the robots, they were never able to get out of the room. This may mean we need to change how we mutate the robots so they can learn to escape the room, or change the scoring metric.

All of the other scores in the trials fell as well, meaning all of the robots started doing better eventually. We need to run more experiments with crossover and mutation combined to find out what the best combination of methods and variables is.

Our new calculations consisted of taking the minimum score over every robot generated so far.  For example, if we were looking at all twenty robots in each generation up to generation 5, we would take the single minimum score out of the 100 robots seen so far. With these new minimum calculations, we were able to produce the following graph:

This was the graph we produced for a Mutation Rate of 5, having a population of 12 and 20 generations ran. The light blue line, labeled as WallBrain in the key, was our minimum guideline we were following. The unseen scores were produced by a hard-coded robot we new worked. The time alloted for each robot was one minute. After looking at the graph, we saw that for the fifth experiment ran with a Mutation Rate of 5, that there were some robots below the minimum guideline score! We were able to see that the lowest score produced out of all of our runs (even other mutation rates of 10, 15, and mutating only one thing at at time) was 0.77749997377396.

We traced this score back to see that it came from the fifth robot in generation eighteen. Then, we looked back and produced the following directed chart to see what our robots brain looked like!

This robots brain is very simple, but our smartest one yet! It covered the most amount of space in one minute. If there was an above threshold of 1.229, then the robot would slightly turn right. It appears that the robot bounced around our arena, successfully covering the most space we have seen yet!

In order to get to this point, we added some important methods into our evolutionary algorithm. The steps we followed are:

1. Initialize the brains of the robots (giving them their random connections to channels)
2. Repeat the following steps until all generations have been formed:

• Evaluate the robot to see how well it did. They are given a score based on our algorithm.
• Select the next set of robots to move onto the next population using our Tournament Selection method.
• Mutate the selected robots and add them to the next population created.

These experiments are all based on 20 generations of robots, each consisting of a population of 12 robots. The Tournament Selection was the process used. Looking at the different mutation rates, it appears that the mutation rate of 5 has done this best with these processes. This is because the graph has sloped down the most exponentially, along with has the lowest overall scores out of the graphs. The mutation rate of 10 did well too, however it is a bit rocky along the way down. We would like for it to be smoother as it is decreasing.

Overall, the minimum scores were closer to our guideline of the WallBrain. Again, mutation rates 5 and 10 had the lowest scores. The graphs overall were a lot more rough compared to just the averages, but we are seeing that we can get some lower scores. 

Last school year Dr. Goadrich, Dr. Jadud, Molly Mattis, and I were able to build and program a robot to enter into the Trinity College Fire Fighting Home Robot Contest. We used an occam-pi library called Plumbing to program the robot. Using a parallel programming language allows for processes to run concurrently and makes changing how the robot acts a much easier task. Our robot was able to place 15th out of 41 contestants. You can see the video here.

While we were pleased with the results of our first run, our plans for the summer are to hopefully create a better candle search algorithm by introducing AI to evolve the robot brain controller. To do this we need to use a simulator which can run a virtual replication of our robot. We will program the simulator to produce robots for our genetic algorithm to evaluate. After evaluation, we will use crossover and mutation AI techniques to generate new robots which will hopefully be better than the last generation of robots.

At first we tried using USARSim as our simulator, but found it to be a bit overly complicated for our needs. For now, we’ve decided to use the Simbad 3D Robot Simulator. As our project gets closer to completion we may switch back to the USARSim simulator. My project partner Eren Corapcioglu has been working with the simulator to produce all the room configuration options the contest has. The Simbad simulator uses Java as its language and Java3D to render the virtual world.

Because our simulator uses Java and our robot uses a parallel language, we needed to find a way to translate Java into a parallel language. For this, we use JCSP. In the beginning I tried translating the occam-pi code into Java line for line. I was able to reproduce our physical robot which simply followed the right wall of the arena. We discovered that in order for our genetic programming to work, we would need to break up the processes into smaller parts. Each day we find different ways to produce the same processes but are easier to evolve.

Over the past few weeks we tried several different techniques in creating a program which will create random robots for our genetic algorithm to evaluate. As Eren mentioned in her last post, one attempt simply created processes and picked information channels randomly. This created rather unintelligent robots who listened to instructions from dead air.

—-Recent Progress —-

Instead, I expanded the brain processes into three main parts: Motor Control, Motor Final, and Alt Control. Alt Control generates a random number of Above and Below sensor threshold values and listens for a sensor to tell it if some threshold for either distance to an obstacle or light level has been reached.

Alt Control then tells a Motor Control “Hey, Motors, do something!” Each Motor Control has an object with values for which direction and speed each wheel should turn and how long the wheels should respond. The Motor Controls then feed into a Motor Final which is a simple alternative process. Motor Final simply keeps the robot from locking up if more than one Motor Control is trying to tell the motors to do different commands. The alternative switch selects which process gets to take place first.

The picture is the graphical representation of this brain configuration using Graphviz. The numbers for each process represent:
Above/Below Thresh: the sensor threshold level to send a signal.
Motor: The left and right wheel motor values. Negative is backwards, positive forwards. And the time period for this movement.
Node Edges: The channel number information and signals are flowing on.
Score: A score generated using an equation based on the time the robot has been in the world (the simulation ends if the robot runs into a wall) and the odometer number.

Today, we have updated these graphs. This first graph shown below represents our basic working brain of the robot. This brain allows the robot to walk around each room like it’s suppose to, but the code itself is hardcoded.

This next graph was created with one of our random brain’s that we tested out. Here our code was not hard coded; it was assigned random sensors and thresholds to venture through. This one was one of our more interesting random brain graphs.

The NOWHERE portion of this graph represents when a channel does not have an input or an output. On our “normal brain” you can see that this NOWHERE option is not visible. This is because when the brain works correctly all of the channels take in an input and an receive at least one output.

Our next task is to make our random brains functional so there is no longer a NOWHERE option for these random brains.

Several papers have been written and published discussing different teachers and students use with occam-pi and how useful they found it. One that I found helpful to my understanding about occam-pi and it’s specific use with creating robots was the Safe Parallelism for Robotic Control. The paper discusses the building of the students’ and professors working together to build a robot that is controlled by the occam langue. It breaks down what a robot might need to do to execute a command and how the occam langue helps the robot run many tasks at a time. It gave some basic example codes which made the paper easy to understand.

Also, as we have been working these last few weeks, we have made progress towards randomize our robots brain and the channels it receives as inputs and what they output. Today, we were able to produce a graph using the DOT language to physically see where the connections have been made with our random brains. One graph example is:

Our goal was to find equations that allowed us to minimize both the Lifetime of the robot and the Odometer of the robot. We did this by using to simple equations which then gave us the value for both the distance and time of the robot.

During these trials, we ran into two problems: the first, if the robot was located in the negative x-axis of the world, and “collided” into a wall, instead of colliding all the way through, the robot would bounce of, and then move towards the way again. It kept repeating this pattern until we stopped the program on our own. After an hour of debugging, we found that this was due to our code since we had turned on the setUsePhysics(true) method (part of the environment description class). This use of physics was telling the robot to bounce of the wall.

Our second problem was the light was not set to a bright enough light, which led to the robot not being able to sense it was there. We also saw that in different rooms, the attenuation of the light was different depending on the size of the room and the location of the wall from the light. We looked into the PointLight class to try to understand why this was happening. It reassured us that the brightness of the light depended on where the walls were located with respect to the light. The PointLight class has a method called setAttenuation(), which brings in three constants: the light’s constant attenuation, the light’s linear attenuation, and the light’s quadratic attenuation. We further looked into these constants and found a relative equation which calculates the lights attenuation. This has helped us to better the brightness of our light in all four rooms.

I created this image using iMindMap. The map shows the channel and signal connections between the different processes. Our Java code will not be completely the same – our motor control will be different in the virtual world, for example.

So far, I have been able to translate the flame.detected, above and below threshold, and much of the brain components. Dr. Goadrich helped with this process by creating a version of the adc.get.reading process so that both the light sensors and range sensors can read from above and below threshold. Our robot now avoids walls (sort of) while looking for the candle in parallel. Now I need to translate the process which makes the robot follow the right wall.

At first, we wanted to create bounding polytopes again. The problem we found with using the BoundingPolytope function in java is that all the regions of the polytope have to be convex. This would have worked for some of the rooms, but then some of the regions would have not been convex if we included all possible areas outside of the room where our robot could potentially notice light coming from.

Once understanding a BoundingPolytope would not work, we then decided to just create a bounding box around the rooms themselves. This was decided because Kathryn pointed out with all the overhead lights in the arena, it would be almost impossible for the sensors we are using on the robot to detect light anywhere outside of the room.

With that said, I hard coded a file with all the possible light locations: two in each room; which did include all possibilities for the lights when one of the four rooms was flipped. These next two links will give you a visual on the flipped rooms:

After hardcoding the lights, I was able to create a general code which opened the file and read the lines. The code broke up the numbers in the line in order to read the vector which contained the light location. Also, it retrieved the location of the smaller and larger bounding boxes which are needed for the code. Taking these three pieces, the bounding boxes were created.