In the previous years the interest and development of robots/mobots in our school (MPSC now MUST) was not that high. Robots where made at those times but the use of hardware and computer peripherals was so limited and proper dissipation of internet access was confined that most of the higher years tried to code things as if they where coding everything.   Kits where not also available that time or it already existed but no information was relayed. Then came attempts on creating line followers but it lacks speed and accuracy and to some it was a failure. Last year 2008, mobile robots was implemented and started out as a competition. And as a prize a higher grade will be given. A standard track was created so there will be no bias during the presentation. All where fun to see that all your hard work is fulfilling. There are mobots that performed very fast and with small lapses on the track, there were also mobots that where very slow but was able to follow the line with an average accuracy of 99.99%. There was also a mobot that runs not so fast and not so slow  but with the lights that glow and the music that it performs while following the line makes it a hot mobot. No robot kits where used but it performed well with only a limited time for accomplishment.

Earlier, a group of graduating Computer Engineers led by Jovencito Talasan attempted and has succeeded on implementing an artificial intelligence mobot with the use of a robot kit called intelli-brain bot. Java was used to control the robot. They used q learning as the method for the mobot to learn the pathways. The development of mobile robots was increasingly progressing and its a good sign that sooner a speed, accuracy and way of learning will be perfect. Below is the video of their mobot and the next post will be the program and flow chart.

One method of Neural Network Programming is Q Learning that will be applied for ai robots. Q learning is is an off-policy method that can be run on top of any strategy wandering in the Markov Decision Process (MDP). It uses the information observed to approximate the optimal function, from which one can construct the optimal policy or to summarize the function of Q learning, only the state-action function needs to be stored and updated.

This the intelliBrain Mobot’s Flow Chart:

The Language used for the Neural Network Programming is JAVA. The Program was then loaded to the controller.

LEARN_RATE = 0.35f; // the learning rate constant
public static float EXPLORE_RATE = 0.15f; // the rate it tries out random
//actions
private static Servo mLeftServo;
private static Servo mRightServo;
byte [][] state;
byte [][]action;
private static float [][] Q; // table of Q-values (action,state)
byte i = -1; // last state
byte a = -1; // last action
byte a1 = 0; // action with highest q-value
float r = 0; // Last reward
public ObstacleAvoidance(byte actions, byte states) // constructor
{
state = new byte[states][4]; // 16 unique states, 4 unique SENSOR
action = new byte[actions][2]; // 4 unique actions with 2 actuators
initAction(action);
initState(state);
Q = new float[actions][states]; // table of Q-values for action-states
//setRandomValues(Q);
setLearnedQValues(Q);
}
public static void main(String [] args)
{
byte actions = 4;
byte states = 16;
long iterMax=100000;
ObstacleAvoidance qutil = new ObstacleAvoidance(actions, states);
ObstacleAvoidance qbrain = new ObstacleAvoidance(actions, states);
byte[] e = new byte[8]; // The environment values
2
byte [] motorCommand = new byte[2]; // Commands to send to motors
mLeftServo=IntelliBrain.getServo(1);
mRightServo=IntelliBrain.getServo(2);
for (long iter=0;iter<=3)
e[0]=1;
else
e[0]=0;
if(e[5]<=4)
e[1]=1;
else
e[1]=0;
if (e[6]<=3)
e[2]=1;
else
e[2]=0;
if (e[7]<=3)
e[3]=1;
else
e[3]=0;
}
public static void initAction(byte [][] state)
{
byte count = 0;
for(byte i=0;i<2;++i)
for(byte j=0;j<2;++j)
{
state[count][0] = i;
state[count][1] = j;
++count;
}
}
public static void initState(byte [][] state)
{
byte count = 0;
for(byte i=0;i<2;++i)
for(byte j=0;j<2;++j)
for(byte k=0;k<2;++k)
for(byte l=0;l<2;++l)
{
state[count][0] = i;
state[count][1] = j;
state[count][2] = k;
state[count][3] = l;
5
++count;
}
}
public static void setRandomValues(float [][] Q)
{
for(byte i=0;i=0)
{
r = getRewardValue(e);
a1 = getMaxAction(j);
Q[a][i] = Q[a][i] + LEARN_RATE * (r + Q[a1][j] - Q[a][i]);
// Q["+a+"]["+i+"] = " + System.out.println(""+Q[a][i]);
}
i = j;
// GREEDY
float rand = (float)Math.random();
if(rand > EXPLORE_RATE)
{
a = getMaxAction(j);
//System.out.println("Q["+a+"]["+j+"] = " + Q[a][j]);
}
// E-GREEDY
else
a = (byte)(Math.random() * 4);
buzzer.beep();
return a;
}
// find the largest Q-value for a given state (j), and return action
public byte getMaxAction(byte state)
8
{
float max = -1000;
byte action = 0;
for(byte a=0;a max)
{
max = Q[a][state];
action = a;
}
}
return action;
}
public byte getState(byte [] e)
{
byte count = 0;
for(byte i=0;i<2;++i)
for(byte j=0;j<2;++j)
for(byte k=0;k<2;++k)
for(byte l=0;l<2;++l)
{
if (e[0] == state[count][0])
if (e[1] == state[count][1])
if (e[2] == state[count][2])
if (e[3] == state[count][3])
return count;
++count;
}
return -1;
}
public float getRewardValue(byte [] e)
{
byte reward=0;
if(a==0)
reward += 3; // good reward
if (e[0] == 1||e[1] == 1||e[2] == 1||e[3] == 1)
reward -= 2; // give bad reward: penalize
return reward ;
}
}

To view the Result of the intellibrain Neural Network Programming on ai Robots. Click here

This is really amazing that it really has become a reality…

Cellphone Controlled Robot

If you can get through the cell phone text speak, you’ll probably enjoy this cool tutorial on how to build a cell phone controlled robot. This bot decodes the key tones, similar to the automated phone systems we’ve all experienced. It uses a chip called a MT8870 DTMF decoder to translate the signal for the Atmega 16 controller or if you can find ways to use other microcontrollers such as Zilog or the PIC. The circuit diagram is pretty hard to read but I think it easier for you guys since you have experiences from designs and hacks on circuits, maybe we missed a downloadable one somewhere. The source code is available. It would be nice to get some feedback from the robot, so you aren’t driving it completely blind. This is similar to the Lego cell phone rover .  In the future this mobot will recognize voices to follow.

Using these robots as a tool, Prof. Ishiguro has invented and explored the field of Android Science which looks at both the appearance and behavior of humanoid robots and their impact on human robot interactions.  Like what he had made can feel when you try to hurt it, it reacts and says something when it’s hurt.

In particular, he is looking to verify the existence of the uncanny valley and to explore how to make androids which sufficiently resemble humans to be likable. Ultimately, the question is now to understand what human presence really means.