In the previous years the interest and development of robots/mobots in our school (MPSC now MUST) was not that high. Robots where made at those times but the use of hardware and computer peripherals was so limited and proper dissipation of internet access was confined that most of the higher years tried to code things as if they where coding everything.   Kits where not also available that time or it already existed but no information was relayed. Then came attempts on creating line followers but it lacks speed and accuracy and to some it was a failure. Last year 2008, mobile robots was implemented and started out as a competition. And as a prize a higher grade will be given. A standard track was created so there will be no bias during the presentation. All where fun to see that all your hard work is fulfilling. There are mobots that performed very fast and with small lapses on the track, there were also mobots that where very slow but was able to follow the line with an average accuracy of 99.99%. There was also a mobot that runs not so fast and not so slow  but with the lights that glow and the music that it performs while following the line makes it a hot mobot. No robot kits where used but it performed well with only a limited time for accomplishment.

Earlier, a group of graduating Computer Engineers led by Jovencito Talasan attempted and has succeeded on implementing an artificial intelligence mobot with the use of a robot kit called intelli-brain bot. Java was used to control the robot. They used q learning as the method for the mobot to learn the pathways. The development of mobile robots was increasingly progressing and its a good sign that sooner a speed, accuracy and way of learning will be perfect. Below is the video of their mobot and the next post will be the program and flow chart.

One method of Neural Network Programming is Q Learning that will be applied for ai robots. Q learning is is an off-policy method that can be run on top of any strategy wandering in the Markov Decision Process (MDP). It uses the information observed to approximate the optimal function, from which one can construct the optimal policy or to summarize the function of Q learning, only the state-action function needs to be stored and updated.

This the intelliBrain Mobot’s Flow Chart:

The Language used for the Neural Network Programming is JAVA. The Program was then loaded to the controller.

LEARN_RATE = 0.35f; // the learning rate constant
public static float EXPLORE_RATE = 0.15f; // the rate it tries out random
//actions
private static Servo mLeftServo;
private static Servo mRightServo;
byte [][] state;
byte [][]action;
private static float [][] Q; // table of Q-values (action,state)
byte i = -1; // last state
byte a = -1; // last action
byte a1 = 0; // action with highest q-value
float r = 0; // Last reward
public ObstacleAvoidance(byte actions, byte states) // constructor
{
state = new byte[states][4]; // 16 unique states, 4 unique SENSOR
action = new byte[actions][2]; // 4 unique actions with 2 actuators
initAction(action);
initState(state);
Q = new float[actions][states]; // table of Q-values for action-states
//setRandomValues(Q);
setLearnedQValues(Q);
}
public static void main(String [] args)
{
byte actions = 4;
byte states = 16;
long iterMax=100000;
ObstacleAvoidance qutil = new ObstacleAvoidance(actions, states);
ObstacleAvoidance qbrain = new ObstacleAvoidance(actions, states);
byte[] e = new byte[8]; // The environment values
2
byte [] motorCommand = new byte[2]; // Commands to send to motors
mLeftServo=IntelliBrain.getServo(1);
mRightServo=IntelliBrain.getServo(2);
for (long iter=0;iter<=3)
e[0]=1;
else
e[0]=0;
if(e[5]<=4)
e[1]=1;
else
e[1]=0;
if (e[6]<=3)
e[2]=1;
else
e[2]=0;
if (e[7]<=3)
e[3]=1;
else
e[3]=0;
}
public static void initAction(byte [][] state)
{
byte count = 0;
for(byte i=0;i<2;++i)
for(byte j=0;j<2;++j)
{
state[count][0] = i;
state[count][1] = j;
++count;
}
}
public static void initState(byte [][] state)
{
byte count = 0;
for(byte i=0;i<2;++i)
for(byte j=0;j<2;++j)
for(byte k=0;k<2;++k)
for(byte l=0;l<2;++l)
{
state[count][0] = i;
state[count][1] = j;
state[count][2] = k;
state[count][3] = l;
5
++count;
}
}
public static void setRandomValues(float [][] Q)
{
for(byte i=0;i=0)
{
r = getRewardValue(e);
a1 = getMaxAction(j);
Q[a][i] = Q[a][i] + LEARN_RATE * (r + Q[a1][j] - Q[a][i]);
// Q["+a+"]["+i+"] = " + System.out.println(""+Q[a][i]);
}
i = j;
// GREEDY
float rand = (float)Math.random();
if(rand > EXPLORE_RATE)
{
a = getMaxAction(j);
//System.out.println("Q["+a+"]["+j+"] = " + Q[a][j]);
}
// E-GREEDY
else
a = (byte)(Math.random() * 4);
buzzer.beep();
return a;
}
// find the largest Q-value for a given state (j), and return action
public byte getMaxAction(byte state)
8
{
float max = -1000;
byte action = 0;
for(byte a=0;a max)
{
max = Q[a][state];
action = a;
}
}
return action;
}
public byte getState(byte [] e)
{
byte count = 0;
for(byte i=0;i<2;++i)
for(byte j=0;j<2;++j)
for(byte k=0;k<2;++k)
for(byte l=0;l<2;++l)
{
if (e[0] == state[count][0])
if (e[1] == state[count][1])
if (e[2] == state[count][2])
if (e[3] == state[count][3])
return count;
++count;
}
return -1;
}
public float getRewardValue(byte [] e)
{
byte reward=0;
if(a==0)
reward += 3; // good reward
if (e[0] == 1||e[1] == 1||e[2] == 1||e[3] == 1)
reward -= 2; // give bad reward: penalize
return reward ;
}
}

To view the Result of the intellibrain Neural Network Programming on ai Robots. Click here

Look around you, be aware of natures unseen danger and these are GERMS.  Yes, germs can harm us and can cost us a lot of money for hospitalization.  How germs are transmitted from one place to another?  Most germs are spread through the air in sneezes or coughs or through bodily fluids like sweat, saliva, semen, vaginal fluid, or blood, and you yourself can transmit these germs to your family your best bet is to limit contact with those substances, as far as possible.  But hugging, talking, and limiting contact to family members is the hardest thing to do especially when you are sick, you need to communicate with them so they can help you.  I should say that the best bet is use the new ECOBOX, the new ecobox featuring ActivePure helps prevent the spread of germs by killing them on surfaces.  Take control of the situation and don’t let the situation control you.  Ecobox quietly sends a solution to the pollution, university studies say that ecobox can  kill up to 99.99% of bacteria, viruses, and mold on surfaces.

For issues concerning starting up a green home based business and other related topics, play the EcoQuest game and know more about Ecoquest’s background here.  While relaxing and playing the EcoQuest game, Ecobox is there to clean up the air .  As the doctor say “prevention is better than cure”, so get one and be safe…

This is really amazing that it really has become a reality…

Cellphone Controlled Robot

If you can get through the cell phone text speak, you’ll probably enjoy this cool tutorial on how to build a cell phone controlled robot. This bot decodes the key tones, similar to the automated phone systems we’ve all experienced. It uses a chip called a MT8870 DTMF decoder to translate the signal for the Atmega 16 controller or if you can find ways to use other microcontrollers such as Zilog or the PIC. The circuit diagram is pretty hard to read but I think it easier for you guys since you have experiences from designs and hacks on circuits, maybe we missed a downloadable one somewhere. The source code is available. It would be nice to get some feedback from the robot, so you aren’t driving it completely blind. This is similar to the Lego cell phone rover .  In the future this mobot will recognize voices to follow.

It’s so amzing that the Samsungs 1.8 inch 160GB HDD can fit medium size PMP’s.  The 1.8-inch 160GB HDD is described as “smaller than a business card” and has a spin rate of 4200 RPM.  The uses of this device besides PMPs to camcorders and DVRs are virtually endless.

Samsung’s HDD is so handy  that we can carry it anywhere we want to bring it and together with it is our files that are needed anytime we want to have it.