View unanswered posts | View active topics It is currently Wed Sep 17, 2014 6:44 pm






Reply to topic  [ 9 posts ] 
Q-Learning 
Author Message
Rookie

Joined: Mon Feb 13, 2012 5:17 am
Posts: 7
Post Q-Learning
Hello.
Is there anybody who used Q-learning on the NXT ? Are the memory and the CPU good enough for such an algorithm ?
I'd like some advises or some tips before giving it a try.
Thank you.


Thu Mar 08, 2012 1:52 pm
Profile
Moderator
Moderator
User avatar

Joined: Tue Sep 14, 2010 9:19 pm
Posts: 496
Post Re: Q-Learning
May people seem to have accomplished it. Take a look at this publication: http://www.cs.surrey.ac.uk/BIMA/People/ ... arning.pdf

_________________
sudo rm -rf /


Thu Mar 08, 2012 2:23 pm
Profile
Rookie

Joined: Mon Feb 13, 2012 5:17 am
Posts: 7
Post Re: Q-Learning
Amazing speed !!!
This document is very interesting. And answers my question :
Quote:
Even with the 320 KB of memory that the NXT supports, in order to implement both the heuristic and the adaptive algorithm for the robot, it was clear early in the design that the NXT’s memory would be insufficient for this program. However, with the program running entirely on the computer the Bluetooth link became a bottleneck. This can happen during any kind of motor movement when the rotation sensors are being queried to determine the amount of movement actually undertaken. This problem was addressed by distributing the processing, allowing part of the controller to run on the NXT and the other part to run on the computer. The standard NXT firmware and LEGO software (NXT-G) was used to program the brick to respond to Bluetooth commands, whilst the iCommand Java package, release 0.5, was used on the controlling computer in order to send and receive data through the Bluetooth link.

Thank you very much.


Thu Mar 08, 2012 2:41 pm
Profile
Rookie

Joined: Mon Feb 13, 2012 5:17 am
Posts: 7
Post Re: Q-Learning
For those interessed by Q-learning, I just found this good tutorial : http://mnemstudio.org/path-finding-q-learning-tutorial.htm


Thu Mar 08, 2012 2:54 pm
Profile
Moderator
Moderator
User avatar

Joined: Tue Sep 14, 2010 9:19 pm
Posts: 496
Post Re: Q-Learning
This probably doesn't contribute anything to the conversation, but I it's sort of relevant, so I'll say it anyway. I was programming a simple Q-learning algorithm a while back, I had this really weird moment when I kept thinking about robot ethics. I mean, in Q-learning, you'r basically programming the robot with rewards and punishments. It's like giving it pain or pleasure depending on the action. (I wasn't inebriated, I promise).

_________________
sudo rm -rf /


Thu Mar 08, 2012 9:36 pm
Profile
Novice
User avatar

Joined: Sat Jul 10, 2010 3:06 pm
Posts: 86
Location: Roanoke, VA
Post Re: Q-Learning
magicode wrote:
This probably doesn't contribute anything to the conversation, but I it's sort of relevant, so I'll say it anyway. I was programming a simple Q-learning algorithm a while back, I had this really weird moment when I kept thinking about robot ethics. I mean, in Q-learning, you're basically programming the robot with rewards and punishments. It's like giving it pain or pleasure depending on the action. (I wasn't inebriated, I promise).

Wow... you have interesting thoughts... :)
For sake of argument, though... (Even if it were an "ethical" question (I personally don't think robots are bound under the same moral code as humans).) People do the same thing to other people. How does a parent train a child? By rewarding them when they do right, and by punishing them when they do wrong. So, why would it be wrong to punish and reward a robot? :lol:

(This could get weird really quick)

//Andrew

_________________
Check out my website! www.RoboDesigners.com

VRC Team 2190

Twitter: @RoboDesigners


Thu Mar 08, 2012 10:20 pm
Profile WWW
Moderator
Moderator
User avatar

Joined: Tue Sep 14, 2010 9:19 pm
Posts: 496
Post Re: Q-Learning
RoboDesigners wrote:
Wow... you have interesting thoughts... :)
For sake of argument, though... (Even if it were an "ethical" question (I personally don't think robots are bound under the same moral code as humans).) People do the same thing to other people. How does a parent train a child? By rewarding them when they do right, and by punishing them when they do wrong. So, why would it be wrong to punish and reward a robot? :lol:

(This could get weird really quick)

Yes, interesting is a polite way to put it. And the punishment-reward system wasn't really the focus of what I was pondering about. It more more about how easy it was to change one of those values. With one keystoke, the 'pain' value of a certain action goes up by 50. I know it's not really a question of ethics; it was just a thought experiment.

_________________
sudo rm -rf /


Thu Mar 08, 2012 11:35 pm
Profile
Rookie

Joined: Mon Feb 13, 2012 5:17 am
Posts: 7
Post Re: Q-Learning
In my opinion, rewards and punishments, in education, are not the problem. Even in self learning you use rewards and punishments. When you succeed you're happy (reward), when you do a mistake you're sad or in angry (punishment).
The problem is the level of the punishment. All parents are confronted with this question.
But, so far, I've never confused my NXT and my children :D
[I hope my English is understandable :oops: ]


Fri Mar 09, 2012 5:33 am
Profile
Novice
User avatar

Joined: Sat Jul 10, 2010 3:06 pm
Posts: 86
Location: Roanoke, VA
Post Re: Q-Learning
Jean-Claude wrote:
The problem is the level of the punishment. All parents are confronted with this question.

This is true...
Jean-Claude wrote:
[I hope my English is understandable :oops: ]

It is. (In fact, it's better than some native English-speakers who have spent too much time texting on their phones...) :)

//Andrew

_________________
Check out my website! www.RoboDesigners.com

VRC Team 2190

Twitter: @RoboDesigners


Fri Mar 09, 2012 9:29 am
Profile WWW
Display posts from previous:  Sort by  
Reply to topic   [ 9 posts ] 

Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  



Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by ST Software for PTF.