View unanswered posts | View active topics It is currently Mon Apr 21, 2014 1:00 am






Reply to topic  [ 16 posts ]  Go to page 1, 2  Next
Out of memory? 
Author Message
Rookie

Joined: Sat Mar 24, 2012 4:33 pm
Posts: 2
Post Out of memory?
I have been playing around with RobotC and for some reason, I keep getting "out of memory for variable allocation". For example this test code:

Code:
typedef struct
{
  ubyte a;
  ubyte b;
  ubyte c;
  ubyte d;
 
} myStruct;

myStruct firstArray[50][50];
myStruct secondArray[50][50];


I define two 50x50 arrays of struct, each struct contains 4 bytes. So 2*50*50*4 = 20kB, which is less than a third of NXT RAM (64kB). Why can't I use whole memory?


Sun Mar 25, 2012 9:39 am
Profile
Moderator
Moderator
User avatar

Joined: Wed Mar 05, 2008 8:14 am
Posts: 3107
Location: Rotterdam, The Netherlands
Post Re: Out of memory?
The firmware also eats up a good chunk of RAM, so does the display memory and several other bits and bobs. I think you have about 15K variable space, excluding constants, code, etc.

- Xander

_________________
| Professional Conduit of Reasonableness
| (Title bestowed upon on the 8th day of November, 2013)
| My Blog: I'd Rather Be Building Robots
| ROBOTC 3rd Party Driver Suite: [Project Page]


Sun Mar 25, 2012 12:30 pm
Profile WWW
Expert

Joined: Thu Sep 29, 2011 11:09 pm
Posts: 184
Location: Michigan USA
Post Re: Out of memory?
Why would ROBOTC only have 15k available? NXC (with John's EFW) allows up to 32k.

_________________
Matt


Mon Mar 26, 2012 3:08 pm
Profile WWW
Rookie

Joined: Fri Jan 27, 2012 6:57 pm
Posts: 40
Post Re: Out of memory?
Perhaps someone from the ROBOTC development team could post a simple explanation/diagram showing how much of the memory on the NXT is used for what? :-)


Mon Mar 26, 2012 3:12 pm
Profile
Moderator
Moderator
User avatar

Joined: Wed Mar 05, 2008 8:14 am
Posts: 3107
Location: Rotterdam, The Netherlands
Post Re: Out of memory?
I dropped them a note to let them know about the thread.

- Xander

_________________
| Professional Conduit of Reasonableness
| (Title bestowed upon on the 8th day of November, 2013)
| My Blog: I'd Rather Be Building Robots
| ROBOTC 3rd Party Driver Suite: [Project Page]


Mon Mar 26, 2012 3:17 pm
Profile WWW
Rookie

Joined: Sat Mar 24, 2012 4:33 pm
Posts: 2
Post Re: Out of memory?
What is the point of having NXT if you can only access 1/4 of his RAM? It seems like very bad concept that firmware consumes most of the RAM. For example leJOS is JAVA, but it AFAIK still gives user over 50kb of free memory. What complicated stuff must RobotC have to take up so much?


Mon Mar 26, 2012 4:10 pm
Profile
Site Admin
Site Admin

Joined: Wed Jan 24, 2007 10:42 am
Posts: 592
Post Re: Out of memory?
I'll let Dick, our firmware expert comment on this - But as of today there is a limit of 15K for program variables in ROBOTC.

Maybe unlike other software packages out there, ROBOTC does use some of the RAM space for it's own purposes (i.e debugger functionality, system variables and parameters, buffers, LCD remote display, messages, NXT OS menus, etc.)

There may be some old functionality that we might be able to squeeze more space available, but I doubt you're ever going to have access to the full 64K unless you're writing code at the ARM-processor level.

If you're specifically unhappy about this, some of the other forum users have mentioned other programming languages for the NXT that you may want to try out for your specific project.

_________________
Timothy Friez
ROBOTC Developer - SW Engineer
tfriez@robotc.net


Mon Mar 26, 2012 4:16 pm
Profile
Creator
Creator

Joined: Fri Feb 09, 2007 9:21 am
Posts: 614
Post Re: Out of memory?
There's lots of functionality found in the ROBOTC firmware that is not found in the other programming solutions. Many of these take significant chunks of RAM. Some of the examples:
  • A fairly large buffer for the "Debug Stream" feature. User programs can write to the Debug Stream which eventually shows up like a terminal window within the ROBOTC IDE. The buffer allows user program to write data even when no PC connection is available and then ROBOTC IDE will "catch up" (i.e. flush buffer) when it next establishes a connection to the robot.
  • AFAIK none of the alternative solutions provide anything close to the interactive debugging functionality of the ROBOTC IDE. There's several substantial RAM buffers associated with the Debugger. PS Don't try to compare the primitive debugging support in some of the alternatives with ROBOTC; nothing else can do four complete debugger window refreshes (all motor status, all sensor status, all visiable variables, etc) in one second.
  • There's several K of RAM reserved for the Datalog buffers.
  • There's several K of RAM reserved for upcoming feature that better utilizes task stacks (20 tasks x 256 byte stack).
  • There's a couple of K of RAM dedicated to support of the HiTechnic TETRIX Motor and Servo Controllers. None of the alternatives (except LABVIEW) support LEGO's TETRIX product.

There's other features that chew up RAM. Like the Flash write routines. I know, you're wondering why flash write routines chew up several K of RAM when single flash sector is only 512 bytes. It's simple:
  • NXT CPU does not support reading from flash while a flash write is in progress. This is a fairly common restriction.
  • If an interrupt occurs during flash write, it cannot be handled until the flash write is processed. Unless the "vector table" and the interrupt handler are stored in RAM.
  • You want interrupts to work during flash write -- so encoders don't drop counts, so USB link doesn't fail due to timeout, so BT link doesn't fail due to timeout, etc.I suspect some of the alternative programming solutions may not have fully relocated all the interrupt handlers into flash.

The ROBOTC firmware Virtual Machine is 5 to 20 times faster than the VM implmentation in the other alternative programming environments (i.e JAVA, NXT-G and derivatives like NXC). One of the ways this performance enhancement is achieved is by relocating about 1K of frequently executed functions into RAM; there's no memory wait state on RAM access but one (or two?) wait states on falsh memory access.

As I recall, about 25% of the RAM is chewed up for various operating system buffers and I/O maps. ROBOTC takes about the same size for this as alternative solutions.

Hope above gives you some insight into RAM usage with the ROBOTC VM.

And yes, there's also several K of unused RAM that's been reserved for future features. One of those features is being released soon involving some really exciting functionality for robotic swarms using standard Wi-Fi peripherals. And also provides direct web access to the NXT.


Tue Mar 27, 2012 4:13 am
Profile
Expert

Joined: Thu Sep 29, 2011 11:09 pm
Posts: 184
Location: Michigan USA
Post Re: Out of memory?
Thanks for the thorough explanation!

Shouldn't some of those be optional though? For example, if a user didn't need the debug stream, datalog, or tetrix controllers, they might prefer to use that RAM for user-variables.

_________________
Matt


Tue Mar 27, 2012 12:55 pm
Profile WWW
Creator
Creator

Joined: Fri Feb 09, 2007 9:21 am
Posts: 614
Post Re: Out of memory?
Quote:
Shouldn't some of those be optional though? For example, if a user didn't need the debug stream, datalog, or tetrix controllers, they might prefer to use that RAM for user-variables.

One of the ease of use features for ROBOTC is that there is only a single firmware file for each supported platform. Which requires it to be an "all inclusive" feature offering.

Couple that with the fact that while all the various NXT firmware (ROBOTC, NXT-G, NXC, LeJos) are written in C code that does not support malloc and free primitives. Which essentially requires all RAM to be statically allocated. NOTE: some of these may support a 'malloc' function but the 'free' function does nothing and memory is not garbage collected. I think you'll find this restriction is the root cause of some of the other implementations crashing when too many 'malloc' (including hidden instances for some native primitives like "sprintf" equivalent) and why they require a restart between every user program invocation.

Regardless of the above points, I do agree with you that it would be nice if some of the large RAM usage features were optional. But this appears to be of interest to only a very small set of ROBOTC users and will likely never be high enough priority to warrant implementation.


Tue Mar 27, 2012 2:08 pm
Profile
Rookie

Joined: Mon Mar 19, 2012 6:00 pm
Posts: 6
Post Re: Out of memory?
One of the many reasons virtual machines or memory intensive libraries are not used on embeded systems. Whenever I am working close to computer hardware I stear clear of virtual-machines.

Ofcourse, it is much more justifiable to have a virtual machine on something like the Vex cortex when you consider the audience & the general code-size. If you think you've outgrown the managed environment, and want more control, consider compiling in to nativley executable code via EasyC (or another C compiler.)

I am not sure exactly how much room you have with the EasyC firmware - but with several tests, I was able to get about 29kb to 32kb from the heap before a stack & heap collision. Obviously, that means you have quite a bit of room between the stack & the heap. (unlike robotC, variables are stored on the stack-frame, so this means local variables as well as those dynamically allocated.)

@Dick Swan I was unaware that free was not supported (malloc is). Regardless, would it really be that hard to have a small bitmap to keep track of the memory? The standard (or rather, partially so) C library that comes with easyC does implement free and malloc(which works). I am not 100% sure about your claim that free isn't supported but I can run a few tests to see if it is. If it isn't, why couldn't one init a simple bitmap to keep track of which memory is and isn't allocated?


Last edited by jeremy__ on Wed Mar 28, 2012 5:42 pm, edited 5 times in total.



Wed Mar 28, 2012 5:21 pm
Profile
Expert

Joined: Wed Jun 30, 2010 7:15 am
Posts: 181
Post Re: Out of memory?
Dick Swan wrote:
But this appears to be of interest to only a very small set of ROBOTC users and will likely never be high enough priority to warrant implementation.
I'd definitely be interested; how many people would need to support this idea for it to become a priority/ something that might happen in the near future? And thank you for the explanation too, I've been wondering about it for a while.

_________________
leonoverweel.com


Wed Mar 28, 2012 5:33 pm
Profile WWW
Creator
Creator

Joined: Fri Feb 09, 2007 9:21 am
Posts: 614
Post Re: Out of memory?
Quote:
@Dick Swan I was unaware that free was not supported (malloc is). Regardless, would it really be that hard to have a small bitmap to keep track of the memory? The standard (or rather, partially so) C library that comes with easyC does implement free and malloc(which works). I am not 100% sure about your claim that free isn't supported but I can run a few tests to see if it is. If it isn't, why couldn't one init a simple bitmap to keep track of which memory is and isn't allocated

Since this is a "ROBOTC for MINDSTORMS" sub-forum I was referring to the altnerative LEGO programming environments and not the VEX Cortex solutions. I've never checked to see the EasyC limitation. But since you've mentioned EasyC there are a couple of significant differences in ROBOTC vs EasyC that impact RAM usage.
  • ROBOTC supports multi-tasking; I believe the limit is 20 tasks. EasyC does not and only has a single task. Each task has its own stack.
  • EasyC does not support debugger functionality (single step, breakpoints, suspend / resume, read/write variables, read/write sensors). EasyC does have a crude I/O pin display that shows whether a pin has value 0 or 1; this is different from an intelligent sensor display that aggregates two or more pins into a intelligent display of quadrature encoder value or ultrasonic range sensor value.
  • EasyC does have a "write to terminal window" which is somewhat similar to ROBOTC's debug stream. But I don't believe it has the large internal RAM buffer for data that handles the case when no communications link to PC is available. And I think EasyC "write to terminal window" can potentially stall user's program execution ROBOTC will not.


Wed Mar 28, 2012 8:32 pm
Profile
Rookie

Joined: Mon Mar 19, 2012 6:00 pm
Posts: 6
Post Re: Out of memory?
Dick Swan wrote:
Quote:
@Dick Swan I was unaware that free was not supported (malloc is). Regardless, would it really be that hard to have a small bitmap to keep track of the memory? The standard (or rather, partially so) C library that comes with easyC does implement free and malloc(which works). I am not 100% sure about your claim that free isn't supported but I can run a few tests to see if it is. If it isn't, why couldn't one init a simple bitmap to keep track of which memory is and isn't allocated

Since this is a "ROBOTC for MINDSTORMS" sub-forum I was referring to the altnerative LEGO programming environments and not the VEX Cortex solutions. I've never checked to see the EasyC limitation. But since you've mentioned EasyC there are a couple of significant differences in ROBOTC vs EasyC that impact RAM usage.
  • ROBOTC supports multi-tasking; I believe the limit is 20 tasks. EasyC does not and only has a single task. Each task has its own stack.
  • EasyC does not support debugger functionality (single step, breakpoints, suspend / resume, read/write variables, read/write sensors). EasyC does have a crude I/O pin display that shows whether a pin has value 0 or 1; this is different from an intelligent sensor display that aggregates two or more pins into a intelligent display of quadrature encoder value or ultrasonic range sensor value.
  • EasyC does have a "write to terminal window" which is somewhat similar to ROBOTC's debug stream. But I don't believe it has the large internal RAM buffer for data that handles the case when no communications link to PC is available. And I think EasyC "write to terminal window" can potentially stall user's program execution ROBOTC will not.


Don't get me wrong, I fully understand why RobotC's firmware is very heavy; the amount of dependancy on the VM is very large. The question is: is it bloated with features users aren't using? Also, I didn't mean to compare EasyC to RobotC, but rather an unmanaged to managed environments on embeded systems.

  • First of all, let me just say, that without synchronization objects, I strongly disagree with 99.99% of the instances in which people are multi-threading. Multi-threading requires one to program the interrupt timer; when it raises an interrupt, have the ISR push the registers used by the current thread to the heap, setup the register for the next thread and ofcourse keep track of a couple other threads. That doesn't take a lot of memory allocation with 5 or six threads. Ofcourse, it does when you have users abusing the threading system with 20 threads and 20 stacks all allocating the same amount of memory (even though most RobotC programs using that many threads have _very_ simple tasks (loops polling devices) and use very little of the stack) on a single-cored system. With all that said, it is still possible to implement a multi-threaded design in easyC by either following the method described above, or more easily, using RegisterRepeatingTimer(technically not multi-threading, but IMHO sufficient.
  • True, but software based testing isn't hard to implement when you use a well established standard programming language like C. It is literally adding the source\header files to a Microsoft Visual C++ (or any other C++\C compiler) and compiling the solution. You can also implement the your dependancies' APIs yourself (and have them simply print to the screen.) I can see why debugging on the device can be more accurate, but it is much quicker and easier to resolve logic errors in your code via a compiler independant of the device in question with very well developed debuggers. Most (especially at this level) errors in the robot's code can be resolved with this method very quickly.
  • I don't understand why the speed in which one can place information on to the terminal for debugging purposes is really that critical. With that said, programming devices like motors & polling sensors isn't exactly super time critical w\ VEX since they take a while to physically respond to the requests anyway (especially when you're debugging the module.)

Also, just out of curiosity, why can't you use a bitmap to keep track of allocated memory? If malloc is supported (and not free) Can't you just allocate all the memory and then keep track of it on your own; setup a bitmap that determines which chuncks of the memory are and are not allocated.

*edit*

Also I would really advice against dedicating 20kb to a single module @OP. Spreaking relativley, that is a lot of memory.


Wed Mar 28, 2012 10:12 pm
Profile
Creator
Creator

Joined: Fri Feb 09, 2007 9:21 am
Posts: 614
Post Re: Out of memory?
Responding to some of the points in the last post.

Almost all VEX and FIRST "tele operation" / "user control" solutions are multi-tasked! Theve have two tasks which are represented by the two human drivers. Notice I used the word "solution" and not "program" to start this paragraph.

ROBOTC VM does support synchronization primitives. They were added sometime in the last year.

I don't think it is easy to add "proper" multi-tasking to a system that has not been designed from the beginning for multi-tasking. My definition of "proper" multi-tasking includes independent tasks with a scheduler that shares CPU cycles among different runnable tasks; ROBOTC has a round robin scheduler for runnable tasks at same priority level and the highest runnable priority task(s) are always executed. I've analyzed the alternative environments for the platforms supported by ROBOTC and none of them had this functionality. Some of the weaknesses included only single task priority, non-runnable tasks burned CPU cycles, timeslices way too long, or simply didn't support multi-tasking.

Regarding "print to PC terminal" screwing up user program. Many users simply implement "print to terminal" commands in every cycle through their main loop which can seriously slow down the main loop. At least one of the alternatives would stall user program execution until "print to PC terminal" passed. The stalling occurred because on interrupt based systems buffer was full and on the non-interrupt solution polling was done on "terminal UART". So writing 100 characcters to "PC terminal" at 100Kbps would take 10 msec. For many programs (e.g. a simple line follower with a 1" wide line and robot moving at 1 meter per second) 10 msec is enough time to break the line following program; when the "print to PC terminal" is enabled you also may need to slow the line following speed to 1/2 or 1/4 of normal. Recall that "print to terminal" will not be the only multi-msec delay.

The "beauty" of the ROBOTC debugger is that:
  • It's always present without having to recompile and redownload your program after enabling conditional compile flags for "print user written debugging information to PC terminal" when you need to perform detailed debugging.
  • Unlike "print to terminal" statements, it doesn't degrade user program performance and works silently in the background. [A good use of multi-tasking, albeit hidden from end user].

The typical ROBOTC end user is not familiar with Visual Studio and wouldn't want to go through the learning curve to debug their program on the PC. Which is one of the reasons that ROBOTC introduced a "emulator" mode about a year ago for testing on PC without requiring a real robot.

I agree with you that Visual Studio represents the "gold standard" for GUI based debugging systems. I work with a number of commercial embedded system programming environments -- IAR, AVR Studio (using GCC tool chain), MPLab (using Microchip's tool chain), Resesas's HEW (using proprietary tool chain), Zilog tool chain. All the commercial embedded tool chains require a separate hardware JTAG unit ($50 to $1000 price) for the debugger. The IAR tool chain ($5K per seat) is comparable to Visual Studio. ROBOTC functionality and user interface is better than most of the rest.


Wed Mar 28, 2012 11:34 pm
Profile
Display posts from previous:  Sort by  
Reply to topic   [ 16 posts ]  Go to page 1, 2  Next

Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  



Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by ST Software for PTF.