File I/O in Flash memory (disk)

6 replies [Last post]
xquest
Title: NooBot+
Joined: 03/05/2010
Posts:
BotPoints: 8
User offline. Last seen 12 years 22 weeks ago.

Hey guys -

We, the IHS Electronics Academy Botball team, would like to know how we can change the disk permissions on the CBC. It has 2 GB of flash, is what we've been told, and we need to store two multidimensional arrays with potentially [very] high resolutions, so we're afraid that the arrays will not fit in main memory (incidentally, how much main memory does the CBC have? Our program is also pretty large). Our mentor has taught us basic file I/O in C, but KIPR told us that writing to the flash has been disabled.

Since it's a version of Linux on the CBC, shouldn't there be like, a chmod or equivalent?

Any help would be appreciated. Thanks in advance!

PiPeep
PiPeep's picture
Title: RocketBot
Joined: 07/19/2009
Posts:
BotPoints: 170
User offline. Last seen 8 years 23 weeks ago.

I think what KIPR meant was that you don't have permissions to write to the CBC's main flash (ROM) that manages the OS. However, you can write to the 2 gig flash drive without having to chmod or anything like that (as far as I understand). You should probably write to /mnt/user/code/[YOUR_PROGRAM_NAME] .

What kind of program is this? If you are writing an array that big, you are likely to experience slowdowns not only from IO, but also from simply processing it. The CBC has (I think) 64 megs of ram, which, although it doesn't sound like much, should be more than enough for any reasonably sized array.

PiPeep
PiPeep's picture
Title: RocketBot
Joined: 07/19/2009
Posts:
BotPoints: 170
User offline. Last seen 8 years 23 weeks ago.

Oh, and welcome to the community site! Feel free to join us on the IRC, irc.mintirc.net #botball, or the "Botballer's Chat" link at the top of the page.

xquest
Title: NooBot+
Joined: 03/05/2010
Posts:
BotPoints: 8
User offline. Last seen 12 years 22 weeks ago.

Thanks PiPeep! Actually we are worried about speed...the array stores a map of the environment, which they were thinking of breaking up into multiple arrays stored in files so as not to mess up by instigating cache misses (incidentally, how much cache memory does the CBC have? I know it's a "modified Harvard architecture" with separate instruction and data caches, so they want to know how small they should make each array). The array(s) would be accessed by a process that would dynamically update the maps and change flags in bitfields, sort of like a rudimentary interrupt system. I'm teaching them tips and tricks for speeding up array processing, but I wouldn't say no to other advice.

So you're saying they should use something like, FILE *fp = fopen("/mnt/user/code/[YOUR_PROGRAM_NAME]", "r+"); with the array files stored in the directory [YOUR_PROGRAM_NAME]? And how would one upload the array files to that directory from KISS-C?

Thanks again.

PiPeep
PiPeep's picture
Title: RocketBot
Joined: 07/19/2009
Posts:
BotPoints: 170
User offline. Last seen 8 years 23 weeks ago.

Sorry, I'm not really an expert on C, but I believe you would have to serialize the array.
"/mnt/user/code/[YOUR_PROGRAM_NAME]" is the directory that stores your program, so you would probably want to write to something like "/mnt/user/code/[YOUR_PROGRAM_NAME]/myLargeArray".
KISS-C doesn't have a system in place for uploading arrays, but you could do something like a reverse of this: http://github.com/catron/CBCJVM/blob/master/installer/install/CodeInstall.c, and copy your serialized array to a flash drive, which you could use on your computer.

matthewbot
Title: YAC
Joined: 04/12/2009
Posts:
BotPoints: 94
User offline. Last seen 12 years 5 weeks ago.

I'd take a look at the mmap() function. It will let you make a large file appear to be entirely in memory as an array, but behind the scenes Linux will efficiently handle on demand loading of pages of the file and writing of any changes you make to that memory back to the disk. Performance varies depending on how much memory is free on the system, but it does work even if there is not enough memory to hold the entire file at once.

As for reading and writing to flash, /mnt/user is writable. I don't remember how much space that partition has, but I think its less than a gigabyte. If you mount a thumb drive in the file manager, it will appear in /mnt/browser/usb.

--
Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius -- and a lot of courage -- to move in the opposite direction.

Albert Einstein

Project Quadcopter: http://quadcopter.wordpress.com/

axyjo
Title: NooBot+
Joined: 03/28/2010
Posts:
BotPoints: 9
User offline. Last seen 12 years 20 weeks ago.

Are you guys trying to use those arrays for things like Dijkstra or A*?