Camera Latency

14 replies [Last post]
matthewbot
Title: YAC
Joined: 04/12/2009
Posts:
BotPoints: 94
User offline. Last seen 12 years 4 weeks ago.

So, I'd just like to know, from anyone whos willing to share, how are you using the camera? We've found it to be extremely challenging because the camera has about a half second of latency. When we took our XBC code over the CBC, it missed just about every time, driving far past the object before the object even begins to move on the CBC screen, then losing sight of it. Since then, we've basically slowed it down a ton, and employed forward-prediction algorithm to attempt to predict the objects actual position, meaning it works most of the time but still has to be run painfully slow when compared to last years bot.

Anyone else come upon a different approach? Also if anyone from KIPR reads this, I know you've probably gotten inquiries before, but are there any changes with regards to the camera for next year? Any chance to return to the CMUCam? Less features, but latency is simply crippling, in our experience at least.

--
Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius -- and a lot of courage -- to move in the opposite direction.

Albert Einstein

Project Quadcopter: http://quadcopter.wordpress.com/

tmac721
tmac721's picture
Title: NooBot+
Joined: 04/14/2009
Posts:
BotPoints: 16
User offline. Last seen 12 years 23 weeks ago.

Yeah we've basically run into the same problems. We pretty much gave up on using the camera for anything crucial after we discovered the lag. We basically only use the camera to check for the presence of objects and use a huge msleep after a single track_update. We haven't found a way to be constantly updating the camera as we move with any usable speed yet either.

matthewbot
Title: YAC
Joined: 04/12/2009
Posts:
BotPoints: 94
User offline. Last seen 12 years 4 weeks ago.

Yeah, ditto with the sleep. During one meeting we spent like 45 min determining why a camera track worked alone but not when called in the program, then we realized that the camera was still seeing the motions it took the robot to get in position :/

--
Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius -- and a lot of courage -- to move in the opposite direction.

Albert Einstein

Project Quadcopter: http://quadcopter.wordpress.com/

catron
catron's picture
Title: ZoomBot
Joined: 06/02/2009
Posts:
BotPoints: 98
User offline. Last seen 10 years 24 weeks ago.

We are using the camera only for seeding, as we can guarantee things won't be moving around *too* fast. We talked to Jorge about the issue, and he is going to try updating the GCC version to 4.3.2.. He thinks this *may* fix some of the driver bugs.

"When you do things right, people won't know you've done anything at all."

matthewbot
Title: YAC
Joined: 04/12/2009
Posts:
BotPoints: 94
User offline. Last seen 12 years 4 weeks ago.

IMO that won't really do anything, even on my linux desktop the very newest version of the driver has the same latency. :/

--
Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius -- and a lot of courage -- to move in the opposite direction.

Albert Einstein

Project Quadcopter: http://quadcopter.wordpress.com/

catron
catron's picture
Title: ZoomBot
Joined: 06/02/2009
Posts:
BotPoints: 98
User offline. Last seen 10 years 24 weeks ago.

Is the driver open sourced? It might be worth a look.

"When you do things right, people won't know you've done anything at all."

matthewbot
Title: YAC
Joined: 04/12/2009
Posts:
BotPoints: 94
User offline. Last seen 12 years 4 weeks ago.

Best I can tell they're using an older version of this driver: http://groups.google.com/group/microdia. It picks up the webcam on my system, but there could always be another driver that supports it.

--
Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius -- and a lot of courage -- to move in the opposite direction.

Albert Einstein

Project Quadcopter: http://quadcopter.wordpress.com/

kiprjorge
Title: KIPR Staff
Joined: 06/10/2009
Posts:
BotPoints: 7
User offline. Last seen 13 years 7 weeks ago.

Hey guys, Jorge here from KIPR. You're right, we're using an older version of the Microdia driver to run the camera, but we had sort of hack it up last minute to get it running. Most of the speed issues have to do with the USB Host drivers on the main board, which are apparently not very stable. We're re-building and testing the software with the GCC 4.3.2 compiler in hopes that some of these issues will be solved. Also, we haven't played with the auto exposure/white balance to see what effect that has, but I'm guessing a fairly strong one.

At any rate, look forward to all of the source for this stuff being posted within the next week or so, in case any of you want to dig in and play with it. Remember though, if you brick your board trying to make modifications/mess with the internal software, that *won't* be covered under our warranty.

I'll make a forum post when the source is up :)

catron
catron's picture
Title: ZoomBot
Joined: 06/02/2009
Posts:
BotPoints: 98
User offline. Last seen 10 years 24 weeks ago.

I CAN'T WAIT!!!! I have been waiting for the source for about 4 months :P Thanks to Jorge and KIPR for all of their efforts!

Jorge: Will you be accepting patches?

"When you do things right, people won't know you've done anything at all."

awright
awright's picture
Title: Roboticist
Joined: 07/06/2009
Posts:
BotPoints: 30
User offline. Last seen 13 years 5 weeks ago.

@matthewbot: Do you really mean "CMUCam" or are you referring to the vision implementation for the XBC? They're two very very different things. (For one thing, I implemented the latter, and someone else perpetrated the former. ;)

Jeremy Rand
Jeremy Rand's picture
Title: Botball Youth Advisory Council
Joined: 04/03/2009
Posts:
BotPoints: 1168
User offline. Last seen 7 years 12 weeks ago.

I believe Matt is referring to the CMUCam, since the XBC's vision system relied heavily on the FPGA, making it unfeasible to add to the Chumby. Personally I hated the CMUCam back when it was in Botball; I think the XBCCam was pretty much a near-ideal vision system. Am I correct in assuming that an XBC-like setup would be very hard to add to the CBC? From my reading of libxrc, it looks like the FPGA handles segmenting while the ARM7 handles the rest; is this correct?

-Jeremy Rand
Senior Programmer, Team SNARC (2012-2013), Norman Advanced (2010-2011), Norman HS (2008-2009), Norman North (2005-2007), Whittier MS (2003-2004)
2012-2013 VP of Tech, 2011 President, Botball YAC (2009-2013)
Mentor, Alcott and Whittier MS