Now that we can move the robot, we need to be able to see where we are going. To give our robot eyes within flash, I first tried a series of images using the still image command from the Rovio api (/Jpeg/CamImg[value].jpg). This produced stuttering, unreliable quality video. I needed to get at the video stream that Rovio broadcasts, the RTSP stream (Rtsp://xxx.xxx.xxx.xxx/webcam). To get the video stream into flash it needed to be transcoded to flv. The go-between for the video was a Red 5 server which took in the live Rovio RTSP stream and output a live flv stream. Catching the flv stream was a netstream connected to a video object. This provided a much cleaner, smoother image for driving the robot.

Not only did the robot need eyes, he also needed to be able to recognize what he was looking at. The objective of the interactive was to have young children track down specific objects on a surface. To give them a prompt and to make the video of the surface a bit more interesting, a target was to be overlayed on the video pointing to where the object was. This meant that the Rovio would need to distinguish between the unique objects on the driving surface. So what set the objects apart from each other? More in the next post.

Recipe for robot eyes:
Wowwee Rovio
Red 5 video server
Flash netstream/video objects

links for 2010-03-07

• JavaScript Image Rotator – Timed Image Swap
  The Rotate Images JavaScript swaps images automatically on a timer. The code supports many options and features, such as:

  Random Rotation
  PHP Images to Rotator
  Random Image Display, Sequential or random rotation. Linked or unlinked images. Linked images can all have the same destination URL or each can specify its own. Opening sub-windows and executing custom function calls onclick are also supported by the code.
  Rotate titles or captions with image rotation.
  Multiple instances are supported.
  Customizable rate of rotation.
  Images of varying dimensions can be included in the rotation.
  Pause/resume onmouseover/mouseout.
  Optional transition filter for Internet Explorer on Windows.[1]
  PHP directory listing as source of images.

  (tags: javascript rotator image)
• slayeroffice | code | image cross fade redux
  Image Cross Fader – This version uses unobtrusive DOM scripting and semantic markup to achieve its goals, and is somewhat less abusive of the CPU in Firefox. It also works in Safari – the original version was written prior to Safari supporting the CSS3 "opacity" property. Opera doesn't support opacity, so the images will just flip in that browser.
  (tags: rotator javascript effects dom)
• Create A Tabbed Content Rotator Using jQuery
  Create a tabbed content rotator (not sure what to call it really) using everyone’s favorite Javascript framework, jQuery and an interface library called jQuery UI
  (tags: rotator jquery javascript)

“I dunno what it is, Just put it out there!”

Eddie Pittman pointed this out: Watch this video taken from an interview with Frank Zappa and substitute the word “Music” with Comics, Animation, Film, or any media of your choice.

Here is a snippet of what he said:

“One thing that did happen during the Sixties was some music of an unusual or experimental nature did get recorded or did get released. Now look at who the executives were in those companies at those times. Not hip young guys. These were cigar-chomping old guys who looked at the product that came and said, ‘I don’t know. Who knows what it is. Record it. Stick it out. If it sells, alright.’ We were better off with those guys than we are now with the supposedly hip young executives who are making the decisions of what people should see and hear in the marketplace. The young guys are more conservative and more dangerous to the art form than the old guys with the cigars ever were.”

More discussion on this can be found over on Cartoon Brew.

I’m speaking at D2WC! Designer and Developer Workflow Conference in Kansas City.

My summer is getting booked up. I heard recently that I will be speaking at D2WC in Kansas City. Woot!

From their site:

What is a workflow conference? It’s an opportunity for Web professionals to learn how to use their tools and collaborate with their teams more effectively. It’s two full days of intense real-world information from leading professionals in the business, including many Adobe Evangelists and Adobe Community Professionals.

I’m on the speaking bill with some great professionals that I admire and respect. Adam Flater, Andy Matthews, Andy Powell, Ben Stucki, Scott Janousek and so many others. It’s going to be a fantastic conference. It’s taking place from June 19-20. This two day event is discounted with Early Bird pricing of $175 (ends April 1). Tickets will be $225 after that. A great price for this much information.

I really love these smaller community centered events. They are a great opportunity to connect with regional professionals that share the same interests as you. Perfect!

On the move. Headed to mLearnCon in June. San Diego, here I come!

A brand spanking new conference put on by the eLearning Guild, mLearnCon is set to happen in San Diego in June. I just got heads up that my session was accepted, so expect to see me there. I’m totally psyched top be a part of this groundbreaking event. It’s is the first mLearning event of this scale that I have heard of, so I’m expecting a lot. I have been to several eLearning Guild events and been part of the organization for a couple years now, so I know that they put awesome happenings together. From their site:

This event focuses on every aspect of mobile learning including management strategies, platforms (SmartPhones, PDAs, iPods, Tablets, etc), operating systems (Android, BlackBerry, iPhone OS X, Palm, Symbian, Windows Mobile, etc.), authoring tools and technologies, content design & development, and best practices. Whether you are working in academic, corporate, government, or military settings … mLearnCon will give you the ideas, information and community your organization needs to succeed.

I’ll be presenting on “Building m-Learning With Your Existing Toolkit”. The session will be focused on providing you, as the mLearning designer and developer, creative ideas to leverage things you already know about media production and interactive design and development in creating mLearning. Many professionals get hung up on the idea that everything has to be an active application to make use of the device or be perceived as being fully functional. I have some great examples, some information to share and code snippets to show that will change the way you think about mobile development.

You should really consider registering for this one. It’s going to be amazing. The keynotes are brilliant, there are Pre-conference Certificate Programs, and hey… it is in San Diego.

links for 2010-03-16

• CSS Border Radius
  A simple, cool online helper for generating CSS3 rounded corners. Sweet
  (tags: css3 border-radius rounded)

HTML Vs. Flash – In Pixton Format

So here’s an interesting thought… could you create Pixton in HTML5? Could you make as good of a webcomic as RobotBeach in Pixton? Hmm.

The tool does not dictate the quality of the output.

With the newly acquired robot eyes, we were left with the question of what distinguishes the objects from each other?

Shape and size: Using these characteristics and constructing custom object (similar to facial) recognition using opencv and haartraining the different objects could be “seen” by the rover. This ended up being a balance of accuracy and speed, of which neither was good enough to use in the application.

Position: Using the rover’s position, both location and orientation, the specific object could be determined from its coordinates. The numbers coming back from the rover, through the beacon, ended up being too cryptic and not reliable enough to give specific positioning information, especially for moving a precise onscreen target. There was also no way to get vertical movement information.

Color: Pairing a unique color with the object to be identified could allow color detection to run the brain of the rover. This ended up being the most streamlined and accurate approach to identifying object in the interactive. Using a vibrant red, green and blue gave enough differentiation between the objects to bring up specific prompts for each target. Running the color detection on the video stream and then creating a box around the identified color gave a specific position in the video frame. The target could then be positioned in the center of the box, and exceptions could be made for outlying false positives. Color scraps could be disregarded and focus could be on heavy concentrations of the specific colors. Below you can see the blue box (hidden when not in debug mode) surrounding the green colored image.

Lighting greatly impacted the color detection on the objects. Above is a picture of two of the colored boxes blacked out by the background window light. The objects needed to be lit from a reliable, consistent source of light. This light would have to be pointed on the object or what was done in this case from within the object. The targets were lit with colored light to attempt to nullify lighting differences due to sunny or cloudy days (the interactive being in a heavily windowed area). Below are images against the window areas. First, the transparent gels of the three different colors (the red needed some help and has a light behind it). Second is a stronger light competing with the self lit color cone.

Update:

After using the previous version of color detection where one color was identified, it was determined that this was a fickle method given external lighting conditions. The color detection was updated with the following:

For aesthetic reasons, the object was to be internally lit. When in the final space, the bulb was not colorful enough and was whitewashed in the rover’s view. We tried to use the colored gels and other manipulations but due to the fact that the rover was looking for one specific color we were not able to tune it in enough to be accurate. Take two.

Now we were looking for two objects. First a strong white light source, which is what we had and were trying to fight the first time, and then look below the light for a color. The first method was based on the blob detection outlined here by Tomek Augustyn (Og2t). This takes the video object, draws it to a bitmap object and then creates another bitmap object using the threshold method finding the brightest spots in the video. After singling out the brightest spots, we take some measurements of the individual rectangles found with getColorBoundsRect(). If they meet the minimum and maximum requirements for size, of which in this case the minimum height is a ratio of the width, then we look for the next piece of information.

Below the bright spot is a colored ring that can be averaged out and determined to be in a specific range. Using the technique outlined here by Justin Windle we have averaged out the ring color and can ask if the RGB values are in the range that we are looking for:

if((colorin >> 16 & 0xFF) < 0x5f && (colorin >> 8 & 0xFF) > 0x84 && (colorin & 0xFF) < 0xc6){do something}

If the color is in the correct range then we are displaying the found target icon over the bright area. This gives a bit more flexibility and accuracy in finding the correct targets.

The Flash Mullet

By now, everyone has either dabbled in Augmented Reality, or chosen to abstain… for those of you just now picking up the familiar black and white Marker and FlarToolkit, beware…

Our Speaker List is Better than Your Speaker List

Seldom has a list of speakers this good been put together. Register for Flash And The City and come bask in the SWFfy goodness…