Now that we can move the robot, we need to be able to see where we are going. To give our robot eyes within flash, I first tried a series of images using the still image command from the Rovio api (/Jpeg/CamImg[value].jpg). This produced stuttering, unreliable quality video. I needed to get at the video stream that Rovio broadcasts, the RTSP stream (Rtsp://xxx.xxx.xxx.xxx/webcam). To get the video stream into flash it needed to be transcoded to flv. The go-between for the video was a Red 5 server which took in the live Rovio RTSP stream and output a live flv stream. Catching the flv stream was a netstream connected to a video object. This provided a much cleaner, smoother image for driving the robot.

Not only did the robot need eyes, he also needed to be able to recognize what he was looking at. The objective of the interactive was to have young children track down specific objects on a surface. To give them a prompt and to make the video of the surface a bit more interesting, a target was to be overlayed on the video pointing to where the object was. This meant that the Rovio would need to distinguish between the unique objects on the driving surface. So what set the objects apart from each other?

Shape and size: Using these characteristics and constructing custom object (similar to facial) recognition using opencv and haartraining the different objects could be “seen” by the rover. This ended up being a balance of accuracy and speed, of which neither was good enough to use in the application.

Position: Using the rover’s position, both location and orientation, the specific object could be determined from its coordinates. The numbers coming back from the rover, through the beacon, ended up being too cryptic and not reliable enough to give specific positioning information, especially for moving a precise onscreen target. There was also no way to get vertical movement information.

Color: Pairing a unique color with the object to be identified could allow color detection to run the brain of the rover. This ended up being the most streamlined and accurate approach to identifying object in the interactive. Using a vibrant red, green and blue gave enough differentiation between the objects to bring up specific prompts for each target. Running the color detection on the video stream and then creating a box around the identified color gave a specific position in the video frame. The target could then be positioned in the center of the box, and exceptions could be made for outlying false positives. Color scraps could be disregarded and focus could be on heavy concentrations of the specific colors. Below you can see the blue box (hidden when not in debug mode) surrounding the green colored image.

Lighting greatly impacted the color detection on the objects. Above is a picture of two of the colored boxes blacked out by the background window light. The objects needed to be lit from a reliable, consistent source of light. This light would have to be pointed on the object or what was done in this case from within the object. The targets were lit with colored light to attempt to nullify lighting differences due to sunny or cloudy days (the interactive being in a heavily windowed area). Below are images against the window areas. First, the transparent gels of the three different colors (the red needed some help and has a light behind it). Second is a stronger light competing with the self lit color cone.

Recipe for robot eyes:
Wowwee Rovio
Red 5 video server
Flash netstream/video objects
Color detection

A year ago I published a post about widgets coming to the television. A year later I have widgets on my TV. Now that I have experienced the widgets first hand, there is not much change to my feelings about the basics of the previous post.

There is still no bridge between the cable content coming over the wire and the widgets on screen. Widget content cannot be populated by on-screen cable content. This leaves out advertising tie-ins and related content that would be automatically populated.

There are however widgets for pulling video content from online. This enables the content to interact with additional features. The main downside to the online content is the timeliness of the content in comparison to traditional counterparts. There is very little to no live television content being delivered through the widget interface, or anywhere online. Any online movie rentals or purchases are delayed by at least a month compared to in-store movies.

HD is another problem area for online content. There are allot of bits to move for HD content and until broadband infrastructure is in place to handle the content, a coaxial cable is the only option.

Online content delivery still needs time to develop and find a viable business model. Timely content will not be allowed to move to an online market unless it promises not to threaten the more profitable traditional models, and until timely content is delivered people will still want cable/satellite and physical dvd rental.

Using a keyboard to navigate a virtual space; be it the internet, a game or some other format, is familiar to most adults and older children. The Wowwee Rovio (rover) can be driven by such keyboard controls or mouse clicks with the built-in interface. Couple of problems with this however:

1. The interactive needed a custom interface so the built-in web interface (and keyboard controls) would not work.
2. The interactive needed to be accessible to children as young as 3 years old.

The custom interface was built with ActionScript3 and so to move the rover, the api was used. With the api, the rover could be sent movements via http requests. These api requests needed to be mapped back to user input and so the web interface was recreated in with as3. The ASDW keys were mapped back to specific movement requests via the api. This is a specific piece of code initiated by a keypress that gets sent to the rover:

switch(e.keyCode){
//forward
case 87: //W key
drive = 1; //1 maps to forward
speed = 1;
request.data = "Cmd=nav&action=18&drive=" + drive + "&speed="+speed; //action 18 is manual drive mode
loader.load(request);
break;
}

The drive variable signals the command/direction for the rover (forward, backward, rotate right) and the speed variable is how fast the command should be executed by the rover. Sending one of these http requests would move the rover a fraction of an inch. Mapping each key to a specific movement and sending multiple commands successively reproduces the motion of the built-in interface. This does not remedy our problem of simplifying the interface for 3 year olds. This is where the custom hardware comes in.

Our custom hardware consists primarily of a joystick but also includes some auxiliary buttons for game functionality which I will describe in a following post. The hardware came from Suzo Happ and also included a UGCI board for USB interface. The UGCI board had an included key-mapping utility which was used to map the joystick and button contacts to key presses. A constant connection with the joystick in any direction sent a repeating key press rather than a key up and down.

With key presses mapped to the joystick control and to the rover’s api through actionscript, the rover could now be driven with custom joystick controls. Now if only our rover could see…

Over the past months at work I have been taking a consumer mobile webcam (robot) and converting it to custom interactive experience. Over the next couple of weeks I will be talking about some of the finer point of taking an off-the-shelf WowWee Rovio (which comes with some fine built-in features) and transforming it into something much more. I will be talking about the following modifications in the coming weeks:

• Custom hardware controls
• Processing the video stream in Flash
• Object detection
• Docks and Beacons

This is on the side of the Rovio box:

I will be proving this incorrect.

I was recently putting together a test plan for some interactives that we are deploying in the next couple of months. As I was putting the pieces together I was realizing that software testing is very similar to something I had previously done, writing educational goals for children with disabilities. I guess I have a unique work history that would reveal this fact but it may be interesting for some in either profession.

First the condition must be set. This would be the circumstances under which the assessment is performed. This could be further broken down for software into the current state of the application and the action that the user takes to initiate the test. In regards to instruction this could be the classroom conditions or the specific format for the test (worksheet, one-on-one, with a behavioral prompt).

Next is the specific behavior that is to be observed. This should be something that is both observable and measurable. While the tests that we are performing are higher level tests (functional/usability), this could be applied simpler unit tests that would be measured by simple logic (does 2 + 2 evaluate to be 4?). Seeing that a video plays without visual stuttering would be observable. This behavior could however use some clarifying in terms of what the visual stuttering would be defined as, or would be left to subjective opinion, which may at some point be acceptable. In the educational realm, the behavior could be a students verbal response to a behavioral prompt or responding to a question with a correct answer.

The final relevant similarity is the criteria used to measure success. In most cases with software testing, the only acceptable criteria is pass or fail, but in some circumstances this could be measured with a success rate/percentage.

Given [condition] the application/student will [behavior] at [criteria]

At it’s basic level both of these tests are about quantifying behavior and removing subjectivity as much as possible. Someone unfamiliar with the inner workings of the application or the student should be able to record the same results as someone who is familiar with the subject.

We are a constantly stimulated people who multitask to the nth degree. We text and instant message and email all while we are having a conversation with the person standing right in front of us. Why then would we not want to bring more content to the largest screen in our house, the television. News of widgets being built into HD TVs for extra news, weather, sports, etc. seems like a nice option and I will admit when I saw it I was attracted. These widgets can bring extra content to your big screen and possibly tie into the content your currently watching. A commercial could bring not only the advertisement of the product but a chance for an impulse buy right there online. You could access character bios for your favorite sci-fi series or bring up a map of the area for the travel show your watching. Now while much of what I have just described is extrapolation from the current technology available, the key point is that television is running the show. Content originates from cable or satellite and extra information is added on.

Why not take it a step further and remove a middle man in the process. Content is created by artists, journalists, and other professionals and then bundled on television. If you want Comedy Central, your going to get the home shopping network as well. If cable is not willing to a la carte or even if they are would we not have more control over our consumption of content if our television is powered by the internet? All of the widgety type functionality I mentioned earlier could be achieved much more seamlessly with an internet delivery than with cable/satellite trying to bundle more services. Television content on the internet is very available but lacks the live feature of television. Being able to come to work the next day and discuss last night’s 30 Rock is not and option if you have to wait for it to show up on Hulu. It seems to me that this content delivery model is in for an overhaul.

For good or for bad the television experience, at some point, involves plopping down on the couch, turning on the television and flipping. Having to not make a decision of what to watch before getting a 3 second preview. In what I hope will be a new emerging media delivery model, streaming internet video does not offer this method of viewing. This is not the only shortfall of bringing the masses into the fold of internet television (hardware being the largest) but it is something to consider. My proposal: have a favorites list of channels/networks that would mimic televisions stations (this could be preset with popular sets and customizable). When “flipping” a “channel” would play a 10-15 second cold opening to draw viewers in and commercials could be inserted at any time after this. These programs could recycle after a 30 minute period to also mimic television operation. This concern may be a way off and is not the ideal operation of internet delivered video but I would like to see more people on board with a new model, this may be a piece in bridging the gap.

I and fellow Iona Group developer, Mark Tovey, had the chance to present a basics of social media for the Morton, IL Economic Development Council’s LEAP (Local Education, Affordable Price) session.

Social networking on the web is just an extension of social networking in person. You are connecting with other people for a variety of purposes: business, social, common interest. As the internet came along we adapted these same ideas to the tools of the time. Chat rooms and email became the tools of socializing. As the web became more of a marketplace of ideas it became more social. Specific tools to socialize became available adding yet another avenue to network with others.

Because social media is so prevalent it is a hot place to be in terms of a business. Advertising dollars are continuing to be thrown at the web and at social media and there is growing marketplace on the internet and especially within social media outlets. Why? This is where the people are gathering and talking, why not talk about you?

One of the tools of social media is Twitter. Twitter is a micro-blogging platform limiting posts (tweets) to 140 characters. This character limitation has its roots in cell phone communication but Twitter has grown since it’s cell platform start. Twitter is a give and take, where you can follow people and have them follow you. Signing up for Twitter is fairly straight forward. Click on the sign up link, enter your information and start tweeting. If you are a business these micro messages would reflect the message you are trying to deliver about your company. Are you representing an individual at your company or are you the official marketing voice of your company? The answer to this may determine the content of the tweet. Twitter is also good for following others and keeping an ear to the ground about your business’s buzz. Are customers talking up your service or criticizing you for an area they found lacking. Responding to good or bad can more than make up for the original offense. Being able to head off some bad word of mouth can turn a negative advertisement into a good one. How do you find these people to listen to? Search is a great option to find people in your area, industry or people who are talking about you. There are some directories that may also be helpful (wefollow.com) or just using the basic search or “find people” functionality on Twitter may be enough to get started. You may also find that by putting out quality information people may start to find you, and if they are not spammers or bots you may very well want to follow them back. If they are interested in your business you are interested in them.

Another tool to connect is Facebook. Facebook was started on the Harvard campus by Mark Zuckerburg and quickly took root on other college campuses and then around the world. Facebook now has 300 million active users and 50% of those users log on any given day. How does a business take advantage of the networks built on Facebook? Usually through a Fan Page. A fan page is a brand or business landing page on Facebook. The sign up process is similar to Twitter. If you want to build a fan page you can use an existing personal Facebook account or create one fresh. In either case you use the create a page link on the Facebook’s home page. From here you can indicate you type of business and the name of the fan page. When you are logged in you need to add some details to the profile and an profile picture to gain a bit more footing as people search and gain information about your business. Add as much information to the profile as possible to enhance the value of your page. Status updates are similar to Twitter’s tweets.

So why do people share information on the social networks? As with anybody we do things for ourselves. We are promoting a self image, whether it be humorous, helpful, emotional, friendly or any number of different characteristics. So sharing something that triggers on emotions is key to encouraging social marketing. A funny video will get shared around as people want to make others laugh.

This advertisement for Dove may get shared as people want to encourage a healthy self image:

The strength of the emotion and impact of the message increases the sharing potential. These are videos but text or still images can be just as viral. Oprah has been a catalyst for coupon sharing with a KFC grilled chicken promotion and more recently a 50% off Payless shoe coupon that was good for two days only. This type of information relies on getting shared through social tools. People also get a great feeling of helping their fellow man by sharing this information around.

Individuals like to help people by sharing information and that is good for business too. Participating in the community shows that you are a valuable resource and full of expertise. Offering tips or how-to’s is the equivalent to a physical coupon in that you are giving some information away for free in the hopes of gaining a paying customer. Listen to your community and share back. Dell has done this in an ironic way by offering advice about social media to small businesses on Facebook. They took their own advice.

LinkedIn is another social tool more designed for professionals within business rather than businesses themselves. This may be a good way to gain credibility with people within your field. There is a similar simple sign up process and a free or subscription level of service.

To utilize these networks to their fullest you will want to connect them. Use your business website to advertise your membership in each of these platforms. Know that to get the most out of social media networking you need to put time in. Leaving a Twitter account dormant could be just as damaging as not having one at all. This may indicate to customers that you are not listening or contributing anything to the community. So communicate with sincerity and with the intention of advancing your business’s place in the community. This is a legitimate marketing venture.

And if you wondering the results of your labor there are a few ways to track statistics. Facebook fan pages and Youtube videos have built-in numbers associated with their content. If you want a bit more granularity you could share links you may have via a url shortener like bit.ly. Many of these services will offer statistics on individual links, such as how many clicks and where the clicks are coming from (both platform and geographic location). And if you are looking for more feed back on whether social media is working, look to the quality of conversation being conducted between you and your customer.

Social Media Delicious links

Many of the social bookmarking sites have apis that offer a wide variety of options for posting, searching, and other combination of actions. Maybe you don’t want to delve into the api, maybe you don’t want to deal with user verification information, or maybe for some reason it just is not working. Well there is a way to pass all of these sites simple url strings that will achieve the basic posting information and these strings are often used for blog bookmarking icons like the ones below (go ahead and click all of them at the bottom of this page and show some love). The following are some of the options available for a handful of bookmarking/sharing sites in the form of a url string.

Twitter –
Simple one option url to post status message:
http://twitter.com/home?
status=Great+post+on+social+bookmarking+http://electricpineapple.net/?p=96+%23electricpineapple

Basic options for most sites include the url, title and additional information (notes, description, etc). Urls may work without encoding but for best results everything should be url encoded.

Delicious -
http://delicious.com/save?
url=http://electricpineapple.net/?p=96&
title=Bookmark%20%27em%20Danno&
notes=Check out this awesome post

Digg -
http://digg.com/submit?
url=http://electricpineapple.net/?p=96&
title=Bookmark%20%27em%20Danno&
bodytext=Great post on social bookmarking&
media=news&
topic=Programming

Media options for digg are news, video or image. Topics are a fixed list of items. Digg had good documentation for this feature.

Facebook -
http://www.facebook.com/share.php?
u=http://electricpineapple.net/?p=96&
t=Bookmark%20%27em%20Danno

LinkedIn -
http://www.linkedin.com/shareArticle?
mini=true&
url=http%3A%2F%2Felectricpineapple.net%2F%3Fp%3D96&
title=Bookmark%20%27em%20Danno&
source=Electric+Pineapple&
summary=Options%20available%20for%20a%20handful%20of%20bookmarking%2F
sharing%20sites%20in%20the%20form%20of%20a%20url%20string

The mini parameter is required to be true. LinkedIn also was well documented.

Reddit -
http://www.reddit.com/submit?
url=http%3A%2F%2Felectricpineapple.net%2F%3Fp%3D3&
title=Bookmark%20%27em%20Danno

Google Bookmarks -
http://www.google.com/bookmarks/mark?
op=edit&
bkmk=http%3A%2F%2Felectricpineapple.net%2F%3Fp%3D96&
title=Bookmark%20%27em%20Danno&
labels=Social%20Bookmarking&
annotation=Options%20available%20for%20a%20handful%20of%20bookmarking%2F
sharing%20sites%20in%20the%20form%20of%20a%20url%20string

The op parameter is required for submitting. Labels and annotation are additional information fields.

Myspace -
http://www.myspace.com/Modules/PostTo/Pages/?
u=http%3A%2F%2Felectricpineapple.net%2F%3Fp%3D96&
t=Bookmark%20%27em%20Danno

Slashdot -
http://slashdot.org/bookmark.pl?
title=Bookmark%20%27em%20Danno&
url=http%3A%2F%2Felectricpineapple.net%2F%3Fp%3D96

Stumbleupon -
http://www.stumbleupon.com/submit?
url=http%3A%2F%2Felectricpineapple.net%2F%3Fp%3D96&
title=Bookmark%20%27em%20Danno

Newsvine -
http://www.newsvine.com/_tools/seed&save?
u=http%3A%2F%2Felectricpineapple.net%2F%3Fp%3D96&
h=Bookmark%20%27em%20Danno

Technorati -
http://technorati.com/faves?
add=http%3A%2F%2Felectricpineapple.net%2F%3Fp%3D96

This is by far a complete list, please add your own favorite sites in the comments.

The difference between setting up Drupal Services for a Flex 3 project and for an Air project is a few slight modifications. Using my tutorial for connecting a Flex project with Services and making the changes below will get Air connected with Drupal Services.

1. You will need to create an Air project instead of a Flex project in the initial setup in Flex Builder (WindowedApplication vs. Application)
2. If you did not give the services-config.xml a full path to Drupal Services for the endpoint uri you will need to do that (http://www.mydomain.com/services/amfphp)

That’s it, 2 steps. Air is up and working.