Camera - Camera Device

How To Add This Control To Your Project

Step #1) Load the most recent release of EZ-Builder
Step #2) Press the Project tab from the top menu bar of EZ-Builder
Step #3) Press Add Control from the button ribbon bar of EZ-Builder
Step #4) Choose the Camera category tab
Step #5) Press the Camera Device icon to add the control to your project




Description

There are many parts to this manual which may be accessed directly here...



Introduction

If a camera is mounted on your robot, this control can be used to track color objects such as Red Balls, track motion, track human faces, watch television, or simply view the camera image. The brightness of the detected colored object can be adjusted manually to remove any false positives. Test in a bright room with the desired colored object and adjust the brightness until only that object is detected. Computer vision is an advanced technology which requires a bright room without shadows. Computer vision is a very experimental technology, and therefore requires a clean and bright environment to accurately detect objects and colors.


What is a Tracking Type?

A tracking type is the function of what to track. For example, to track the color RED, you would only select the Color tracking type. Each tracking type has configuration settings which are on their own tabs. If you have COLOR checked as a tracking type, then the configuration for the COLOR tracking type is in the tab labelled COLOR.

On the Camera Control, select the TRACKING tab. This will display a selection of tracking type checkboxes. Select one or more of the types of tracking type to detect objects. If this is your first time, start with only one at a time to learn how they work. Rarely will you track more than one type at once.


Different Kinds of Tracking Types

Color

This will track the specified predefined color from the COLOR tab. The color can be selected between Red, Green or Blue. The Color tab will also give you settings to adjust the brightness and size of the object. Hold the object in front of the camera while changing the settings. This is the basic color tracking method which is used in RoboScratch's "wait for color".

Motion

This observes changes within the camera image. Motion should not be confused with the "Movement" Setting from the configuration menu, as they are different things and should not be used together. The Motion Tracking Type will detect a change in the camera image and return the area of the change. For example, if your camera is stationary and you wave your hand in a small area of the camera image, you will see the motion display of your hand moving. If the entire robot moves during Motion Tracking Type, the entire image would have been considered changed and that is not useful for tracking. So, Motion Tracking Type is only used on stationary cameras. Meaning, your robot is not moving.

QR Code

The QR Code has additional settings in the Configuration/Settings menu of the Camera Control (the little gear icon). You can use the QR Code Generator (Add Control->Camera) to create your own QR Codes and print out. Otherwise, the QR Code Tracking Type will detect QR Codes and set the variable, or execute the code within the QR Code. Visit the Camera's setting page (gear icon) to configure more options for QR Codes. QR Code recognition is CPU intensive and therefore slow. We do not recommend using QR Code for tracking. It is best used for identification and triggering events, not movement tracking such as demonstrated with Multi Color or Motion, etc...

Multi Color

This is an advanced and more detailed tracking method over the generic Color tracking type. With Multi Color, you can specify and fine tune your own color, as well as add multiple colors. An ez-script variable will be configured to hold the name of the current detected color that has been detected. The multi color definitions can be trained in the Multi Color tab. If you name a custom color "Red", this does not override the basic predefined color tracking types. The basic color tracking (discussed above), has three predefined colors. This multicolor tracking type is separate and has no relationship to those predefined colors.

Face

The face tracking type will attempt to detect faces within the image. The Face tracking uses calculations to detect eyes and a mouth. This does require additional processing and will slow the framerate. Also, detecting images within a video stream with many complicated objects will return false positives. Use the Face Tracking against a white wall or non-complicated background. If you wish to detect specific faces, use the Object Tracking type, as you can train your face as an object.

Glyph

This tracking type will look for images that consist of black and white squares. There are set Glyphs which will be used for this tracking type. The glyph images can be found on the Camera's Manual page on the website (accessed by pressing the ? next to the X on the control). Remember, every control has a ? (question mark) next to the X (close). Pressing the ? (question mark) will bring you to the help page for the control. If you download and print the Glyph PDF from the website, this tracking type will detect the glyph. You may also visit the Camera Settings (gear icon) to configure augmented reality overlays on the glyph. This means the camera will super-impose an image over the detected glyph. When tracking Glyph images, the glyph will only execute its respective tracking script once until another glyph has been detected or the ClearGlyph ControlCommand() has been called. To see all available ControlCommand(), press the Cheat Sheet tab when editing scripts.

Glyph Downloads - Glyph PDF

Custom Haar

Unless you hare experienced with computer vision, ignore this tracking type. In brief, this tracking type is used for computer vision specialists who generate HAAR Cascades. This is very cpu intensive and will most likely bring a slower computer to a halt. If the input cascade is not optimized, it will also have great affects on framerate processing performance. Leave this option to the experts :)

Object

This is an advanced computer vision learning tracking type that gives you the ability to teach the robot an object (such as a face, logo, etc.) and have it detect it. Computer vision learning is very experimental and requires patience and consistent lighting. The object training is done in the Object tab, which is settings for the object. Remember, settings for specific tracking types are configured within their tab in the Camera Control. Some tracking types have additional setting/parameters within the Camera Settings (gear icon).

*Note: For best results with object training, consider that you are teaching the robot specific details of the object, not the object itself. This means the entire object does not need to be in the training square. Only details which are unique to that object need to be in the training square. For example, you may not wish to teach the robot a can of soda/cola by putting the entire can in the square. Merely put the logo in the square, or any other identifying features.


Tracking with Servo or Movements

The above section describes tracking types, that is what the robot is looking for. This section describes how to react when the robot detects an object from the selected tracking type. Examples of reaction types are Servo, Movement, Script, or all three. The Config menu of the Camera Control has many options. On the first page of the config menu, you will find options for servo and movement tracking reaction types.

Servo
If servo tracking is checked, the control assumes the camera is mounted on the specified servo. The servo will be moved from left, right, up and down to track the colored object based on the servo settings that you have provided.

Movement
First, do not confuse Movement with Motion. The motion tracking type (as discussed earlier) is for detecting a moving object in the camera view. The Movement reaction type (as discussed here) is having your robot physically move toward an object. If Movement is checked, the robot will follow the object when it is visible using the project's Movement Panel. The robot will move forward, left or right to follow the desired object. A robot has only one way to "move", and that is defined in the project's Movement Panel. The Movement Panel may be an Auto Position, HBridge, or more. Selecting this Movement option will configure the robot to move toward an image by controlling the Movement Panel.

Script
On the Script tab of the Camera control config menu is a Tracking Start and Tracking End script. When an object is detected from the tracking type, the Tracking Start script will execute. When the camera is no longer tracking an object, the Tracking End will execute.


Video Resolutions and Performance

Machine vision and computer recognition is a very high cpu intensive process. The cameras for computer vision are much less resolution than what you, as a human, would use for recording a birthday party. If you were to run computer vision to recognize objects and decode frames at HD quality, your computer response would grind to a halt.

160x120 = 57,600 Bytes per frame = 1,152,000 Bytes per second
320x240 = 230,400 Bytes per frame = 4,608,000 Bytes per second
640x480 = 921,600 Bytes per frame = 18,432,000 Bytes per second

So at 320x240, your CPU is processing complex algorithm on 4,608,000 Bytes per second. Soon as you move to a mere 640x480, it's 18,432,000 Bytes per second.

To expand on that, 4,608,000 Bytes per second is just the data, not including the number of CPU instructions per step of the algorithm(s). Do not let television shows, such as Person Of Interest, make you believe that machine vision and CPU processing is as accessible in real-time, but many of us are working on it! We can put 4,608,000 Bytes into perspective by relating that to a 2 minute MP3 file. Imagine your computer processing a 2 minute MP3 file in less than 1 second - that is what vision processing for recognition is doing at 320x240 resolution. Soon as you increase the resolution, the CPU has to process an exponentially larger amount of data. Computer vision recognition does not require as much resolution as your human eyes, as it looks for patterns, colors or shapes.

If you want to record a birthday party, do not use a robot camera - buy a real camera. EZ-Robot cameras and software algorithms are designed for vision recognition for robotics only.


Variable Overlay

Right-click on the video image of the camera control and you will be presented with a menu. The menu allows you to Add or Delete text and variables which are displayed with the video stream. To move or relocate the variable position, simply click and drag it around the video image.


Choose Video Devices

When the camera is stopped (not running), additional video devices may be selected or entered. Clicking on the video device input, a drop down will be displayed with selections. The current selected EZ-B IP Address will be displayed, as well as an USB or local cameras. There will also be video devices displayed for third party supported robots, such as the AR Parrot Drone. Additionally, you may manually enter a JPEG Snapshot HTTP address or EZB Camera address.

JPEG Snapshot HTTP Video Device - a third party camera or video source which provides JPEG images may be selected as a video source. Simply enter the complete HTTP URL of the video source. If the HTTP video source requires authentication, consult the documentation of your device for correct syntax. You may test the JPEG URL in a web browser to verify that it is valid and that only the JPEG is displayed, not any HTML. Example: http://192.168.100.2/cgi-bin/snapshot.cgi or http://192.168.100.2/image.jpg, etc..

EZB Camera Device - when the Camera Control is added to a project, the current EZ-B Index #0 ip address is displayed as the default device. If the EZ-B Index #0 ip address has been changed, to accommodate client mode for example, you will manually need to edit the IP Address of the camera video device. Here is the syntax, example: EZB://192.168.0.1 or EZB://192.168.0.1:24.

AR Parrot Drone Device - if using the AR Parrot Drone as a video device, the AR Parrot Drone Movement Panel control must be added, configured and connected to the drone. Please view the AR Parrot Drone Movement Panel documentation for more information.


Use Camera Programmatically In Code

You may also instruct the Camera Device Control to change settings programmatically through code. The ControlCommand() can be called in EZ-Script to change the Camera Device settings. learn more about the ControlCommand() in the Activities section HERE.



Video Notes

The videos tutorials below are a little dated due to the rapidly evolving interface and features for EZ-Builder. You will be able to extract what is necessary for basic tracking with the videos even though the interface is not identical to EZ-Builder today. :)


Video

Related Tutorials & Activities

Camera Focus and Cleaning

Camera Focus and Cleaning

Introduction In this lesson, we will explain how to focus and clean your camera. The only time you will need to repeat this process is if your camera becomes dirty. The lens of your robot may have...

Last Modified on 5/5/2017

Created by EZ-Robot


Color Tracking with Movement

Color Tracking with Movement

Introduction This tutorial demonstrates how to enable Movement tracking to have your Six or Roli robot track a ball. There are two kinds of tracking behaviors, each with their own parameters. Do no...

Last Modified on 9/29/2015

Created by EZ-Robot


Detect Multiple Colors

Detect Multiple Colors

One of the features that makes ez-robot so special is the camera that can detect faces, objects, glyphs, qr codes and multiple colors. For this tutorial, we will use the JD Humanoid. However, this p...

Last Modified on 5/14/2015

Created by EZ-Robot


Movement Panels

Movement Panels

One of the powerful features of EZ-Builder is the concept of Movement Panels. A Movement Panel is a control which is responsible for how your robot moves in directions (forward, left, right, stop, r...

Last Modified on 2/11/2017

Created by EZ-Robot


Microsoft Cognitive Emotion

Microsoft Cognitive Emotion

How would you like your robot to know if you're happy or sad? This lesson demonstrates how to do just that, using the Microsoft Cognitive Services Plugin for EZ-Builder. Learn more about the plugin...

Last Modified on 12/13/2017

Created by EZ-Robot


Microsoft Cognitive Vision

Microsoft Cognitive Vision

Join the Robot overlord DJ Sures and Professor E as they demonstrate how to use the Microsoft Cognitive Vision plugin with EZ-Builder and Revolution EZ-Robots. This episode teaches machine learning...

Last Modified on 12/13/2017

Created by EZ-Robot


Face Detection with Blockly

Face Detection with Blockly

Robot overlord DJ Sures and Professor E show you how to use Blockly for face recognition. What You Will Need JD, Six, Roli or Developer Kit Windows PC or Mobile with EZ-Builder installed...

Last Modified on 5/31/2017

Created by EZ-Robot


Face Detection with RoboScratch

Face Detection with RoboScratch

Robot overlord DJ Sures and Professor E show you how to use RoboScratch to train a robot to recognize your face. What You Will Need JD, Six, or Roli Windows PC or Mobile with EZ-Builder i...

Last Modified on 5/31/2017

Created by EZ-Robot


DIY Autonomous Vehicle

DIY Autonomous Vehicle

Interested in autonomous vehicles and curious how to make your own? Well, we have what you are looking for! This activity demonstrates how to make an autonomous car out of a modified AdventureBot. H...

Last Modified on 6/23/2017

Created by EZ-Robot


Face Detection with EZ-Script

Face Detection with EZ-Script

Robot overlord DJ Sures and Professor E show you how to use EZ-Script for face recognition. What You Will Need JD, Six, Roli or Developer Kit Windows PC with EZ-Builder installed [...

Last Modified on 6/21/2017

Created by EZ-Robot


Vision Object Training & Recognition

Vision Object Training & Recognition

Teach your robot to train and recognize multiple objects using machine learning with the camera. What You Will Need JD, Six, Roli, AdventureBot or Developer Kit Windows PC with EZ-Builder...

Last Modified on 8/6/2017

Created by EZ-Robot


QR Code Detect

QR Code Detect

Get your robot to recognize QR codes. You'll also learn how to generate your own QR codes using EZ-Builder. What You Will Need JD, Six, Roli, AdventureBot or Developer Kit Windows PC or M...

Last Modified on 8/6/2017

Created by EZ-Robot


Line Following with AdventureBot

Line Following with AdventureBot

Learn how to get your AdventureBot to follow a line using EZ-Bits and painters tape on the floor. What You Will Need AdventureBot or Battle Flipper Windows PC with EZ-Builder installed...

Last Modified on 8/8/2017

Created by EZ-Robot


Detecting Glyphs & Augmented Reality

Detecting Glyphs & Augmented Reality

Learn how glyphs are used and detected with the robot camera. We also add augmented reality to super-impose images on the recognized glyphs. What You Will Need JD, Six, Roli, AdventureBot...

Last Modified on 8/22/2017

Created by EZ-Robot


Glyphs to Control Robot Movement

Glyphs to Control Robot Movement

Learn how to use Glyphs to control your robot's movement panel with the camera. When the glyph is far away, the robot will move forward. When the glyph is close, the robot will move reverse. This is...

Last Modified on 8/21/2017

Created by EZ-Robot


Color Tracking With Servos

Color Tracking With Servos

Learn how to program your robot to track colors with servos using the camera. The horizontal and vertical servos that hold the camera head of your robot will move to follow an object of a specific c...

Last Modified on 8/22/2017

Created by EZ-Robot


Line Following with Roli

Line Following with Roli

Robot Overlord DJ Sures and Professor E demonstrate how to program Roli to follow a line using the camera. The camera device will use a multicolor filter to identify blue painters tape. The move fea...

Last Modified on 8/22/2017

Created by EZ-Robot


Detect Multiple Colors

One of the features that makes ez-robot so special is the camera that can detect faces, objects, glyphs, qr codes and multiple colors. For this tutorial, we will use the JD Humanoid. However, this process can easily be applied to any ez-robot configuration.

Last Modified on 9/15/2015

Created by ezrobot


Time Lapse With A Gopro

Here's a video which shows how you can use a GoPro for time lapse photography with Roli. - https://www.youtube.com/watch?v=IreUCKTxbbg Question - which control is being used? I'd like to know where to enter the script to allow Roli to move. I assumed it was either the camera control or the Auto Position Movement Panel. I entered the script in each control but Roli isn't responding. Here is the script from the video: Any suggestions?

Last Modified on 9/15/2015

Created by tom.caine


Use Camera As A Button

You can use a Camera in EZ-Builder as a button without any external code. I created a custom multicolor named "Dark" and set it a orange/red range and I set it to the maximum size for detection. This is what worked best for me as I discovered the orange/red color range was reflected off my hand as it moved toward the camera to block the view. I started out using the darkest shade of blue possible but...

Last Modified on 9/17/2015

Created by JustinRatliff


Related Products

EZ-B v4 Camera

Optimized for computer vision, this camera connects directly to the EZ-B v4/2 Wi-Fi Robot Controller. Give your ezrobot the gift of sight with crisp 640x480 resolution...

EZ-B v4/2 and Camera Combo

The EZ-B v4/2 WiFi Controller and Camera combo start customizing your robot with ez-bits, or create a robot from scratch. This is the brain and...

EZ-B v4/2 Developer Kit

Build your robot or IoT project from scratch. This kit includes the powerful ez-bv4/2 WiFi robot controller, vision camera, servos, battery holder,...

Related Articles: N/A