Award
Congratulations DJ Sures, for answering the question and has received an EZ-Credit award! Share what you have learned and create a tutorial to help others by clicking here.

Assistance Requested

Help faraonc with their question and receive $10 of EZ-Credit to get more robots and parts from our store. The following information was provided about their previous efforts searching tutorials for a resolution.

faraonc claims to have checked these sources:
 
#1

We are trying to get a video stream of the Roli's camera in the "Capture Element" which is like a display panel.

My question is if there is anyway we can find the robot's camera as a device.

Code:


using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Runtime.InteropServices.WindowsRuntime;
using Windows.Foundation;
using Windows.Foundation.Collections;
using Windows.UI.Xaml;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Controls.Primitives;
using Windows.UI.Xaml.Data;
using Windows.UI.Xaml.Input;
using Windows.UI.Xaml.Media;
using Windows.UI.Xaml.Navigation;
using Windows.Media.Capture;
using Windows.Storage;
using Windows.Devices.Enumeration;

using System.Threading.Tasks; // Used to implement asynchronous methods

using Windows.Devices.Sensors; // Orientation sensor is used to rotate the camera preview
using Windows.Graphics.Display; // Used to determine the display orientation
using Windows.Graphics.Imaging; // Used for encoding captured images
using Windows.Media; // Provides SystemMediaTransportControls

using Windows.Media.MediaProperties; // Used for photo and video encoding

using Windows.Storage.FileProperties; // Used for image file encoding
using Windows.Storage.Streams; // General file I/O
using Windows.System.Display; // Used to keep the screen awake during preview and capture
using Windows.UI.Core; // Used for updating UI from within async operations
using System.Diagnostics;

// The Blank Page item template is documented at http://go.microsoft.com/fwlink/?LinkId=402352&clcid=0x409

namespace EZ_B
{


///
/// An empty page that can be used on its own or navigated to within a Frame.
///
public sealed partial class MainPage : Page
{

EZB _ezb;
EZBv4Video video;
MediaCapture _mediaCapture;


bool _isInitialized;
bool _isPreviewing;
bool _isRecording;
bool _externalCamera;
bool _mirroringPreview;

private readonly DisplayRequest _displayRequest = new DisplayRequest();
private readonly DisplayInformation _displayInformation = DisplayInformation.GetForCurrentView();
private DisplayOrientations _displayOrientation = DisplayOrientations.Portrait;

private readonly SimpleOrientationSensor _orientationSensor = SimpleOrientationSensor.GetDefault();
private SimpleOrientation _deviceOrientation = SimpleOrientation.NotRotated;

private static readonly Guid RotationKey = new Guid("C380465D-2271-428C-9B83-ECEA3B4A85C1");

bool flag;

public MainPage()
{
this.InitializeComponent();



}


//ASYNC IS REALLY IMPORTANT//////////////////
private async void Button_Click(object sender, RoutedEventArgs e)
{

if ((Button)sender == inputButton)
{
_ezb = new EZB();
flag = true;


//AWAIT IS REALLY IMPORTANT//////////////////
await _ezb.Connect("192.168.1.1");

if (!_ezb.IsConnected)
{
greetingOutput.Text = "NOT CONNECTED!";

return;
}

if (_ezb.IsConnected)
{
greetingOutput.Text = "CONNECTED!";
MusicSynth m = new MusicSynth(_ezb);
m.PlayNote(MusicSynth.NotesEnum.C1, 1000);


//CAMERA
//CameraCaptureUI ui = new CameraCaptureUI();


video = new EZBv4Video();
await video.Start(_ezb, "192.168.1.1", 23);
video.CameraSetting(EZBv4Video.CameraSettingsEnum.Res320x240);
video.CameraSetting(EZBv4Video.CameraSettingsEnum.START);



await InitializeCameraAsync();


}
}



return;
}

private async Task InitializeCameraAsync()
{

if (_mediaCapture == null)
{

// Get available devices for capturing pictures
var allVideoDevices = await DeviceInformation.FindAllAsync(DeviceClass.VideoCapture);

// Get the desired camera by panel
DeviceInformation cameraDevice =
allVideoDevices.FirstOrDefault(x => x.EnclosureLocation != null &&
x.EnclosureLocation.Panel == Windows.Devices.Enumeration.Panel.Back);

// If there is no camera on the specified panel, get any camera
cameraDevice = cameraDevice ? allVideoDevices.FirstOrDefault();

if (cameraDevice == null)
{
greetingOutput.Text = "No camera device found.";
return;
}

// Create MediaCapture and its settings
_mediaCapture = new MediaCapture();

// Register for a notification when video recording has reached the maximum time and when something goes wrong
//_mediaCapture.RecordLimitationExceeded += MediaCapture_RecordLimitationExceeded;

var mediaInitSettings = new MediaCaptureInitializationSettings { VideoDeviceId = cameraDevice.Id };

// Initialize MediaCapture
try
{
await _mediaCapture.InitializeAsync(mediaInitSettings);
_isInitialized = true;
}
catch (UnauthorizedAccessException)
{
greetingOutput.Text = "The app was denied access to the camera";
}
catch (Exception ex)
{
Debug.WriteLine("Exception when initializing MediaCapture with {0}: {1}", cameraDevice.Id, ex.ToString());
}

// If initialization succeeded, start the preview
if (_isInitialized)
{
// Figure out where the camera is located
if (cameraDevice.EnclosureLocation == null || cameraDevice.EnclosureLocation.Panel == Windows.Devices.Enumeration.Panel.Unknown)
{
// No information on the location of the camera, assume it's an external camera, not integrated on the device
_externalCamera = true;
}
else
{
// Camera is fixed on the device
_externalCamera = false;

// Only mirror the preview if the camera is on the front panel
_mirroringPreview = (cameraDevice.EnclosureLocation.Panel == Windows.Devices.Enumeration.Panel.Front);
}

await StartPreviewAsync();

//UpdateCaptureControls();
}
}
}


private async Task StartPreviewAsync()
{
// Prevent the device from sleeping while the preview is running
_displayRequest.RequestActive();

// Set the preview source in the UI and mirror it if necessary
PreviewControl.Source = _mediaCapture;
PreviewControl.FlowDirection = _mirroringPreview ? FlowDirection.RightToLeft : FlowDirection.LeftToRight;

// Start the preview
try
{
await _mediaCapture.StartPreviewAsync();
_isPreviewing = true;
}
catch (Exception ex)
{
Debug.WriteLine("Exception when starting the preview: {0}", ex.ToString());
}

// Initialize the preview to the current orientation
//if (_isPreviewing)
//{
// await SetPreviewRotationAsync();
//}
}


}

}

I have found a few conversations, tutorials and activities that may help with your question. Take a look at these links. I've sorted them by what I believe to be most relevant but that is not always the case as I'm still learning.


Also, consider reviewing the Learn section for informative lessons and activities. Check it out!


#2

The camera is not a device because it is not connected to the computer through any ports. It is wifi and a data steam.

The uwp source code provides example to view the camera.

What additional functionality are you looking for?

#3

First of all, thank you for replying to this email. This is one of the late night thing we are doing, and we kind of posted this in a hurry.

Do you know which UWP source code is it? We had problems using and running the UniversalBot project and had to manually add the classes. We probably missed it along the process.

#4

The file is called "EZBv4Video.cs"

The "OnImageDataReady" event will be raised for every new frame. The data in the OnImageDataReady event is a jpeg byte array.

#5

We tried running the UWP source code (UniversalBot), but for some reason it would not event handle or manage the event "OnImageDataReady."

We have also tried making our own codes, and decoding the jpeg into bitmap, so that we can use the ImageSource using XAML. Unfortunately, it did not work, and we are not sure if the event "OnImageDataReady" is actually being raised.

This is the last part of the puzzle that we need for our research, and we are kind of desperate.

The codes are attached below.

Code:


namespace EZ_B
{
public sealed partial class MainPage : Page
{
EZB _ezb;
EZBv4Video _video;
WriteableBitmap bm;
DispatcherTimer _timer = new DispatcherTimer();

public MainPage()
{
this.InitializeComponent();
_timer.Interval = TimeSpan.FromMilliseconds(500);
_timer.Tick += _timer_Tick;
}

private async void Button_Click(object sender, RoutedEventArgs e)
{
if ((Button)sender == connectButton)
{
_ezb = new EZB();
await _ezb.Connect("192.168.1.1");

if (!_ezb.IsConnected)
{
connectStatus.Text = "NOT CONNECTED!";
return;
}

if (_ezb.IsConnected)
{
_timer.Start();
connectStatus.Text = "CONNECTED!";
_video = new EZBv4Video();
_video.OnImageDataReady += Video_OnImageDataReady;
await _video.Start(_ezb, _ezb.ConnectedEndPointAddress, 24);
_video.OnStart += Video_OnStart;
_video.CameraSetting(EZBv4Video.CameraSettingsEnum.Res640x480);

//_video.CameraSetting(EZBv4Video.CameraSettingsEnum.START);

if (_video.IsRunning)
{
cameraStatus.Text = "CAMERA STREAMING!";
}
else
{
cameraStatus.Text = "CAMERA NOT STREAMING!";

}

}
}
return;
}

//end of Button_Click
private async void Video_OnImageDataReady(byte[] imageData)
{

tappedStatus.Text = "OnIMAGE!";

MusicSynth m = new MusicSynth(_ezb);

m.PlayNote(MusicSynth.NotesEnum.C1, 1000);

await Dispatcher.RunAsync(

Windows.UI.Core.CoreDispatcherPriority.Normal,

async () =>

{

MemoryStream ms = new MemoryStream(imageData);

var decoder = await BitmapDecoder.CreateAsync(BitmapDecoder.JpegDecoderId, ms.AsRandomAccessStream());

var sbit = await decoder.GetSoftwareBitmapAsync();

bm = new WriteableBitmap((int)decoder.PixelWidth, (int)decoder.PixelHeight);

sbit.CopyToBuffer(bm.PixelBuffer);

imageSource.Source = bm;

});

?
//////////////////////////////////////commented out
//await Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () =>

//{

// imageSource.Source = (BitmapSource)imageData.AsBuffer();

//});

}


private void Video_OnStart()

{

tappedStatus.Text = "OnStart!";

}

private async void _timer_Tick(Object sender, Object e)

{

tappedStatus.Text = string.Format("ADC 0: {0}", await _ezb.ADC.GetADCValue12Bit(EZ_B.ADC.ADCPortEnum.ADC0));

}

}


}

#6

Code:

<Page
x:Class="EZ_B.MainPage"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation";
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml";
xmlns:local="using:EZ_B"

xmlns:d="http://schemas.microsoft.com/expression/blend/2008";

xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006";

mc:Ignorable="d">

<StackPanel Background="{ThemeResource ApplicationPageBackgroundThemeBrush}" Margin="0,20,0,0">

<Button x:Name="connectButton" Content="Connect" Click="Button_Click"/>

<TextBlock x:Name="connectStatus"/>

<TextBlock x:Name="cameraStatus"/>

<TextBlock x:Name="tappedStatus"/>
<Image x:Name = "imageSource" Width ="200" Height="200" Visibility= "Visible" Stretch="Fill" />

</StackPanel>
</Page>

#7

Please properly format your code when pasting in the forum. There are UBB CODE legend directly to the right of the input form.

1) Add a breakpoint in the event.

2) The assignment of the OnStart event is after you start the camera...

3) Do not set camera settings until you actually get a camera to work

4) Check the _ezb.GetLastErrorMsg to see if the camera is actually connecting

The code should be...

Code:


_video = new EZBv4Video();
_video.OnImageDataReady += Video_OnImageDataReady;
_video.OnStart += Video_OnStart;
await _video.Start(_ezb, _ezb.ConnectedEndPointAddress, 24);

#8

I am confused on when the camera actually starts.

#9

The camera starts when you execute the method _camera.Start();

The issue is going to be clear once you follow my directions. Please review my previous response.

#10

The event "OnImageDataReady" was never raised. The breakpoint inside the event handler was never even reached.

There was no printed error as well.