Asked — Edited
Resolved Resolved by DJ Sures!

Camera Uwp/Wpf

We are trying to get a video stream of the Roli's camera in the "Capture Element" which is like a display panel.

My question is if there is anyway we can find the robot's camera as a device.


using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Runtime.InteropServices.WindowsRuntime;
using Windows.Foundation;
using Windows.Foundation.Collections;
using Windows.UI.Xaml;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Controls.Primitives;
using Windows.UI.Xaml.Data;
using Windows.UI.Xaml.Input;
using Windows.UI.Xaml.Media;
using Windows.UI.Xaml.Navigation;
using Windows.Media.Capture;
using Windows.Storage;
using Windows.Devices.Enumeration;

using System.Threading.Tasks;         // Used to implement asynchronous methods

using Windows.Devices.Sensors;        // Orientation sensor is used to rotate the camera preview
using Windows.Graphics.Display;       // Used to determine the display orientation
using Windows.Graphics.Imaging;       // Used for encoding captured images
using Windows.Media;                  // Provides SystemMediaTransportControls

using Windows.Media.MediaProperties;  // Used for photo and video encoding

using Windows.Storage.FileProperties; // Used for image file encoding
using Windows.Storage.Streams;        // General file I/O
using Windows.System.Display;         // Used to keep the screen awake during preview and capture
using Windows.UI.Core;                // Used for updating UI from within async operations
using System.Diagnostics;

// The Blank Page item template is documented at http://go.microsoft.com/fwlink/?LinkId=402352&clcid=0x409

namespace EZ_B
{


    /// 
    /// An empty page that can be used on its own or navigated to within a Frame.
    /// 
    public sealed partial class MainPage : Page
    {

        EZB _ezb;
        EZBv4Video video;
        MediaCapture _mediaCapture;


        bool _isInitialized;
        bool _isPreviewing;
        bool _isRecording;
        bool _externalCamera;
        bool _mirroringPreview;

        private readonly DisplayRequest _displayRequest = new DisplayRequest();
        private readonly DisplayInformation _displayInformation = DisplayInformation.GetForCurrentView();
        private DisplayOrientations _displayOrientation = DisplayOrientations.Portrait;

        private readonly SimpleOrientationSensor _orientationSensor = SimpleOrientationSensor.GetDefault();
        private SimpleOrientation _deviceOrientation = SimpleOrientation.NotRotated;

        private static readonly Guid RotationKey = new Guid("C380465D-2271-428C-9B83-ECEA3B4A85C1");

        bool flag;

        public MainPage()
        {
            this.InitializeComponent();
            


        }


        //ASYNC IS REALLY IMPORTANT//////////////////
        private async void Button_Click(object sender, RoutedEventArgs e)
        {

            if ((Button)sender == inputButton)
            {
                _ezb = new EZB();
                flag = true;


                //AWAIT IS REALLY IMPORTANT//////////////////
                await _ezb.Connect("192.168.1.1");

                if (!_ezb.IsConnected)
                {
                    greetingOutput.Text = "NOT CONNECTED!";

                    return;
                }

                if (_ezb.IsConnected)
                {
                    greetingOutput.Text = "CONNECTED!";
                    MusicSynth m = new MusicSynth(_ezb);
                    m.PlayNote(MusicSynth.NotesEnum.C1, 1000);


                    //CAMERA
                    //CameraCaptureUI ui = new CameraCaptureUI();
                    

                    video = new EZBv4Video();
                    await video.Start(_ezb, "192.168.1.1", 23);
                    video.CameraSetting(EZBv4Video.CameraSettingsEnum.Res320x240);
                    video.CameraSetting(EZBv4Video.CameraSettingsEnum.START);

         

                    await InitializeCameraAsync();

 
                }
            }

           

            return;
        }

        private async Task InitializeCameraAsync()
        {

            if (_mediaCapture == null)
            {

                // Get available devices for capturing pictures
                var allVideoDevices = await DeviceInformation.FindAllAsync(DeviceClass.VideoCapture);

                // Get the desired camera by panel
                DeviceInformation cameraDevice =
                    allVideoDevices.FirstOrDefault(x => x.EnclosureLocation != null &&
                    x.EnclosureLocation.Panel == Windows.Devices.Enumeration.Panel.Back);

                // If there is no camera on the specified panel, get any camera
                cameraDevice = cameraDevice ? allVideoDevices.FirstOrDefault();

                if (cameraDevice == null)
                {
                    greetingOutput.Text = "No camera device found.";
                    return;
                }

                // Create MediaCapture and its settings
                _mediaCapture = new MediaCapture();

                // Register for a notification when video recording has reached the maximum time and when something goes wrong
                //_mediaCapture.RecordLimitationExceeded += MediaCapture_RecordLimitationExceeded;

                var mediaInitSettings = new MediaCaptureInitializationSettings { VideoDeviceId = cameraDevice.Id };

                // Initialize MediaCapture
                try
                {
                    await _mediaCapture.InitializeAsync(mediaInitSettings);
                    _isInitialized = true;
                }
                catch (UnauthorizedAccessException)
                {
                    greetingOutput.Text = "The app was denied access to the camera";
                }
                catch (Exception ex)
                {
                    Debug.WriteLine("Exception when initializing MediaCapture with {0}: {1}", cameraDevice.Id, ex.ToString());
                }

                // If initialization succeeded, start the preview
                if (_isInitialized)
                {
                    // Figure out where the camera is located
                    if (cameraDevice.EnclosureLocation == null || cameraDevice.EnclosureLocation.Panel == Windows.Devices.Enumeration.Panel.Unknown)
                    {
                        // No information on the location of the camera, assume it's an external camera, not integrated on the device
                        _externalCamera = true;
                    }
                    else
                    {
                        // Camera is fixed on the device
                        _externalCamera = false;

                        // Only mirror the preview if the camera is on the front panel
                        _mirroringPreview = (cameraDevice.EnclosureLocation.Panel == Windows.Devices.Enumeration.Panel.Front);
                    }

                    await StartPreviewAsync();

                    //UpdateCaptureControls();
                }
            }
        }


        private async Task StartPreviewAsync()
        {
            // Prevent the device from sleeping while the preview is running
            _displayRequest.RequestActive();

            // Set the preview source in the UI and mirror it if necessary
            PreviewControl.Source = _mediaCapture;
            PreviewControl.FlowDirection = _mirroringPreview ? FlowDirection.RightToLeft : FlowDirection.LeftToRight;

            // Start the preview
            try
            {
                await _mediaCapture.StartPreviewAsync();
                _isPreviewing = true;
            }
            catch (Exception ex)
            {
                Debug.WriteLine("Exception when starting the preview: {0}", ex.ToString());
            }

            // Initialize the preview to the current orientation
            //if (_isPreviewing)
            //{
            //    await SetPreviewRotationAsync();
            //}
        }


    }

    }


ARC Pro

Upgrade to ARC Pro

Become a Synthiam ARC Pro subscriber to unleash the power of easy and powerful robot programming

PRO
Synthiam
#1  

The camera is not a device because it is not connected to the computer through any ports. It is wifi and a data steam.

The uwp source code provides example to view the camera.

What additional functionality are you looking for?

#2  

First of all, thank you for replying to this email. This is one of the late night thing we are doing, and we kind of posted this in a hurry.

Do you know which UWP source code is it? We had problems using and running the UniversalBot project and had to manually add the classes. We probably missed it along the process.

PRO
Synthiam
#3  

The file is called "EZBv4Video.cs"

The "OnImageDataReady" event will be raised for every new frame. The data in the OnImageDataReady event is a jpeg byte array.

#4  

We tried running the UWP source code (UniversalBot), but for some reason it would not event handle or manage the event "OnImageDataReady."

We have also tried making our own codes, and decoding the jpeg into bitmap, so that we can use the ImageSource using XAML. Unfortunately, it did not work, and we are not sure if the event "OnImageDataReady" is actually being raised.

This is the last part of the puzzle that we need for our research, and we are kind of desperate.

The codes are attached below.


namespace EZ_B
{
    public sealed partial class MainPage : Page
    {
        EZB _ezb;
        EZBv4Video _video;
        WriteableBitmap bm;
        DispatcherTimer _timer = new DispatcherTimer();

        public MainPage()
        {
            this.InitializeComponent();
            _timer.Interval = TimeSpan.FromMilliseconds(500);
            _timer.Tick += _timer_Tick;
        }

        private async void Button_Click(object sender, RoutedEventArgs e)
        {
            if ((Button)sender == connectButton)
            {
                _ezb = new EZB();
                await _ezb.Connect("192.168.1.1");

                if (!_ezb.IsConnected)
                {
                    connectStatus.Text = "NOT CONNECTED!";
                    return;
                }

                if (_ezb.IsConnected)
                {
                     _timer.Start();
                    connectStatus.Text = "CONNECTED!";
                    _video = new EZBv4Video();
                    _video.OnImageDataReady += Video_OnImageDataReady;
                    await _video.Start(_ezb, _ezb.ConnectedEndPointAddress, 24);
                    _video.OnStart += Video_OnStart;
                    _video.CameraSetting(EZBv4Video.CameraSettingsEnum.Res640x480);

                    //_video.CameraSetting(EZBv4Video.CameraSettingsEnum.START);

                    if (_video.IsRunning)
                    {
                        cameraStatus.Text = "CAMERA STREAMING!";
                    }
                    else
                    {
                        cameraStatus.Text = "CAMERA NOT STREAMING!";

                    }

                }
            }
            return;
        }

        //end of Button_Click
        private async void Video_OnImageDataReady(byte[] imageData)
        {

            tappedStatus.Text = "OnIMAGE!";

            MusicSynth m = new MusicSynth(_ezb);

            m.PlayNote(MusicSynth.NotesEnum.C1, 1000);

            await Dispatcher.RunAsync(

            Windows.UI.Core.CoreDispatcherPriority.Normal,

            async () =>

            {

                MemoryStream ms = new MemoryStream(imageData);

                var decoder = await BitmapDecoder.CreateAsync(BitmapDecoder.JpegDecoderId, ms.AsRandomAccessStream());

                var sbit = await decoder.GetSoftwareBitmapAsync();

                bm = new WriteableBitmap((int)decoder.PixelWidth, (int)decoder.PixelHeight);

                sbit.CopyToBuffer(bm.PixelBuffer);

                imageSource.Source = bm;

            });

?
//////////////////////////////////////commented out
            //await Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () =>

            //{

            //    imageSource.Source = (BitmapSource)imageData.AsBuffer();

            //});

        }


        private  void Video_OnStart()

        {

            tappedStatus.Text = "OnStart!";

        }

        private async void _timer_Tick(Object sender, Object e)

        {

            tappedStatus.Text = string.Format("ADC 0: {0}", await _ezb.ADC.GetADCValue12Bit(EZ_B.ADC.ADCPortEnum.ADC0));

        }

    }


}
#5  
<Page
    x:Class="EZ_B.MainPage"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:local="using:EZ_B"

    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"

    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"

    mc:Ignorable="d">

    <StackPanel Background="{ThemeResource ApplicationPageBackgroundThemeBrush}" Margin="0,20,0,0">

        <Button x:Name="connectButton" Content="Connect" Click="Button_Click"/>

        <TextBlock x:Name="connectStatus"/>

        <TextBlock x:Name="cameraStatus"/>
        
        <TextBlock x:Name="tappedStatus"/>
        <Image x:Name = "imageSource" Width ="200" Height="200" Visibility= "Visible" Stretch="Fill" />

    </StackPanel>
</Page>
PRO
Synthiam
#6  

Please properly format your code when pasting in the forum. There are UBB CODE legend directly to the right of the input form.

  1. Add a breakpoint in the event.

  2. The assignment of the OnStart event is after you start the camera...

  3. Do not set camera settings until you actually get a camera to work

  4. Check the _ezb.GetLastErrorMsg to see if the camera is actually connecting

The code should be...


_video = new EZBv4Video();
_video.OnImageDataReady += Video_OnImageDataReady;
_video.OnStart += Video_OnStart;
await _video.Start(_ezb, _ezb.ConnectedEndPointAddress, 24);

#7  

I am confused on when the camera actually starts.

PRO
Synthiam
#8  

The camera starts when you execute the method _camera.Start();

The issue is going to be clear once you follow my directions. Please review my previous response.

#9  

The event "OnImageDataReady" was never raised. The breakpoint inside the event handler was never even reached.

There was no printed error as well.

PRO
Synthiam
#10  

You're obviously not connected to the camera. Step through the code and see where it's not happening:)

#11  

Thank you. We finally got it working.

PRO
Synthiam
#12  

I knew you would:D Congrats!