Make an ARC Skill

Example: Camera Custom Tracking Type

As you've seen in the previous tutorial step about detecting and attaching to the camera, there are a bunch of events that you can use. One of the events allows you to create a custom tracking type as a plugin, which is real cool!

Uses for creating a custom tracking type is if you want to experiment with OpenCV or any other vision libraries. Because ARC leverages .Net, we recommend the x86 nuget install of EMGUCV (https://github.com/emgucv/emgucv). Installing from NUGET is the easiest and most convenient.

The camera events that we'll use for creating a custom tracking type are...


        // assign an event that raises when the camera wants to initialize tracking types
        _camera.Camera.OnInitCustomTracking += Camera_OnInitCustomTracking;

        // assign an event that raises with a new frame that you can use for tracking
        _camera.OnCustomDetection += Camera_OnCustomDetection;

Once inside the OnCustomDetection() event, you have access to a bunch of different bitmaps throughout the flow of the detection process. They are...


// **********************
// From the EZ_B.Camera class
// **********************

    /// 
    /// This is a temporary bitmap that we can use to draw on but is lost per tracking type
    /// 
    public volatile Bitmap _WorkerBitmap;

    /// 
    /// This is the resized original bitmap that is never drawn on. Each tracking type uses this as the main source image for tracking, and then draws on the _OutputBitmap for tracking details
    /// 
    public volatile AForge.Imaging.UnmanagedImage _OriginalBitmap; // resized image that we process

    /// 
    /// Image that is outputted to the display. We draw on this bitmap with the tracking details
    /// 
    public volatile AForge.Imaging.UnmanagedImage _OutputBitmap;

    /// 
    /// Raw image unsized directly from the input device
    /// 
    public volatile AForge.Imaging.UnmanagedImage _RawUnsizedBitmap;

    /// 
    /// Last image for the GetCurrentImage 
    /// 
    public volatile AForge.Imaging.UnmanagedImage _RawUnsizedLastBitmap;

Understanding the images available, the ones we care about for creating a tracking type of our own are...


    /// 
    /// This is the resized original bitmap that is never drawn on. Each tracking type uses this as the main source image for tracking, and then draws on the _OutputBitmap for tracking details
    /// 
    public volatile AForge.Imaging.UnmanagedImage _OriginalBitmap; // resized image that we process

    /// 
    /// Image that is outputted to the display. We draw on this bitmap with the tracking details
    /// 
    public volatile AForge.Imaging.UnmanagedImage _OutputBitmap;

This is because we can use the _OriginalBitmap for our detection, and then draw on the _OutputBitmap where our detection was.

Example This is an example that fakes detection by drawing a rectangle on the _OutputBitmap that bounces around the screen. It moves with every frame in the CustomDetection event.


    // faking an object being tracked
    int _xPos = 0;
    int _yPos = 0;
    bool _xDir = true;
    bool _yDir = true;

    private EZ_B.ObjectLocation[] Camera_OnCustomDetection(EZ_Builder.UCForms.FormCameraDevice sender) {

      if (_isClosing)
        return new ObjectLocation[] { };

      if (!_camera.Camera.IsActive)
        return new ObjectLocation[] { };

      List objectLocations = new List();

      try {

        // This is demonstrating how you can return if an object has been detected and draw where it is
        // The camera control will start tracking when more than one ObjectLocation is returned
        // We're just putting fake bouncing rectable of a detected rect which will be displayed as a tracked object on the screen in the camera device

        if (_xDir)
          _xPos += 10;
        else
          _xPos -= 10;

        if (_yDir)
          _yPos += 10;
        else
          _yPos -= 10;

        var r = new Rectangle(_xPos, _yPos, 50, 50);

        if (r.Right > _camera.Camera._OutputBitmap.Width)
          _xDir = false;
        else if (r.Left  _camera.Camera._OutputBitmap.Height)
          _yDir = false;
        else if (r.Top <= 0)
          _yDir = true;

        var objectLocation = new ObjectLocation(ObjectLocation.TrackingTypeEnum.Custom);
        objectLocation.Rect = r;
        objectLocation.HorizontalLocation = _camera.Camera.GetHorizontalLocation(objectLocation.CenterX);
        objectLocation.VerticalLocation = _camera.Camera.GetVerticalLocation(objectLocation.CenterY);
        objectLocations.Add(objectLocation);

        AForge.Imaging.Drawing.Rectangle(_camera.Camera._OutputBitmap, r, Color.MediumSeaGreen);
      } catch (Exception ex) {

        EZ_Builder.EZBManager.Log(ex);
      }

      return objectLocations.ToArray();
    }


ARC Pro

Upgrade to ARC Pro

Get access to the latest features and updates before they're released. You'll have everything that's needed to unleash your robot's potential!

PRO
Synthiam
#25  

When the popup says it doesn’t detect visual studio, you can still skip and continue. I wonder why it’s not detecting it? We had a hard time trying to find a proper way of detecting - even Microsoft’s suggestion didn’t actually work eye roll

ill look into it a bit further and see if we can find a better way of detecting

PRO
USA
#26  

@DJ: It's easy to find the Visual Studio 2017 and up: Microsoft: https://github.com/Microsoft/vswhere/wiki/Find-MSBuild

Some quick c# code to use with .NET: https://github.com/ppedro74/Utils/blob/master/FindVisualStudio/Program.cs

using System;
using System.Diagnostics;
using System.IO;

namespace FindVisualStudio
{
    internal class Program
    {
        static string ProcessStart(string fileName, string args)
        {
            var processStartInfo = new ProcessStartInfo
            {
                Arguments = args,
                CreateNoWindow = true,
                FileName = fileName,
                RedirectStandardOutput = true,
                UseShellExecute = false,
                WindowStyle = ProcessWindowStyle.Hidden,
                WorkingDirectory = Path.GetDirectoryName(fileName),
            };

            using (var process = Process.Start(processStartInfo))
            {
                process.WaitForExit();
                return process.StandardOutput.ReadToEnd().Trim();
            }
        }

        private static string GetInstallationPath(string vsWhere)
        {
            var installationPath = ProcessStart(vsWhere, "-latest -products * -requires Microsoft.Component.MSBuild -property installationPath");
            return installationPath;
        }

        private static string GetProductLineVersion(string vsWhere)
        {
            var version = ProcessStart(vsWhere, "-latest -property catalog_productLineVersion");
            return version;
        }


        private static void Main(string[] args)
        {
            var programFiles = Environment.GetFolderPath(Environment.SpecialFolder.ProgramFilesX86);
            var vsWhere = Path.Combine(programFiles, "Microsoft Visual Studio", "Installer", "vswhere.exe");
            if (!File.Exists(vsWhere))
            {
                throw new FileNotFoundException("Cannot find Microsoft Visual Studio's vswhere.exe utility.", vsWhere);
            }

            var version = GetProductLineVersion(vsWhere);
            Console.WriteLine($"Visual Studio Version: {version}");

            var installationPath = GetInstallationPath(vsWhere);
            if (!Directory.Exists(installationPath))
            {
                throw new DirectoryNotFoundException(installationPath);
            }

            Console.WriteLine($"Visual Studio installation Path: {installationPath}");

        }
    }
}
PRO
USA
#27   — Edited

Quote:

With Visual Studio 2017 Update 2 or newer installed, you can find vswhere at %ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe, or to make sure it's always available in your repo see Installing for an option using NuGet.
Because is only available with "Visual Studio 2017 Update 2", you can add the vswhere package to your ARC project, and use your version (nuget) to detect older 2017 or legacy Visual Studio Versions (although does not make sense for ARC). https://devblogs.microsoft.com/setup/vswhere-now-searches-older-versions-of-visual-studio/

PRO
Synthiam
#28   — Edited

We went this route and it didn’t work on my computer - because I had a preview of visual studio installed which isn’t in that directory path. Microsoft had numerous suggestions of detecting visual studio. The one which worked for our various installations was a registry check.

apparently with the above individual, the registry didn’t work either. I’ll have to combine a few methods.

everything looks simple from the outside - until you have a hundred thousand+ installations of your software. That’s when you run into things like this lol

PRO
USA
#29   — Edited

@DJ: I agree sometimes the things go out of script easily.

I avoid going through the registry keys, unless is recommended by the vendor. A lot of people blame the changes (keys,  entries are renamed etc), but, that is normal if I own my product is my business and is part of the software evolution. Some products you can break the support contract agreement if you query directly the database, or if you look elsewhere outside of the public API.

Is true story some years ago a "rogue" developer on my team released a Sharepoint integration using a mix of APIs and database queries, everything worked well with multiple clients, until one day the Microsoft Black suits visit one of the customers to follow up on an unrelated support ticket, and they basically used "unsupported" card and left the client hanging, and we had problems too, unfortunately the Rogue developer went to another galaxy ... and the team suffered the consequences.

That does not mean I'm not tempted to do it...:)

I used the vswhere before and I would say is almost 99% bulletproof, is used with Xamarin, NVIDIA, Intel setups. If you add vswhere.exe to your project (nuget package) you cover scenarios where the tool is not present or have been deleted (broken uninstalls).

The other fallback could be ask the user the visual studio version.

The other reason to avoid registry is due to Visual Studio uses a private exclusive registry keys to store more stuff:  http://www.visualstudioextensibility.com/2017/07/15/about-the-new-privateregistry-bin-file-of-visual-studio-2017/

So the things are getting more complex.

The above post is only part of the "Full solution" for example I have one setup with visual studio 2017 c# installed and Visual studio 2019 with Python and C++, vswhere will return 2019 version,  but my c# is done with VS2017.

If you are generating customized vs version project files, maybe a fallback (ask the VS version) will cover more bases.

PRO
Synthiam
#30  

Yes - Microsoft has a few pages on how to identify visual studio and we tried them all during testing - the one we went with was with registry. I'm going to combine the two as using only one method apparently doesn't work for all cases.