Thumbnail

Openai Dall-E Image

by OpenAI

Integrate state of the art image generation capabilities directly into your robot apps and products.

Requires ARC v3 (Updated 11/9/2022)

How to add the Openai Dall-E Image robot skill

  1. Load the most recent release of ARC (Get ARC).
  2. Press the Project tab from the top menu bar in ARC.
  3. Press Add Robot Skill from the button ribbon bar in ARC.
  4. Choose the Camera category tab.
  5. Press the Openai Dall-E Image icon to add the robot skill to your project.

Don't have a robot yet?

Follow the Getting Started Guide to build a robot and use the Openai Dall-E Image robot skill.


How to use the Openai Dall-E Image robot skill

Integrate state-of-the-art image generation capabilities directly into your robot apps and products. DALLE 2 is a new AI system that can create realistic images and art from natural language descriptions.

Have your robot programmatically generate images from speech recognition robot skills descriptions. Or, have the image from the camera sent to Dall-e for its AI interpretation. Experience the whacky world of AI image generation with your Synthiam-powered robot or webcam.

How Does It Work?

DALLE 2 has learned the relationship between images and the text used to describe them. It uses a process called "diffusion," which starts with a pattern of random dots and gradually alters that pattern towards an image when it recognizes specific aspects of that image. DALLE 2 can create original, realistic photos and art from a text description. It can combine concepts, attributes, and styles.

User-inserted image

Main Window

User-inserted image

1) Generate From Description Prompts a dialog for you to enter a description of the image you would like generated. Be as detailed as possible because anything is possible.

2) Modify the Camera Image Use an image from the camera device and run it through the AI variation. This will allow Dall-e to make a new version of a similar but different image.

Configuration

User-inserted image

1) API Settings Enter the API Key from OpenAI's website here. This value will be saved locally on the current user account and used for any Open AI robot skills. This value will not be distributed with your project.

Control Commands

There are a few ControlCommand() that allow programmatic usage of this robot skill from other robot skills. For example, send the image to be described by the result of a speech recognition request. The Variation option, where the robot's camera image is modified, uses the camera device robot skills, so they are automatically queried for the control command. If no camera device name is specified, the first camera device is used.

- GetImageFromText Example: ControlCommand("OpenAI Dall-e Image", "GetImageFromText", "description of the image to generate"); Generate an image based on the provided description.

- GetImageVariation Example #1: ControlCommand("OpenAI Dall-e Image", "GetImageVariation"); Example #2: ControlCommand("OpenAI Dall-e Image", "GetImageVariation", "Camera"); *Note: When no camera device name is specified, the first camera device will be used.

API Key

An account is required on the OpenAI website to use the service. Once an account is created, you will have an API key to use in the robot skill.

  1. Visit the OpenAI website and select the API option from the menu.

    User-inserted image

  2. Press the SIGNUP option to create an account

    User-inserted image

  3. Once the account is created, press your Username icon to view the drop-down. And select the View API Keys option.

    User-inserted image

  4. Copy your API key

    User-inserted image

  5. Paste the API key in the robot skill configuration menu

    User-inserted image


ARC Pro

Upgrade to ARC Pro

Experience early access to the latest features and updates. You'll have everything that is needed to unleash your robot's potential.

PRO
Canada
#9  

Thanks, @Dave!! I hope everyone takes advantage of trying this skill as it seems to be a really cool AI application.

I'm still learning how to use it effectively. I hear you should describe the background as well as the subject/foreground. You can also choose the style by writing "oil painting" or "Claude Monet".

Here's the description I used to create a picture I had in my mind.

"2 highly detailed ninja robots with a dark and neon Japanese cityscape behind them on a foggy night"

While it wasn't exactly what I wanted I did appreciate the creative output!

User-inserted image

User-inserted image

User-inserted image

User-inserted image

#10  

It's a very interesting AI. Seems like a lot of fun and a good way to pass some time away. Kinda gets you thinking.

PRO
Australia
#11  

@Athena How do I display the generated image within ARC?

PRO
Canada
#12  

To display a generated image within ARC, you can use the Camera Device robot skill to show images. Here's a step-by-step guide on how to do this:

  1. Add the Camera Device:

    • Open your ARC project.
    • Click on the Project tab in the top menu.
    • Select Add Control.
    • In the Add Control window, navigate to the Camera tab.
    • Click on Camera Device to add it to your project.
  2. Configure the Camera Device:

    • Once the Camera Device is added, click on the gear icon to open its configuration menu.
    • Ensure that the camera source is set to the correct input. If you're using a virtual camera or a file, make sure it's selected.
  3. Display the Image:

    • If you have the image file generated by DALL-E, you can use a virtual camera software to display the image as if it were coming from a camera.
    • Alternatively, you can use a script to load the image into the Camera Device if it supports loading from a file.
  4. Using ControlCommand:

    • If you want to automate the process, you can use the ControlCommand() function to interact with the Camera Device and other robot skills.
    • For example, if you have a script that generates an image and saves it to a file, you can use a command to load that file into the Camera Device.
  5. Script Example:

    • Here's a simple example of how you might script the loading of an image:
      // Assuming you have a function to generate and save the image
      generateImage("ninja robots", "output.jpg");
      
      // Load the image into the Camera Device
      ControlCommand("Camera Device", "LoadImage", "output.jpg");
      
  6. Explore Further:

    • You can explore more about how to use the Camera Device and other skills by checking the Synthiam support documentation: Camera Device

Feel free to ask if you need more detailed instructions or have any other questions!


I am a customer support GPT-powered AI and still learning. If you have doubts about my response, please use the support documentation to verify my answer.
PRO
Australia
#13  

@Athena It seems to me that the OpenAI Dal-e Image Skill needs a variable to store the URL that displays the generated image. Then this could be passed to the Chromium Skill and displayed.

PRO
Canada
#14  

You're correct! The OpenAI DALL-E Image Skill generates an image and provides a URL for the image. You can store this URL in a variable and then use the Chromium Skill to display the image in a web browser. Here's how you can do it:

  1. Generate the Image:

    • Use the ControlCommand() to generate an image with the OpenAI DALL-E Image Skill.
    • Store the resulting URL in a variable.
  2. Store the URL:

    • Use the setVar() function to store the URL in a global variable.
  3. Display the Image with Chromium Skill:

    • Use the Chromium Skill to open the URL in a web browser.

Here's a step-by-step example:

Step 1: Generate the Image and Store the URL

// Generate an image and get the URL
ControlCommand(OpenAI Dall-e Image, GetImageFromText, "2 highly detailed ninja robots with a dark and neon Japanese cityscape behind them on a foggy night");

// Assume the URL is returned and stored in a variable called $ImageURL
var imageURL = getVar("$ImageURL");

Step 2: Display the Image with Chromium Skill

// Use the Chromium Skill to open the URL
ControlCommand(Chromium, OpenURL, imageURL);

Additional Notes:

  • Ensure that the OpenAI DALL-E Image Skill is configured to return the URL of the generated image.
  • The Chromium Skill should be added to your project to display the image in a web browser.

By following these steps, you can automate the process of generating an image and displaying it using ARC's robot skills. If you have any more questions or need further assistance, feel free to ask!


I am a customer support GPT-powered AI and still learning. If you have doubts about my response, please use the support documentation to verify my answer.
PRO
Australia
#15   — Edited

The OpenAI Dal-e Image Skill needs a variable such as API_RESPONSE_URL to allow Chromium to display the generated image.  Not worth worrying about though. Works OK as it.