EZB Server (BlueBerry Server)

Raspberry Pi
Hardware Platform
Raspberry Pi

This server program turns a Raspberry Pi into an EZ-B V4.

VERY IMPORTANT:

Although Raspberry PI is a fantastic small computer can't replace low level micro-controllers, is not a Real Time device.

For specific low level operations e.g. PWM, I2C, SPI, Analog ports you will need additional add-ons i.e. Bonnets, HATS or serial devices.

Connecting cables or devices should be done offline

Raspberry PI is not an Arduino or an EZB-controller, does not have mechanisms to handle small short-circuits i.e. resistors between the headers and the micro controller ports.

Is a 3.3v logic device, in general, applying 5 V to an input that is not tolerant to 5 V may damage the microcontroller circuitry.

Yes and is very sensitive too touch: The Raspberry Pi board has exposed circuitry and electronic components which are subject to irreparable damage or destruction by static electricity.
You don't have to see, hear or feel a static discharge. Damage can be done without knowing it.

User-inserted image


Regarding the software:
Copyright (c) 2019 Pedro Pereira
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
of the Software, and to permit persons to whom the Software is furnished to do
so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY,
WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

The server is open source code and is written in Python although I have a love-and-hate relationship with Python language, two annoying things: it's not statically-typed language (not good for people with Dyslexia or fat fingers:)) and uses indentation to start or end blocks (so yes a single space can break your code) nevertheless it's easy available.

Required components:

Code:

sudo apt-get install python3  python3-pyaudio

Clone the source code repository on your raspberry pi home folder:

Code:

cd ~/
git clone https://github.com/ppedro74/BlueberryServer.git
Start the server:

Code:

cd ~/BlueberryServer/Blueberry.Server.Python/
python3 Main.py
If everything is OK this is an expected output:
User-inserted image

you can see the Raspberry PI IP: 192.168.18.88 and two ports 10023 and 10024 replacements for the original EZB Port's 23 and 24.
The server is ready to accept connections and it will quit if you press ENTER.

Start EZ-Builder:
User-inserted image

Press the Antenna and you will see your hostname IP and  port press the Type: EZ-B

This is an expected debug output:
User-inserted image


Add a camera control (Don't worry if you don't have one):
User-inserted image

Note 1: the connection button will turn green and will be green as long you have a connection with the server.
Pick the Camera type
User-inserted image

If you don't have a RPI camera .... A fake camera image showing a frame counter will be displayed.


About this version
I've updated the code base to support :
1) line arguments
2) USB Cameras

you will need to install:

Code:

sudo apt-get install python3-opencv

update your local repository:

Code:

cd ~/BlueBerryServer
git pull
If you have changes the above pull request may fail, Please copy or memorize the changes and when you are ready run:

Code:

git reset --hard
the above command will revert any local changes, then run: 

Code:

git pull
you can get help with the arguments running:

Code:

python3 Main.py -h
the expected result:

Code:

1299-root-INFO-Starting... platform=linux hostname=rpi-buster-4g
usage: Main.py [-h] [--ezbaddr EZBADDR] [--ezbport EZBPORT]
[--camaddr CAMADDR] [--camport CAMPORT] [--camwidth CAMWIDTH]
[--camheight CAMHEIGHT] [--camfps CAMFPS]
[--jpgquality JPGQUALITY] [--audio]
[--audiooutputindex AUDIOOUTPUTINDEX]
[--camtype [{none,picamera,videocapture}]]
[--videocaptureindex VIDEOCAPTUREINDEX] [--uart0 UART0]
[--uart1 UART1] [--uart2 UART2] [--pca9685 [{none,servo,pwm}]]
[--pantilthat] [--maestro MAESTRO]

optional arguments:
-h, --help show this help message and exit
--ezbaddr EZBADDR EZB Server IP address (default: 0.0.0.0)
--ezbport EZBPORT EZB Server TCP port (default: 10023)
--camaddr CAMADDR Camera Server IP Address (default: 0.0.0.0)
--camport CAMPORT Camera Server TCP Port (default: 10024)
--camwidth CAMWIDTH Camera Video's Width (default: 640)
--camheight CAMHEIGHT
Camera Video's Height (default: 480)
--camfps CAMFPS Camera Video's frames per second (default: 15)
--jpgquality JPGQUALITY
Jpeg's quality (0-100) (default: 95)
--audio enable audio output (default: False)
--audiooutputindex AUDIOOUTPUTINDEX
AudioOutput device index (default: 0)
--camtype [{none,picamera,videocapture}]
(default: none)
--videocaptureindex VIDEOCAPTUREINDEX
VideoCapture index (default: 0)
--uart0 UART0 UART 0's serial device e.g. /dev/serial0 com4
(default: None)
--uart1 UART1 UART 1's serial device e.g. /dev/serial0 com4
(default: None)
--uart2 UART2 UART 2's serial device e.g. /dev/serial0 com4
(default: None)
--pca9685 [{none,servo,pwm}]
servo=controller for servos, pwm=controller for pwm
ports (default: none)
--pantilthat enable Pimoroni Pan-Tilt HAT
https://shop.pimoroni.com/products/pan-tilt-hat
(default: False)
--maestro MAESTRO enable Pololu Maestro serial device e.g. /dev/ttyACM0
com40 (default: None)
by default EZB server Raspberry PI digital ports and I2C are enabled.

To enable the audio:

Code:

python3 Main.py --audio
To enable the Raspberry PI Camera:  

Code:

python3 Main.py --camtype=picamera
Supported Capabilities
  • Can stream Audio v4 codec
    Supports the streaming Audio v4 codec to play music, speech and sound effects out of the DAC on the controller.
  • Can stream video v4 codec
    The controller supports the streaming Video v4 codec on a TCP or UART or USB connection.
  • I2C Master
    The controller supports the EZ-Builder I2C commands for Master mode.
  • PWM servos on digital ports
    The controller supports PWM Servos on digital ports. These are also called Hobby servos.
  • PWM servos on digital ports can release their position
    PWM servo driver on digital ports support the feature to release their holding position.
  • Read/Write Digital I/O Ports
    The ports marked as being digital will respond to Read and Write commands of boolean logic. The status of the digital port will be either TRUE or FALSE when voltage is detected or not, respectively.
  • Servo speed for PWM servos on digital ports
    The PWM (Hobby) Servo driver supports speed commands to adjust the sweep speed on digital ports.

About this version
Audio Support:

Audio is supported through the existent Raspberry PI setup: Audio Jack, HDMI, I2S, USB.

Update your setup and install sound utils:

Code:

sudo apt-get update && sudo apt-get upgrade
sudo apt-get install alsa-utils
then run the following test:

Code:

speaker-test -c 2 -t wav
If you hear a lady's voice saying "FRONT LEFT" and "FRONT RIGHT", then your sound device is good and its software driver was loaded.

IF your audio is OK, you can enable the audio support Main.py line 155:

Code:

setup_PyAudioPlayerController()
remove the comment #

EZ-Builder Test setup:

User-inserted image

Connect, and the press Say (EZ-B)
Click to show supported capabilities
Supported Capabilities
  • Can stream Audio v4 codec
    Supports the streaming Audio v4 codec to play music, speech and sound effects out of the DAC on the controller.
  • Can stream video v4 codec
    The controller supports the streaming Video v4 codec on a TCP or UART or USB connection.
  • I2C Master
    The controller supports the EZ-Builder I2C commands for Master mode.
  • PWM servos on digital ports
    The controller supports PWM Servos on digital ports. These are also called Hobby servos.
  • PWM servos on digital ports can release their position
    PWM servo driver on digital ports support the feature to release their holding position.
  • Read/Write Digital I/O Ports
    The ports marked as being digital will respond to Read and Write commands of boolean logic. The status of the digital port will be either TRUE or FALSE when voltage is detected or not, respectively.
  • Servo speed for PWM servos on digital ports
    The PWM (Hobby) Servo driver supports speed commands to adjust the sweep speed on digital ports.

About this version
Digital Input/Output:

Digital IO is supported through the Raspberry PI GPIO.  
The EZB protocol only supports ports 0 to 23. the raspberry PI has GPIO # greater than 23, and some digital ports have optional features: I2C, SPI0, SPI1/I2S, to avoid interference/damaging the optional features the software allows mapping the EZB ports 0..23 to the RPI GPiOS.

Current EZB-RPI digital port map:
User-inserted image

Notes:
Blue: D4,D5,D6,D12,D13,D22,D23 are equivalent to GPI (4,5,6,12,13,22,23)
Red: D0,D1,D2,D3 are mapped to GPIO24,GPIO25,GPIO26,GPIO27

map is done on Main.py (lines #78-89):

Code:

def setup_digital_ports():
if sys.platform == "linux" or sys.platform == "linux2":
import RpiGPIODigitalController
com = RpiGPIODigitalController.RpiGPIODigitalController(logging.DEBUG)
ComponentRegistry.ComponentRegistry.register_controller(com)
#+-----+---------+--B Plus--+-----------+-----+
#| BCM | Name | Physical | Name | BCM |
#+-----+---------+----++----+-----------+-----+
#| | 3.3v | 1 || 2 | 5v | |
#| 2 | SDA.1 | 3 || 4 | 5v | |
#| 3 | SCL.1 | 5 || 6 | GND | |
#| 4 | | 7 || 8 | TxD | 14 |
#| | GND | 9 || 10 | RxD | 15 |
#| 17 | CE1.1 | 11 || 12 | CE0.1/BCLK| 18 |
#| 27 | | 13 || 14 | GND | |
#| 22 | | 15 || 16 | | 23 |
#| | 3.3v | 17 || 18 | | 24 |
#| 10 | MO.0 | 19 || 20 | GND | |
#| 9 | MI.0 | 21 || 22 | | 25 |
#| 11 | CLK.0 | 23 || 24 | CE0.0 | 8 |
#| | GND | 25 || 26 | CE1.0 | 7 |
#| 0 | SDA.0 | 27 || 28 | SCL.0 | 1 |
#| 5 | | 29 || 30 | GND | |
#| 6 | | 31 || 32 | | 12 |
#| 13 | | 33 || 34 | GND | |
#| 19 |LRCK/MI.1| 35 || 36 | CE2.1 | 16 |
#| 26 | | 37 || 38 | MO.1/SDI | 20 |
#| | GND | 39 || 40 | CLK.1/SDO | 21 |
#+-----+---------+----++----+-----------+-----+
#| BCM | Name | Physical | Name | BCM |
#+-----+---------+--B Plus--+-----------+-----+
#Generic pins (excluded: uart, i2c, spi, i2s):
ComponentRegistry.ComponentRegistry.register_component("D4", DigitalController.DigitalPort(com, 4))
ComponentRegistry.ComponentRegistry.register_component("D5", DigitalController.DigitalPort(com, 5))
ComponentRegistry.ComponentRegistry.register_component("D6", DigitalController.DigitalPort(com, 6))
ComponentRegistry.ComponentRegistry.register_component("D12", DigitalController.DigitalPort(com, 12))
ComponentRegistry.ComponentRegistry.register_component("D13", DigitalController.DigitalPort(com, 13))
ComponentRegistry.ComponentRegistry.register_component("D22", DigitalController.DigitalPort(com, 22))
ComponentRegistry.ComponentRegistry.register_component("D23", DigitalController.DigitalPort(com, 23))
#Remapped
ComponentRegistry.ComponentRegistry.register_component("D0", DigitalController.DigitalPort(com, 24))
ComponentRegistry.ComponentRegistry.register_component("D1", DigitalController.DigitalPort(com, 25))
ComponentRegistry.ComponentRegistry.register_component("D2", DigitalController.DigitalPort(com, 26))
ComponentRegistry.ComponentRegistry.register_component("D3", DigitalController.DigitalPort(com, 27))
com.start()
else:
import FakeDigitalController
com = FakeDigitalController.FakeDigitalController(logging.DEBUG)
ComponentRegistry.ComponentRegistry.register_controller(com)
for port in range(24):
ComponentRegistry.ComponentRegistry.register_component("D" + str(port), DigitalController.DigitalPort(com, port))
com.start()

The digital port implementation can be enabled/disabled in the code Main.py line #157:

enabled:

Code:

setup_digital_ports()
disabled:

Code:

#setup_digital_ports()

Very important:
Raspberry PI does not have resistors.
You shouldn't alternate the Port direction (Input / Output) on the EZ-Builder, one silly example: Two controls one ReadDigital and SetDigital both on same port. You connect a Button (Output) to be used as an Input (ReadDigital), then you configure the same port as SetDigital (Output), if you proceed with that setup/configuration you will be connecting two outputs together when you use the SetDigital port and that is not good.
You can damage the port and/or the Raspberry PI.
Click to show supported capabilities
Supported Capabilities
  • Can stream video v4 codec
    The controller supports the streaming Video v4 codec on a TCP or UART or USB connection.
  • I2C Master
    The controller supports the EZ-Builder I2C commands for Master mode.
  • PWM servos on digital ports
    The controller supports PWM Servos on digital ports. These are also called Hobby servos.
  • PWM servos on digital ports can release their position
    PWM servo driver on digital ports support the feature to release their holding position.
  • Read/Write Digital I/O Ports
    The ports marked as being digital will respond to Read and Write commands of boolean logic. The status of the digital port will be either TRUE or FALSE when voltage is detected or not, respectively.
  • Servo speed for PWM servos on digital ports
    The PWM (Hobby) Servo driver supports speed commands to adjust the sweep speed on digital ports.

About this version
I2C Support:

https://www.raspberrypi.org/forums/viewtopic.php?t=7664

Quote:

Since the RPi board already has 1.8K resistors on the I2C lines, any pull-ups included on a breakout board are superfluous and can be removed. On the other hand, they probably won't cause any problems as long as they aren't too big or too small in value.
1) 18 RGB LED Block  https://www.ez-robot.com/Shop/AccessoriesDetails.aspx?prevCat=104&productNumber=1357
Is not detected ! Does not work.
I don't know why maybe the existent pull-up resistors 4.7K should be removed ? Maybe the add-on MCU firmware needs some tweak.

2) EZ-Robot MPU6050:
User-inserted image


EZ-Script:

Code:

:loop
ControlCommand("MPU6050", RunOnce)
Sleep(100)
goto(loop)
Setup:
User-inserted image


Wiring:  
User-inserted image


Software control/logic:
EZ-Builder MPU6050 control.
Click to show supported capabilities
Supported Capabilities
  • Can stream video v4 codec
    The controller supports the streaming Video v4 codec on a TCP or UART or USB connection.
  • I2C Master
    The controller supports the EZ-Builder I2C commands for Master mode.
  • PWM servos on digital ports
    The controller supports PWM Servos on digital ports. These are also called Hobby servos.
  • PWM servos on digital ports can release their position
    PWM servo driver on digital ports support the feature to release their holding position.
  • Servo speed for PWM servos on digital ports
    The PWM (Hobby) Servo driver supports speed commands to adjust the sweep speed on digital ports.

About this version
Servo support:

Add two servos (D0 and D1):
User-inserted image


Press Center on D0 servo expected debug:
User-inserted image


Press Center on D1 servo expected debug:
User-inserted image


So that's it, your fake servos are working !!!

Support for servo add-ons (controllers):
1) Pololu Maestro usb controllers e.g. 24-Channel USB servo Controller https://www.pololu.com/product/1356
Uncomment the line (remove #) 166 on Main.py:

Code:

setup_serial_MaestroServoController("/dev/ttyACM0" if sys.platform == "linux" or sys.platform == "linux2" else "com40")
2) Adafruit 16-Channel PWM https://www.adafruit.com/product/2327 or a generic PCA9685 board
Uncomment the line (remove #) 172 on Main.py:

Code:

setup_i2c_PCA9685ServoController(i2c_com)
3) Pimoroni Pan-Tilt HAT https://shop.pimoroni.com/products/pan-tilt-hat
Uncomment the line (remove #) 176 on Main.py:

Code:

setup_i2c_PimoroniPanTiltHatServoController(i2c_com)

Configure servo timings:

Servos have different timings you should adapt the settings (min and max) to your needs.
Some examples:

Code:

#ez-robot servos: 560-2140 us
ComponentRegistry.ComponentRegistry.register_component("S"+str(port), ServoController.ServoPort(com, port, 560, 2140))
Relevant values: 560 and 2140 they should match your Min (1) and Max (180) servo position 

Pimoroni Pan-Tilt HAT:

Code:

ComponentRegistry.ComponentRegistry.register_component("S"+str(port), ServoController.ServoPort(com, port, 575, 2325))
Relevant values: 575 and 2325 they should match your Min (1) and Max (180) servo position
Click to show supported capabilities
Supported Capabilities
  • Can stream video v4 codec
    The controller supports the streaming Video v4 codec on a TCP or UART or USB connection.
  • PWM servos on digital ports
    The controller supports PWM Servos on digital ports. These are also called Hobby servos.

About this version
Required components:

Code:

sudo apt-get install python3  python3-pyaudio


Clone the source code repository on your raspberry pi home folder:

Code:

cd ~/
git clone https://github.com/ppedro74/BlueberryServer.git
Click to show supported capabilities
Supported Capabilities
  • Can stream video v4 codec
    The controller supports the streaming Video v4 codec on a TCP or UART or USB connection.
#1  
@ptp, I have made some tests.
I succeeded in launching the server and connecting ARC and the PiCam. The good news is the camera works very well, with all the fun stuff (color tracking...), with no latency.
I have also succeeded in adding D0 & D1 servos, even if I don't know what to do with it :D.
I still cannot use a Digital control to switch a LED, but i don't know if it has been implemented yet...

I will test it furthermore tomorrow (it is late now in France)
#2   — Edited

Quote:

I have also succeeded in adding D0 & D1 servos, even if I don't know what to do with it
To use Servos you will need extra hardware there are at least two I've used and are supported and third one (Pimoroni) I don't own one but i did a "blind" implementation and one member of the forum confirmed that works.
Without additional hardware, you have fake/dummy servos you can imagine them working:)

Quote:

I still cannot use a Digital control to switch a LED, but i don't know if it has been implemented yet...
Yes is implemented. I added a version 4 to explain that capability, bear in mind the ports you used are available through the existent mapping.
#3  
@fredebec:

I've Buster, updated. Please run the updates & upgrades.

Code:

sudo apt-get update && sudo apt-get upgrade

EZ-Builder:

User-inserted image


RPI3 Setup 1:
User-inserted image


RPI4b Setup (With additional hardware: Google Voice Hat):
User-inserted image


Both setups use the same D4,D5,D6 ports.

Wiring:
Yellow = 3v3,
Red = Gpio4
Green = Gpio5
Blue = Gpio6

Bear in mind RPI does not have resistors between the SOC ports and the headers like EZB.
The RGB breakout https://www.robotshop.com/en/rgb-led-breakout.html has 3 resistors and can be connected to 5v or 3v3.
#4  
@ptp:
The digital ports works ! I have successfully switch on my LED on my pi through ARC for the first time.

For the servos, now that the fake ones work (:D), I will try the generic PCA9685 board I have. I will play a little with it today....

Finally, I have an error when I enable audio support (the speaker-test was OK):

Code:

pi@fredebec-pi:~/BlueberryServer/Blueberry.Server.Python $ python3 Main.py
26189-root-INFO-Starting... platform=linux hostname=fredebec-pi
Traceback (most recent call last):
File "Main.py", line 197, in
main()
File "Main.py", line 155, in main
setup_PyAudioPlayerController()
File "Main.py", line 147, in setup_PyAudioPlayerController
com.start()
File "/usr/lib/python3.7/contextlib.py", line 239, in helper
return _GeneratorContextManager(func, args, kwds)
File "/usr/lib/python3.7/contextlib.py", line 82, in __init__
self.gen = func(*args, **kwds)
File "/home/pi/BlueberryServer/Blueberry.Server.Python/PyAudioPlayerController.py", line 82, in start
start=False)
File "/usr/lib/python3/dist-packages/pyaudio.py", line 750, in open
stream = Stream(self, *args, **kwargs)
File "/usr/lib/python3/dist-packages/pyaudio.py", line 441, in __init__
self._stream = pa.open(**arguments)
OSError: [Errno -9997] Invalid sample rate

Thanks a lot for this great BlueBerry
#5  

Quote:

Finally, I have an error when I enable audio support (the speaker-test was OK):
I fixed the error. You will need to update your repository:

Code:

cd ~/BlueBerryServer
git pull
If you have changes the above pull request may fail, Please copy or memorize the changes done e.g. activating the audio, and when you are ready run:

Code:

git reset --hard
the above commands will revert any local changes, then run (again):

Code:

git pull
After this pull the audio is disabled (commented) so you will need to enable the audio.

Let me know if it works for you too.
#6  
@ptp, it's OK now, no more error when I start the server.
However, I cannot use the the speech synthesis control. ARC tells me that "This control is not supported on this operating system". I think it is because I use ARC on the pi with mono, right ?
I will try with ARC on my win10 PC (not available right now...)
#7  
Just installed your BlueBerryServer, it runs like a charm!!!
Very fast camera stream, an no connection issues on my Rasberry Pi 3!!:D

But on my Raspberry Pi Zero the Camera is not being recognized...
1218-root-ERROR-Error loading PiCameraController ex=Failed to enable connection: Out of resources

I should mention, that it is the AIY Vision kit, so the camera is being routed thru the VisonBonnet!

P.S. I had to install psutil to get the whole thing working...
sudo pip3 install psutil
#8  
Could there be a possibility of swapping cams?
Let's say you have a USB and a normal Cam for the RPI and you want to switch.
#9  
I have some scripts used in another projects, I'll tweak them to allow streaming an USB camera.

Bear in mind the Raspberry PI cpu performance is even less than an old Intel Pentium, the reason why performs well video operations is due to a special combination of codecs and hardware design for example the Raspberry PI Camera blows away a regular USB camera.

Camera board vs. USB camera:
https://www.raspberrypi.org/forums/viewtopic.php?t=85899  

Nevertheless I'll came back to this topic.
#10  
@Reaper21:
update your repository (check version 6 details)
then you can run:

Code:

python3 Main.py --camtype=videocapture --videocaptureindex=1
videocaptureindex is used to specify the  camera device, in my setup index 0 is the raspberry pi camera and 1 is the usb camera.

you can troubleshoot your setup:
install the following packages:

Code:

sudo apt-get install v4l-utils ffmpeg
then you can list the video devices:

Code:

v4l2-ctl --list-devices
1) Output of a RPI 4 with USB camera:

Code:

bcm2835-codec-decode (platform:bcm2835-codec):
/dev/video10
/dev/video11
/dev/video12

USB Camera-B4.09.24.1 (usb-0000:01:00.0-1.4):
/dev/video0
the USB camera index is 0

2) Output of a RPI Zero with PI Camera and USB camera:

Code:

bcm2835-codec-decode (platform:bcm2835-codec):
/dev/video10
/dev/video11
/dev/video12

mmal service 16.1 (platform:bcm2835-v4l2):
/dev/video0

Microsoft LifeCam HD-5000: Mi (usb-20980000.usb-1.4):
/dev/video1
/dev/video2
The USB camera mounts two devices but only the first one videocaptureindex=1 is valid.

if you want you can obtain more details regarding a video device:

Code:

ffmpeg -f video4linux2 -list_formats all -i /dev/video1

#12  
Hi @ptp

I have a strange behavior with the server when picamera is enabled. As soon as I launch the server, the camera video occupies all the screen on top of every windows. The video is on all virtual desktops, as if it was a wallpaper, but on top of everything.

User-inserted image



The strange thing is that if i do a screen capture, the camera video is not captured... It is the same if i take control of the pi via realvnc, i don't see the video on the host computer.

User-inserted image



Do you have any idea of what is is happening?
Thanks

Fredebec
#13  
@Fredebec
Fixed. Please update your local repository: (git pull)

I left the camera preview active by mistake, when you have the preview enabled the PI overlays the camera feed on top of HDMI image output, is pure hardware, that is why you can't capture is not in the gpu memory.