[Solved] Access to dev/video0 for Skill

eric.jastram 2017-08-30 05:52:14 UTC #1

So we have gotten to a point in our project building on a previously implemented video service… we am using ffmpeg to stream video from /dev/video0 to a url which is receiving the video stream… it works fine when i run through the terminal, but when i run it from my go implementation, I get " Cannot open video device /dev/video0: Permission denied"… I can only assume this is because my skill is ran in sh as a lower level user… if i chmod 777 /dev/video0, it works as expected. how would i go about gaining access to this video device in my skill?

How to speed up camera video capture

Important Upgrade: MIND Version 0.5.3 & Simulator Version 0.1.2

renee 2017-08-30 06:01:34 UTC #2

We are hoping to achieve a mpeg1 stream to an HTTP server so if there is another way this might be accomplished any tips would be appreciated! TIA

Mingzhao.Xu 2017-08-30 06:34:40 UTC #3

Hi, eric

you are correct, skills are not running as root, and we are not planning to open permission of device files to all skills directly because of security reason.

Currently, for the camera, we only provide functions to take photos and videos, but no protocols for real-time video stream now, we are still working on it, sorry about this.

I think an acceptable way now to have a video stream is chmod on your own HEXA to allow skills access video device. Thanks!

eric.jastram 2017-08-30 07:21:32 UTC #4

So this would mean our skill would not be able to be distributed on the skill store?
I completely agree that allowing full access to /dev would be a terrible security concern, but the camera is a core piece of the functionality of this robot, and a key ingredient for using the robot for any type of remote control.

Mingzhao.Xu 2017-08-30 09:25:50 UTC #5

You raised a very good point. This feature will be an important step in our roadmap.

Currently, if your skill requires changing device file’s permission in MIND, the skill cannot be distributed from Skill Store.

We will release a new version of MIND which adds a simple API to allow skills to get the permission of low-level device files. You can do this manually on your HEXA as you are developing right now, and we’ll let you know once we release the new version.

In the long term, we will have protocols in MIND system to fully control video devices and also other devices files, in this way it will be more secure, and we can optimize the performance continuously. As I said before, this will be a very important step in our roadmap.

renee 2017-08-31 02:42:49 UTC #6

One thing to note about the development of our Skill. It’s essentially ready except we cannot send mpeg1 video to the LetsRobot.tv servers without elevated privileges.

The anticipated API updates are the only roadblock for the new HEXA skill.

github.com

r3n33/HexaLetsRobotSkill/blob/master/robot/src/letsrobotskill.go

package LetsRobotSkill

import (
	"net/http"
	"io/ioutil"
	"strconv"
	"encoding/json"

	"mind/core/framework/drivers/hexabody"
	"mind/core/framework/drivers/infrared"
	"mind/core/framework/log"
	"mind/core/framework/skill"

	"github.com/graarh/golang-socketio/transport"
	"github.com/graarh/golang-socketio"
)

type RobotCommand struct {
	RobotId 	string 	`json:"robot_id"`
	Command 	string 	`json:"command"`

This file has been truncated. show original

Looking forward to the updates! Thank you!

Mingzhao.Xu 2017-08-31 06:45:02 UTC #7

The skill is cool! Control HEXA with socket.io in real-time looks like a good idea.

We’ll release new version contains the feature we talk about and some other updates in about 1 week, and we’ll let you know once it’s released.

Before that, you can continue developing and testing by changing your own HEXA’s permission. Thanks for your understanding!

eric.jastram 2017-09-11 02:12:18 UTC #8

So with the new video permissions enabled we now get a different error when running our skill… if we run our skill first or before running any other skills we get “[video4linux2,v4l2 @ 0x146b2e0] ioctl(VIDIOC_S_PARM): Invalid argument
/dev/video0: Invalid argument”… but… after running the explorer skill, this error doesn’t print, but also, in both instances, the video never streams… at this point i am loosing my mind trying to figure out what changed and will have to take a break.

note: i have included the following device permissions “video”,“disk”,“netdev”

Mingzhao.Xu 2017-09-11 04:21:53 UTC #9

@eric.jastram

we tested our new version with example of go-opencv and it works, so I think there’s no problem to access /dev/video0 file.

You mentioned run through the terminal on HEXA works, right? I think you can test to use Command of package os/exec in your skill to run the worked binary before. If there is no error I guess the problem is you’ve missed some configure step of video devices.

Also, you can try to add a new user to “video” group on Linux terminal, su new-user and then run your-binary-command to see if it can work, what we’ve done for skills is almost like this.

I hope these can help you. If all of these can not work, some pieces of code would help us to understand what’s happening a lot. Thanks!

eric.jastram 2017-09-19 02:05:05 UTC #10

after taking several days break from the hexa, the video device is working. thank you

gun_sun 2018-04-29 15:41:36 UTC #11

hi @eric.jastram,
what about the fps? It cost 200+ ms to get a YUV420 frame by v4l2. what’s the fps in your skill?
It cost 100+ ms to encode a YUV420 frame to flv1 by ffmpeg libs. Is it same to your skill?

Thanks.

eric.jastram 2018-04-29 17:08:40 UTC #12

It has been a while since I played with that project., but frame rate was never an issue for us… running at 24fps, nice and smooth… but there was a delay of 400-600ms…