Package rst.vision¶
Not documented
Messages¶
Message FaceWithGazeCollection¶
-
class
rst.vision.
FaceWithGazeCollection
¶ Collection of
FaceWithGaze
instances.Auto-generated.
-
element
¶ Type: array of rst.vision.FaceWithGaze
The individual elements of the collection.
Constraints regarding the empty collection, sorting, duplicated entries etc. are use case specific.
-
message FaceWithGazeCollection {
/**
* The individual elements of the collection.
*
* Constraints regarding the empty collection, sorting, duplicated
* entries etc. are use case specific.
*/
repeated FaceWithGaze element = 1;
}
Message FaceLandmarksCollection¶
-
class
rst.vision.
FaceLandmarksCollection
¶ Collection of
FaceLandmarks
instances.Auto-generated.
-
element
¶ Type: array of rst.vision.FaceLandmarks
The individual elements of the collection.
Constraints regarding the empty collection, sorting, duplicated entries etc. are use case specific.
-
message FaceLandmarksCollection {
/**
* The individual elements of the collection.
*
* Constraints regarding the empty collection, sorting, duplicated
* entries etc. are use case specific.
*/
repeated FaceLandmarks element = 1;
}
Message LocatedFace¶
-
class
rst.vision.
LocatedFace
¶ This type extends the regular Face type with a 3D location.
Code author: Leon Ziegler <lziegler@techfak.uni-bielefeld.de>
@create_collection
-
face
¶ Type: rst.vision.Face
The face as a part of an image.
-
location
¶ Type: rst.geometry.Translation
The 3d location of the face.
-
orientation
¶ Type: rst.geometry.Rotation
The 3d orientation of the face. The provided rotation should be passed as a relative rotation from the default pose when the person looks straight ahead. In this case the axes of the corresponding coordinate system are defined as follows:
- x axis: points in the direction of the person’s view
- y axis: points to the person’s right side
- z axis: points downwards
-
message LocatedFace {
/**
* The face as a part of an image.
*/
required vision.Face face = 1;
/**
* The 3d location of the face.
*/
optional geometry.Translation location = 2;
/**
* The 3d orientation of the face. The provided rotation should be passed
* as a relative rotation from the default pose when the person looks
* straight ahead. In this case the axes of the corresponding coordinate
* system are defined as follows:
*
* * x axis: points in the direction of the person's view
* * y axis: points to the person's right side
* * z axis: points downwards
*/
optional geometry.Rotation orientation = 3;
}
Message FaceWithGaze¶
-
class
rst.vision.
FaceWithGaze
¶ An object of this type represents a human face detected in an image including gaze informations.
Code author: Birte Carlmeyer <bcarlmey@techfak.uni-bielefeld.de>
@create_collection
-
region
¶ Type: rst.vision.Face
The region of the image which corresponds to the face.
-
lid_closed
¶ Type: BOOL
If this is true, the eyelids are closed.
-
horizontal_gaze_estimation
¶ Type: FLOAT64
Unit: radian
Horizontal gaze estimation angle. A relative rotation from the default gaze when the person looks straight into the camera. Positive values means that the person is looking upwards and negative value means that the person is looking downwards.
-
vertical_gaze_estimation
¶ Type: FLOAT64
Unit: radian
Vertical gaze estimation angle. A relative rotation from the default gaze when the person looks straight into the camera. Positive values means that the person is looking to the right and negative values means that the person is looking to the left (from the persons point of view).
-
message FaceWithGaze {
/**
* The region of the image which corresponds to the face.
*/
required Face region = 1;
/**
* If this is true, the eyelids are closed.
*/
optional bool lid_closed = 2;
/**
* Horizontal gaze estimation angle. A relative rotation from the
* default gaze when the person looks straight into the camera.
* Positive values means that the person is looking upwards and
* negative value means that the person is looking downwards.
*/
// @unit(radian)
optional double horizontal_gaze_estimation = 3;
/**
* Vertical gaze estimation angle. A relative rotation from the
* default gaze when the person looks straight into the camera.
* Positive values means that the person is looking to the right and
* negative values means that the person is looking to the left
* (from the persons point of view).
*/
// @unit(radian)
optional double vertical_gaze_estimation = 4;
}
Message EncodedImage¶
-
class
rst.vision.
EncodedImage
¶ An image encoded in a certain format.
Code author: Viktor Richter <vrichter@techfak.uni-bielefeld.de>
-
encoding
¶ Type: rst.vision.EncodedImage.Encoding
The encoding used to encode an image into binary data.
-
data
¶ Type: OCTET-VECTOR
Holds the binary image data.
Can be decoded using the provided encoding information.
-
message EncodedImage {
/**
* Identifies an image/file encoding format.
*/
enum Encoding {
/**
* Portable Pixmap Format.
*
* An open uncompressed image file format.
* @see https://en.wikipedia.org/wiki/Netpbm_format
*/
PPM = 1;
/**
* Portable Network Graphics format (ISO/IEC 15948).
*
* An open lossless compressed image file format.
* @see https://en.wikipedia.org/wiki/Portable_Network_Graphics
*/
PNG = 2;
/**
* Also known as JPEG (ISO/IEC 10918).
*
* An image file format with adjustable lossy compression
* created by the Joint Photographic Experts Group. The
* compression is based on the discrete cosine transform.
* @see https://en.wikipedia.org/wiki/JPEG
*/
JPG = 3;
/**
* Also known as JPEG2000 (ISO/IEC 15444).
*
* An image file format with adjustable lossy compression
* created by the Joint Photographic Experts Group. This
* format superseeds JPG and can use discrete cosine transform
* or wavelet based methods for compression.
* @see https://en.wikipedia.org/wiki/JPEG_2000
*/
JP2 = 4;
/**
* Tagged Image File Format.
*
* A computer file format for raster graphics developed by
* adobe.
* @see https://en.wikipedia.org/wiki/Tagged_Image_File_Format
*/
TIFF = 5;
}
/**
* The encoding used to encode an image into binary data.
*/
required Encoding encoding = 2;
/**
* Holds the binary image data.
*
* Can be decoded using the provided encoding information.
*/
required bytes data = 1;
}
Message Encoding¶
-
class
rst.vision.EncodedImage.
Encoding
¶ Identifies an image/file encoding format.
-
PPM
¶ -
= 1
Portable Pixmap Format.
An open uncompressed image file format.
See also
-
PNG
¶ -
= 2
Portable Network Graphics format (ISO/IEC 15948).
An open lossless compressed image file format.
-
JPG
¶ -
= 3
Also known as JPEG (ISO/IEC 10918).
An image file format with adjustable lossy compression created by the Joint Photographic Experts Group. The compression is based on the discrete cosine transform.
See also
-
JP2
¶ -
= 4
Also known as JPEG2000 (ISO/IEC 15444).
An image file format with adjustable lossy compression created by the Joint Photographic Experts Group. This format superseeds JPG and can use discrete cosine transform or wavelet based methods for compression.
See also
-
TIFF
¶ -
= 5
Tagged Image File Format.
A computer file format for raster graphics developed by adobe.
-
enum Encoding {
/**
* Portable Pixmap Format.
*
* An open uncompressed image file format.
* @see https://en.wikipedia.org/wiki/Netpbm_format
*/
PPM = 1;
/**
* Portable Network Graphics format (ISO/IEC 15948).
*
* An open lossless compressed image file format.
* @see https://en.wikipedia.org/wiki/Portable_Network_Graphics
*/
PNG = 2;
/**
* Also known as JPEG (ISO/IEC 10918).
*
* An image file format with adjustable lossy compression
* created by the Joint Photographic Experts Group. The
* compression is based on the discrete cosine transform.
* @see https://en.wikipedia.org/wiki/JPEG
*/
JPG = 3;
/**
* Also known as JPEG2000 (ISO/IEC 15444).
*
* An image file format with adjustable lossy compression
* created by the Joint Photographic Experts Group. This
* format superseeds JPG and can use discrete cosine transform
* or wavelet based methods for compression.
* @see https://en.wikipedia.org/wiki/JPEG_2000
*/
JP2 = 4;
/**
* Tagged Image File Format.
*
* A computer file format for raster graphics developed by
* adobe.
* @see https://en.wikipedia.org/wiki/Tagged_Image_File_Format
*/
TIFF = 5;
}
Message FaceLandmarks¶
-
class
rst.vision.
FaceLandmarks
¶ FaceLandmarks represent the result of an 68 point face landmark detection.
For example images see: http://ibug.doc.ic.ac.uk/resources/facial-point-annotations/
For readability the 68 face landmarks are split into points of different face parts. The point order is preserved.
Code author: Viktor Richter <vrichter@techfak.uni-bielefeld.de>
@create_collection
-
jaw
¶ Type: array of rst.math.Vec2DInt
Constraint:
len(value) == 17
17 Points of the jaw.
jaw[0] = top-left point of the face jaw[8] = the center of the chin jaw[16] = top-right point of the face
-
nose
¶ Type: array of rst.math.Vec2DInt
Constraint:
len(value) == 4
4 Points of the nasal bridge.
nose[0] = top-point of the nasal bridge nose[3] = bottom-point of the nasal bridge (tip of the nose)
-
nose_wings
¶ Type: array of rst.math.Vec2DInt
Constraint:
len(value) == 5
5 Points of the nose wings.
nose_wings[0] = left end of the lower nasal edge nose_wings[2] = the center of the lower nasal edge nose_wings[4] = right end of the lower nasal edge
-
right_brow
¶ Type: array of rst.math.Vec2DInt
Constraint:
len(value) == 5
5 Points of the rigt brow.
right_brow[0] = left end of the right brow right_brow[4] = right end of the right brow
-
left_brow
¶ Type: array of rst.math.Vec2DInt
Constraint:
len(value) == 5
5 Points of the left brow.
left_brow[0] = left end of the left brow left_brow[4] = right end of the left brow
-
right_eye
¶ Type: array of rst.math.Vec2DInt
Constraint:
len(value) == 6
6 Points of the border of the right eye.
right_eye[0] = left peak of the right eye ... = right upper eye-lid right_eye[3] = right peak of the right eye ... = right lower eye-lid
-
left_eye
¶ Type: array of rst.math.Vec2DInt
Constraint:
len(value) == 6
6 Points of the border of the left eye.
right_eye[0] = left peak of the left eye ... = left upper eye-lid right_eye[3] = right peak of the left eye ... = left lower eye-lid
-
outer_lips
¶ Type: array of rst.math.Vec2DInt
Constraint:
len(value) == 12
12 Points of the outer border of the lips.
outer_lips[0] = left corner of the outer lips ... = left upper edge outer_lips[3] = center of the upper lip ... = right upper edge outer_lips[6] = right corner of the outer lips ... = right lower edge outer_lips[9] = center of the lower lip ... = left lower edge
-
inner_lips
¶ Type: array of rst.math.Vec2DInt
Constraint:
len(value) == 8
8 Points of the inner border of the lips.
inner_lips[0] = left corner of the inner lips inner_lips[2] = center of the upper lip inner_lips[4] = right corner of the inner lips inner_lips[6] = center of the lower lip
-
message FaceLandmarks {
/**
* 17 Points of the jaw.
*
* jaw[0] = top-left point of the face
* jaw[8] = the center of the chin
* jaw[16] = top-right point of the face
*/
// @constraint(len(value) == 17)
repeated .rst.math.Vec2DInt jaw = 1;
/**
* 4 Points of the nasal bridge.
*
* nose[0] = top-point of the nasal bridge
* nose[3] = bottom-point of the nasal bridge (tip of the nose)
*/
// @constraint(len(value) == 4)
repeated .rst.math.Vec2DInt nose = 2;
/**
* 5 Points of the nose wings.
*
* nose_wings[0] = left end of the lower nasal edge
* nose_wings[2] = the center of the lower nasal edge
* nose_wings[4] = right end of the lower nasal edge
*/
// @constraint(len(value) == 5)
repeated .rst.math.Vec2DInt nose_wings = 3;
/**
* 5 Points of the rigt brow.
*
* right_brow[0] = left end of the right brow
* right_brow[4] = right end of the right brow
*/
// @constraint(len(value) == 5)
repeated .rst.math.Vec2DInt right_brow = 4;
/**
* 5 Points of the left brow.
*
* left_brow[0] = left end of the left brow
* left_brow[4] = right end of the left brow
*/
// @constraint(len(value) == 5)
repeated .rst.math.Vec2DInt left_brow = 5;
/**
* 6 Points of the border of the right eye.
*
* right_eye[0] = left peak of the right eye
* ... = right upper eye-lid
* right_eye[3] = right peak of the right eye
* ... = right lower eye-lid
*/
// @constraint(len(value) == 6)
repeated .rst.math.Vec2DInt right_eye = 6;
/**
* 6 Points of the border of the left eye.
*
* right_eye[0] = left peak of the left eye
* ... = left upper eye-lid
* right_eye[3] = right peak of the left eye
* ... = left lower eye-lid
*/
// @constraint(len(value) == 6)
repeated .rst.math.Vec2DInt left_eye = 7;
/**
* 12 Points of the outer border of the lips.
*
* outer_lips[0] = left corner of the outer lips
* ... = left upper edge
* outer_lips[3] = center of the upper lip
* ... = right upper edge
* outer_lips[6] = right corner of the outer lips
* ... = right lower edge
* outer_lips[9] = center of the lower lip
* ... = left lower edge
*/
// @constraint(len(value) == 12)
repeated .rst.math.Vec2DInt outer_lips = 8;
/**
* 8 Points of the inner border of the lips.
*
* inner_lips[0] = left corner of the inner lips
* inner_lips[2] = center of the upper lip
* inner_lips[4] = right corner of the inner lips
* inner_lips[6] = center of the lower lip
*/
// @constraint(len(value) == 8)
repeated .rst.math.Vec2DInt inner_lips = 9;
}
Message HeadObjects¶
-
class
rst.vision.
HeadObjects
¶ A list of head objects.
Code author: Vasil Khalidov <vasil.khalidov@idiap.ch>
-
head_objects
¶ Type: array of rst.vision.HeadObject
Not documented
-
message HeadObjects {
repeated vision.HeadObject head_objects = 1;
}
Message HeadObject¶
-
class
rst.vision.
HeadObject
¶ Focus on image coordinate systems (vision-based).
A head object represented by its ID, location in the image and head rotation angles.
id : head object identifier
Todo
what does the above mean?
Code author: Vasil Khalidov <vasil.khalidov@idiap.ch>
-
tracking_info
¶ Type: rst.tracking.TrackingInfo
Not documented
-
region
¶ Type: rst.geometry.BoundingBox
Head location in the input image.
-
position
¶ Type: rst.math.Vec3DFloat
Not documented
-
pose
¶ Type: rst.math.Vec3DFloat
x
-> panPan head rotation angle. Positive: person looks to her or his right side.y
-> tiltTilt head rotation angle. Positive: person looks up.z
-> rollRoll head rotation angle. Positive: person’s head rolled to his right shoulder.
-
faces
¶ Type: array of rst.vision.HeadObject.LabeledFace
Todo
semantic description
Repeated field because in cases of stereo processing there might be faces from both cameras.
-
speaking_probability
¶ Type: FLOAT32
Not documented
-
vfoa_target
¶ Type: OCTET-VECTOR
String description of the visual focus of attention of this head.
-
identity
¶ Type: rst.classification.ClassificationResult
Not documented
-
gender
¶ Type: rst.classification.ClassificationResult
Not documented
-
age
¶ Type: rst.classification.ClassificationResult
Not documented
-
message HeadObject {
optional tracking.TrackingInfo tracking_info = 1;
/**
* Head location in the input image.
*/
optional geometry.BoundingBox region = 2;
optional math.Vec3DFloat position = 3;
/**
* @ref .x -> pan
*
* Pan head rotation angle. Positive: person looks to her or his
* right side.
*
* @ref .y -> tilt
*
* Tilt head rotation angle. Positive: person looks up.
*
* @ref .z -> roll
*
* Roll head rotation angle. Positive: person's head rolled to
* his right shoulder.
*/
optional math.Vec3DFloat pose = 4;
message LabeledFace {
required bytes label = 1;
required vision.Face face = 2;
}
/**
* @todo "semantic description"
*
* Repeated field because in cases of stereo processing there
* might be faces from both cameras.
*/
repeated LabeledFace faces = 6;
optional float speaking_probability = 7;
/**
* String description of the visual focus of attention of this
* head.
*/
optional bytes vfoa_target = 8;
optional classification.ClassificationResult identity = 9;
optional classification.ClassificationResult gender = 10;
optional classification.ClassificationResult age = 11;
}
Message LabeledFace¶
-
class
rst.vision.HeadObject.
LabeledFace
¶ Not documented
-
label
¶ Type: OCTET-VECTOR
Not documented
-
face
¶ Type: rst.vision.Face
Not documented
-
message LabeledFace {
required bytes label = 1;
required vision.Face face = 2;
}
Message Face¶
-
class
rst.vision.
Face
¶ An object of this type represents a human face detected in an image.
The region of the image which corresponds to the face is represented as a
BoundingBox
.Code author: Johannes Wienke <jwienke@techfak.uni-bielefeld.de>
-
region
¶ Type: rst.geometry.BoundingBox
Bounding box defining the location of the detected face in a camera image.
-
confidence
¶ Type: FLOAT64
Constraint:
0 <= value <= 1
Indicates the confidence of the detector that this detection result is actually a real face. Higher values indicate a higher confidence.
-
message Face {
/**
* Bounding box defining the location of the detected face in a
* camera image.
*/
required geometry.BoundingBox region = 1;
/**
* Indicates the confidence of the detector that this detection
* result is actually a real face. Higher values indicate a higher
* confidence.
*/
// @constraint(0 <= value <= 1)
optional double confidence = 2;
}