Package rst.vision¶
In computer science, image processing is any form of signal processing for which the input is an image.
This package contains data types for representing images (Image) or things which can be recognized in images
(e.g. Face).
See also
- Wikipedia article containing the definition above
- http://en.wikipedia.org/wiki/Image_processing
Messages¶
Message SimpleXYZImageCollection¶
-
class
rst.vision.SimpleXYZImageCollection¶ Collection of
SimpleXYZImageinstances.Auto-generated.
-
element¶ Type: array of rst.vision.SimpleXYZImageThe individual elements of the collection.
Constraints regarding the empty collection, sorting, duplicated entries etc. are use case specific.
-
message SimpleXYZImageCollection {
/**
* The individual elements of the collection.
*
* Constraints regarding the empty collection, sorting, duplicated
* entries etc. are use case specific.
*/
repeated SimpleXYZImage element = 1;
}
Message LocatedXYZImageCollection¶
-
class
rst.vision.LocatedXYZImageCollection¶ Collection of
LocatedXYZImageinstances.Auto-generated.
-
element¶ Type: array of rst.vision.LocatedXYZImageThe individual elements of the collection.
Constraints regarding the empty collection, sorting, duplicated entries etc. are use case specific.
-
message LocatedXYZImageCollection {
/**
* The individual elements of the collection.
*
* Constraints regarding the empty collection, sorting, duplicated
* entries etc. are use case specific.
*/
repeated LocatedXYZImage element = 1;
}
Message LaserScanCollection¶
-
class
rst.vision.LaserScanCollection¶ Collection of
LaserScaninstances.Auto-generated.
-
element¶ Type: array of rst.vision.LaserScanThe individual elements of the collection.
Constraints regarding the empty collection, sorting, duplicated entries etc. are use case specific.
-
message LaserScanCollection {
/**
* The individual elements of the collection.
*
* Constraints regarding the empty collection, sorting, duplicated
* entries etc. are use case specific.
*/
repeated LaserScan element = 1;
}
Message LocatedFaceCollection¶
-
class
rst.vision.LocatedFaceCollection¶ Collection of
LocatedFaceinstances.Auto-generated.
-
element¶ Type: array of rst.vision.LocatedFaceThe individual elements of the collection.
Constraints regarding the empty collection, sorting, duplicated entries etc. are use case specific.
-
message LocatedFaceCollection {
/**
* The individual elements of the collection.
*
* Constraints regarding the empty collection, sorting, duplicated
* entries etc. are use case specific.
*/
repeated LocatedFace element = 1;
}
Message Faces¶
-
class
rst.vision.Faces¶ A list of faces with information about the dimensions of a source image used by a face detector. This allows interpretation of the bounding box data contained in the Face types.
Code author: Sebastian Wrede <swrede@techfak.uni-bielefeld.de>
-
faces¶ Type: array of rst.vision.FaceList of faces.
-
width¶ Type: UINT32Unit: pixel
Width of the source image.
-
height¶ Type: UINT32Unit: pixel
Height of the source image.
-
message Faces {
/**
* List of faces.
*/
repeated Face faces = 1;
/**
* Width of the source image.
*/
// @unit(pixel)
required uint32 width = 2;
/**
* Height of the source image.
*/
// @unit(pixel)
required uint32 height = 3;
}
Message AugmentedPointCloudSet3DFloat¶
-
class
rst.vision.AugmentedPointCloudSet3DFloat¶ A set of augmented 3D Point clouds.
Code author: Jan Moringen <jmoringe@techfak.uni-bielefeld.de>
-
clouds¶ Type: array of rst.vision.AugmentedPointCloud3DFloatEmpty collection of clouds is allowed. The order of cloud objects is not significant.
-
message AugmentedPointCloudSet3DFloat {
/**
* Empty collection of clouds is allowed.
* The order of cloud objects is not significant.
*/
repeated AugmentedPointCloud3DFloat clouds = 1;
}
Message AugmentedPointCloud3DFloat¶
-
class
rst.vision.AugmentedPointCloud3DFloat¶ Associates a set of scalar values (such as color information) to each point of a point cloud.
Code author: Johannes Wienke <jwienke@techfak.uni-bielefeld.de>
-
points¶ Type: rst.geometry.PointCloud3DFloatGeometry of the point cloud augmented by the data in
channels.
-
channels¶ Type: array of rst.vision.AugmentedPointCloud3DFloat.ChannelAn optional set of scalar information to attach to each point of the point cloud.
-
message AugmentedPointCloud3DFloat {
/**
* One channel of color information attached to a set of 3D points.
*
* Each channel contains a vector of the same size as the original point
* cloud and attaches the value of one color channel to each point of the
* target point cloud.
*
* The following conventions for channel names and respective values exist:
*
* "r", "g", "b"
*
* RGB values represented between 0 and 1.
*
* "intensity"
*
* Grayscale image value. Between 0 and 1.
*
* "u", "v"
*
* Forms YUV color information in combination with an
* "intensity" channel. Between 0 and 1.
*
* Inspired by the design of the sensor_msgs/ChannelFloat32 ROS
* message type.
*/
message Channel {
/**
* The name of the color channel.
*/
required string name = 1;
/**
* The actual values for the represented color channel for
* each point of the target point cloud.
*
* N-th value is associated to n-th point in @ref .points.
*/
// @constraint(len(value) = len(.points.points))
repeated float values = 2 [packed = true];
}
/**
* Geometry of the point cloud augmented by the data in @ref
* .channels.
*/
required .rst.geometry.PointCloud3DFloat points = 1;
/**
* An optional set of scalar information to attach to each point
* of the point cloud.
*/
repeated Channel channels = 2;
}
Message Channel¶
-
class
rst.vision.AugmentedPointCloud3DFloat.Channel¶ One channel of color information attached to a set of 3D points.
Each channel contains a vector of the same size as the original point cloud and attaches the value of one color channel to each point of the target point cloud.
The following conventions for channel names and respective values exist:
“r”, “g”, “b”
RGB values represented between 0 and 1.“intensity”
Grayscale image value. Between 0 and 1.“u”, “v”
Forms YUV color information in combination with an “intensity” channel. Between 0 and 1.Inspired by the design of the sensor_msgs/ChannelFloat32 ROS message type.
-
name¶ Type: ASCII-STRINGThe name of the color channel.
-
message Channel {
/**
* The name of the color channel.
*/
required string name = 1;
/**
* The actual values for the represented color channel for
* each point of the target point cloud.
*
* N-th value is associated to n-th point in @ref .points.
*/
// @constraint(len(value) = len(.points.points))
repeated float values = 2 [packed = true];
}
Message LocatedXYZImage¶
-
class
rst.vision.LocatedXYZImage¶ A simple point cloud represented in 2D structure with information from where it was taken.
By adding information about the pose of the camera and its perceived view frustum, one can reconstruct the location and the circumstances under which the provided point cloud was captured.
Code author: Leon Ziegler <lziegler@techfak.uni-bielefeld.de>
@create_collection
-
image¶ Type: rst.vision.SimpleXYZImageThe background model.
-
camera¶ Type: rst.geometry.CameraPoseThe camera’s pose in 3d space.
-
frustum¶ Type: rst.geometry.ViewFrustumThe camera’s view frustum.
-
message LocatedXYZImage {
/**
* The background model.
*/
required vision.SimpleXYZImage image = 1;
/**
* The camera's pose in 3d space.
*/
required geometry.CameraPose camera = 2;
/**
* The camera's view frustum.
*/
required geometry.ViewFrustum frustum = 3;
}
Message SimpleXYZImage¶
-
class
rst.vision.SimpleXYZImage¶ Constraint:
.width * .height = len(.x)Constraint:
.width * .height = len(.y)Constraint:
.width * .height = len(.z)Saves 3D data (X, Y, Z values) which have a 2D ordering.
This kind of depth-data is produced by e.g. Kinect or Swissranger cameras.
x,y,zare 1D arrays which save separately the X, Y, Z values of the 3D points.Code author: Agnes Swadzba <aswadzba@techfak.uni-bielefeld.de>
@create_collection
-
width¶ Type: UINT32Constraint:
value > 0Unit: pixel
Width of the underlying 2D ordering structure.
-
height¶ Type: UINT32Constraint:
value > 0Unit: pixel
Height of the underlying 2D ordering structure.
-
x¶ Type: array of FLOAT32The values are ordered by linearizing the underlying 2D structure row-wise.
-
message SimpleXYZImage {
/**
* Width of the underlying 2D ordering structure.
*/
// @constraint(value > 0)
// @unit(pixel)
required uint32 width = 1;
/**
* Height of the underlying 2D ordering structure.
*/
// @constraint(value > 0)
// @unit(pixel)
required uint32 height = 2;
/**
* The values are ordered by linearizing the underlying 2D structure row-wise.
*/
repeated float x = 3;
/**
* See @ref .x.
*/
repeated float y = 4;
/**
* See @ref .x.
*/
repeated float z = 5;
};
Message LaserScan¶
-
class
rst.vision.LaserScan¶ An object of this type represents a single laser scan carried out by an appropriate planar laser range-finder.
Code author: Leon Ziegler <lziegler@techfak.uni-bielefeld.de>
@create_collection
-
scan_values¶ Type: array of FLOAT32Unit: meter
A set of values coming from the laserscanning device.
-
scan_angle¶ Type: FLOAT32Unit: radian
The angular range that is covered by the scan.
This value depends on the used hardware and equals the sensor’s one-dimensional field of view. The values from
scan_valuesare evenly distributed over this angular range.
-
message LaserScan {
/**
* A set of values coming from the laserscanning device.
*/
// @unit(meter)
repeated float scan_values = 1;
/**
* The angular range that is covered by the scan.
*
* This value depends on the used hardware and equals the sensor's
* one-dimensional field of view. The values from @ref
* .scan_values are evenly distributed over this angular range.
*/
// @unit(radian)
required float scan_angle = 2;
}
Message Images¶
-
class
rst.vision.Images¶ A collection of images.
Code author: Johannes Wienke <jwienke@techfak.uni-bielefeld.de>
-
images¶ Type: array of rst.vision.ImageThe images contained in the collection.
Order of elements does not matter.
-
message Images {
/**
* The images contained in the collection.
*
* Order of elements does not matter.
*/
repeated Image images = 1;
}
Message LocatedFace¶
-
class
rst.vision.LocatedFace¶ This type extends the regular Face type with a 3D location.
Code author: Leon Ziegler <lziegler@techfak.uni-bielefeld.de>
@create_collection
-
face¶ Type: rst.vision.FaceThe face as a part of an image.
-
location¶ Type: rst.geometry.TranslationThe 3d location of the face.
-
orientation¶ Type: rst.geometry.RotationThe 3d orientation of the face. The provided rotation should be passed as a relative rotation from the default pose when the person looks straight ahead. In this case the axes of the corresponding coordinate system are defined as follows:
- x axis: points in the direction of the person’s view
- y axis: points to the person’s right side
- z axis: points downwards
-
message LocatedFace {
/**
* The face as a part of an image.
*/
required vision.Face face = 1;
/**
* The 3d location of the face.
*/
optional geometry.Translation location = 2;
/**
* The 3d orientation of the face. The provided rotation should be passed
* as a relative rotation from the default pose when the person looks
* straight ahead. In this case the axes of the corresponding coordinate
* system are defined as follows:
*
* * x axis: points in the direction of the person's view
* * y axis: points to the person's right side
* * z axis: points downwards
*/
optional geometry.Rotation orientation = 3;
}
Message Face¶
-
class
rst.vision.Face¶ An object of this type represents a human face detected in an image.
The region of the image which corresponds to the face is represented as a
BoundingBox.Code author: Johannes Wienke <jwienke@techfak.uni-bielefeld.de>
-
region¶ Type: rst.geometry.BoundingBoxBounding box defining the location of the detected face in a camera image.
-
confidence¶ Type: FLOAT64Constraint:
0 <= value <= 1Indicates the confidence of the detector that this detection result is actually a real face. Higher values indicate a higher confidence.
-
message Face {
/**
* Bounding box defining the location of the detected face in a
* camera image.
*/
required geometry.BoundingBox region = 1;
/**
* Indicates the confidence of the detector that this detection
* result is actually a real face. Higher values indicate a higher
* confidence.
*/
// @constraint(0 <= value <= 1)
optional double confidence = 2;
}
Message Image¶
-
class
rst.vision.Image¶ An uncompressed image with a certain pixel format, color and depth.
The binary intensity data is contained in the
datafield. The remaining fields contain information about the interpretation, that is dimensions (widthxheight), color (color_mode,channels), depth (depth) and channel layout (data_order,channels), of that data.Suggested interpretation of RSB timestamps:
create
Image grab timeTodo
- to be more precisely defined. e.g. what does this mean
- for rolling shutter cameras?
Code author: Johannes Wienke <jwienke@techfak.uni-bielefeld.de>
-
data¶ Type: OCTET-VECTORConstraint:
len(value) = (.width * .height * .channels * .depth) / 8Stores the actual pixel data of the image.
The number of bytes and their interpretation depend on the values of the remaining fields.
-
width¶ Type: UINT32Unit: pixel
The number of pixels in each row of the image.
-
height¶ Type: UINT32Unit: pixel
The number of pixels in each column of the image.
-
channels¶ Type: UINT32Unit: number
The number of color channels in the image.
The interpretation of these channels depends on
color_mode.
-
depth¶ Type: rst.vision.Image.DepthThe number of bits used to encode
-
color_mode¶ Type: rst.vision.Image.ColorModeColorspace and layout of the color channels of the image.
Determines pixel layout in combination with
data_order.
-
data_order¶ Type: rst.vision.Image.DataOrderLayout of the color planes of the image.
Determines pixel layout in combination with
color_mode.
message Image {
/**
* Depth of one color channel.
*
* Reflecting the definitions from OpenCV's IplImage.
*/
enum Depth {
DEPTH_8U = 8;
DEPTH_16U = 16;
DEPTH_32F = 32;
}
/**
* Colorspace(s) and layout of color channel(s).
*/
enum ColorMode {
/**
* The image consists of only one channel containing intensity
* values.
*
* MUST be used with @ref .channels = 1.
*/
COLOR_GRAYSCALE = 0;
/**
* The image has three color channels containing the red,
* green and blue color components (note the order of the
* channels, not to be confused with @ref .COLOR_BGR).
*
* MUST be used with @ref .channels = 3.
*/
COLOR_RGB = 1;
/**
* The image has three color channels containing the blue,
* green and red color components (note the order of the
* channels, not to be confused with @ref .COLOR_RGB).
*
* MUST be used with @ref .channels = 3.
*/
COLOR_BGR = 2;
/**
* The image has three color channels, encoded in the YUV
* colorspace, and uses the YUVYUV pixel layout.
*
* MUST be used with @ref .channels = 3.
*/
COLOR_YUV = 4;
/**
* The image has three color channels, encoded in the YUV
* colorspace, and uses the YUYV pixel layout.
*
* MUST be used with @ref .channels = 3.
*/
COLOR_YUV422 = 8;
}
/**
* Describes storage layout of pixels with respect to channels.
*/
enum DataOrder {
/**
* Separate planes.
*
* Example for @ref .ColorMode.COLOR_RGB:
*
* .. parsed-literal::
*
* R R R ... (@ref .width x @ref .height times)
* G G G ... (@ref .width x @ref .height times)
* B B B ... (@ref .width x @ref .height times)
*/
DATA_SEPARATE = 0;
/**
* Interleaved image.
*
* Example for @ref .ColorMode.COLOR_RGB:
*
* .. parsed-literal::
*
* R G B R G B R G B ... (@ref .width x @ref .height times)
*/
DATA_INTERLEAVED = 1;
}
/**
* Stores the actual pixel data of the image.
*
* The number of bytes and their interpretation depend on the
* values of the remaining fields.
*/
// @constraint(len(value) = (.width * .height * .channels * .depth) / 8)
required bytes data = 1;
/**
* The number of pixels in each row of the image.
*/
// @unit(pixel)
required uint32 width = 2;
/**
* The number of pixels in each column of the image.
*/
// @unit(pixel)
required uint32 height = 3;
/**
* The number of color channels in the image.
*
* The interpretation of these channels depends on @ref
* .color_mode.
*/
// @unit(number)
optional uint32 channels = 4 [default = 3];
/**
* The number of bits used to encode
*/
optional Depth depth = 5 [default = DEPTH_8U];
/**
* Colorspace and layout of the color channels of the image.
*
* Determines pixel layout in combination with @ref .data_order.
*/
optional ColorMode color_mode = 6 [default = COLOR_RGB];
/**
* Layout of the color planes of the image.
*
* Determines pixel layout in combination with @ref .color_mode.
*/
optional DataOrder data_order = 7 [default = DATA_SEPARATE];
}
Message Depth¶
enum Depth {
DEPTH_8U = 8;
DEPTH_16U = 16;
DEPTH_32F = 32;
}
Message ColorMode¶
-
class
rst.vision.Image.ColorMode¶ Colorspace(s) and layout of color channel(s).
-
COLOR_GRAYSCALE¶ -
= 0 The image consists of only one channel containing intensity values.
MUST be used with
channels= 1.
-
COLOR_RGB¶ -
= 1 The image has three color channels containing the red, green and blue color components (note the order of the channels, not to be confused with COLOR_BGR).
MUST be used with
channels= 3.
-
COLOR_BGR¶ -
= 2 The image has three color channels containing the blue, green and red color components (note the order of the channels, not to be confused with COLOR_RGB).
MUST be used with
channels= 3.
-
enum ColorMode {
/**
* The image consists of only one channel containing intensity
* values.
*
* MUST be used with @ref .channels = 1.
*/
COLOR_GRAYSCALE = 0;
/**
* The image has three color channels containing the red,
* green and blue color components (note the order of the
* channels, not to be confused with @ref .COLOR_BGR).
*
* MUST be used with @ref .channels = 3.
*/
COLOR_RGB = 1;
/**
* The image has three color channels containing the blue,
* green and red color components (note the order of the
* channels, not to be confused with @ref .COLOR_RGB).
*
* MUST be used with @ref .channels = 3.
*/
COLOR_BGR = 2;
/**
* The image has three color channels, encoded in the YUV
* colorspace, and uses the YUVYUV pixel layout.
*
* MUST be used with @ref .channels = 3.
*/
COLOR_YUV = 4;
/**
* The image has three color channels, encoded in the YUV
* colorspace, and uses the YUYV pixel layout.
*
* MUST be used with @ref .channels = 3.
*/
COLOR_YUV422 = 8;
}
Message DataOrder¶
enum DataOrder {
/**
* Separate planes.
*
* Example for @ref .ColorMode.COLOR_RGB:
*
* .. parsed-literal::
*
* R R R ... (@ref .width x @ref .height times)
* G G G ... (@ref .width x @ref .height times)
* B B B ... (@ref .width x @ref .height times)
*/
DATA_SEPARATE = 0;
/**
* Interleaved image.
*
* Example for @ref .ColorMode.COLOR_RGB:
*
* .. parsed-literal::
*
* R G B R G B R G B ... (@ref .width x @ref .height times)
*/
DATA_INTERLEAVED = 1;
}
