Tutorial

Recording

Dealing with Errors

See also

Error-handling-related Commandline Options
Commandline options for controlling behavior in case of (serious) errors.

When recording data, an important decision to make is how to deal with errors. In RSBag, there are two high-level possibilities:

Abort the entire recording as soon as the first error is detected

This approach has the advantage that error will not go unnoticed. On the downside, data may be lost unnecessarily when a recording is aborted due to a minor error.

Try to continue, ignoring all errors

This obvious disadvantage of this approach is that errors may go unnoticed during recording and lead to unpleasant surprises later. The potential advantage consists in the fact that recording may continue and produce useful result despite some errors. This approach may make sense when recording data which is hard/expensive to produce.

These strategies are chosen via the on-error commandline options.

Recording using a local Spread daemon

This example assumes

  • A Spread daemon running on the local machine and listening on port 4803
  • Image, Sound and Proprioception message are published (for example by running the streaming components for Nao on the RSB channels
    • /nao/vision/top
    • /nao/audio/all
    • /nao/proprioception
  • Image, Sound and Proprioception message are encoded using the RST data types
    • rst.vision.Image
    • rst.audition.SoundChunk
    • rst.kinematics.ProprioceptionState
$ bag record -o /tmp/nao.tide
             'spread:/nao/vision/top?name=4803'
             'spread:/nao/audio/all?name=4803'
             'spread:/nao/proprioception?name=4803'

The resulting file will (unless a different channel allocation is specified) contain one channel for each of the three RSB channels.

Note

The file extension (“tide” in this case) determines the used file format backend. Currently, only the TIDELog file format is supported.

Recording using a remote Spread daemon

This example assumes

  • A Spread daemon running on the machine “remote” and listening on port 4803
  • Same data types as above
$ ./bag record -o /tmp/nao.tide
               'spread://remote:4803/nao/vision/top'
               'spread://remote:4803/nao/audio/all'
               'spread://remote:4803/nao/proprioception'

Remote-controlled Recording

The remote control interface can be used, for example, as follows

$ bag record -c 'spread:/control' -o /tmp/everything.tide spread:

Then, without terminating the above bag record process, the RPC Call program can be used to issue commands:

$ call 'spread:/control/terminate()'

Inspection

Inspecting a File

When invoked with just a file name, bag info will display some basic information about the stored data. For example

$ bag info 1314442144.tide

should produce output similar to the following:

File "1314442144.tide"
  Events  : 7,620
  Start   : 2011-08-27T12:49:17.356679+02:00
  End     : 2011-08-27T12:56:41.333822+02:00
  Duration: 443.977143d0
  Channel "/KuHa/test/:UTF-8-STRING"
    Type    : (RSB-EVENT UTF-8-STRING)
    Events  : 28
    Start   : 2011-08-27T12:49:41.037310+02:00
    End     : 2011-08-27T12:56:10.710000+02:00
    Duration: 389.67269d0
    Rate    : 0.0718551767125379d0
  Channel "/isr/hyp/:UTF-8-STRING"
    Type    : (RSB-EVENT UTF-8-STRING)
    Events  : 30
    Start   : 2011-08-27T12:49:53.493381+02:00
    End     : 2011-08-27T12:54:31.182873+02:00
    Duration: 277.689492d0
    Rate    : 0.1080343364235043d0
  Channel "/isr/param/:UTF-8-STRING"
    Type    : (RSB-EVENT UTF-8-STRING)
    Events  : 22
    Start   : 2011-08-27T12:49:41.398174+02:00
    End     : 2011-08-27T12:56:11.095187+02:00
    Duration: 389.697013d0
    Rate    : 0.05645411503320914d0
  Channel "/nao/mss/:UTF-8-STRING"
    Type    : (RSB-EVENT UTF-8-STRING)
    Events  : 33
    Start   : 2011-08-27T12:49:41.370000+02:00
    End     : 2011-08-27T12:56:11.096949+02:00
    Duration: 389.726949d0
    Rate    : 0.08467466795579487d0
  Channel "/nao/proprioception/:.rst.ProprioceptionData"
    Type    : (RSB-EVENT .rst.ProprioceptionData)
    Events  : 2,298
    Start   : 2011-08-27T12:49:17.356679+02:00
    End     : 2011-08-27T12:56:32.623617+02:00
    Duration: 435.266938d0
    Rate    : 5.279518840918719d0
  Channel "/runner/statechanges/:UTF-8-STRING"
    Type    : (RSB-EVENT UTF-8-STRING)
    Events  : 33
    Start   : 2011-08-27T12:49:23.262380+02:00
    End     : 2011-08-27T12:56:10.688253+02:00
    Duration: 407.425873d0
    Rate    : 0.08099632887084714d0
  Channel "/nao/vision/top/:.rst.ImageMessage"
    Type    : (RSB-EVENT .rst.ImageMessage)
    Events  : 926
    Start   : 2011-08-27T12:49:17.614445+02:00
    End     : 2011-08-27T12:56:31.452848+02:00
    Duration: 433.838403d0
    Rate    : 2.1344352957153956d0
  Channel "/audio/mic1:.rst.SoundMessage"
    Type    : (RSB-EVENT .rst.SoundMessage)
    Events  : 4,250
    Start   : 2011-08-27T12:49:17.356679+02:00
    End     : 2011-08-27T12:56:41.333822+02:00
    Duration: 443.977143d0
    Rate    : 9.572564865124148d0

Note

The file extension (“tide” in this case) determines the used file format backend. Currently, only the TIDELog file format is supported.

Cat

Printing the Contents of Log Files

When invoked with a log file, bag cat prints the contents of one or more channels to its standard output stream. This is probably most useful for channels containing textual data, since binary content usually requires a form of decoding.

Example

$ bag cat isr.tide | head
<sr db_start="80" db_utt="80" />
<sr db_start="45" db_utt="46" />
<?xml version="1.0" encoding="UTF-8" standalone="no" ?><speech_hyp [...]
<?xml version="1.0" encoding="UTF-8" standalone="no" ?><speech_hyp [...]
<?xml version="1.0" encoding="UTF-8" standalone="no" ?><speech_hyp [...]
<?xml version="1.0" encoding="UTF-8" standalone="no" ?><speech_hyp [...]
<?xml version="1.0" encoding="UTF-8" standalone="no" ?><speech_hyp [...]
<speech_hyp xmlns:bxml="http://www.sleepycat.com/2002/dbxml" [...]
<TIMESTAMP>1290275288296</TIMESTAMP>

Selecting Channels for Processing

The default behavior of bag cat is to print the contents of all channels found in the given file. However, when only a subset of the recorded channels should be printed, the --channel commandline option can be used to select channels. The following example prints the contents all channels matching one of the regular expressions hyp and param.

$ bag cat -c 'hyp' -c 'param' /tmp/nao.tide

Controlling the Output Format

Like the logger, bag cat supports different output formats. These are controlled using the --style option.

Example

$ bag cat --style 'programmable/template :template "${create} ${data}\\n"' mydata.tide
2011-12-13T17:03:25.533535+01:00 #(98 108 117 98)
2011-12-13T17:03:25.534054+01:00 #(98 108 117 98)
2011-12-13T17:03:25.534121+01:00 #(98 108 117 98)
[...]

Loading Data Type Definitions

For some log files, bag cat needs to know about additional data type definitions to be able to decode entries stored in the log file. In order to load these data type definitions, the common --idl-path and common --load-idl command-line options are used.

Example with $RST-PROTO pointing to the root directory of the rst-proto project:

$ bag cat -I$RST-PROTO/proto/stable/ -I$RST-PROTO/proto/sandbox/ -l$RST-PROTO/proto/\*\*/\*.proto data-file.tide

Replay

Playback of All Recorded Channels

To play all recorded events of a log file on their original channels, two commandline arguments have to be specified:

  1. The log file from which events should be read
  2. The “base” scope to which events should be published

In the following example, the specified scope is / which causes all events to be played on their original scopes since recorded event scopes are appended to the specified base scope

$ bag play /tmp/nao.tide 'spread://remote:4803'

For details regarding the URI syntax involved in transport and channel specifications, see URIs.

Selecting Channels for Playback

The default behavior of bag play is to play back all channels found in the given file. However, when only a subset of the recorded channels should be played back, the --channel commandline option can be used to select channels for playback. The following example plays all channels matching one of the regular expressions vision and audio.

$ bag play -c 'vision' -c 'audio' /tmp/nao.tide 'spread:?name=5555'

Selecting Temporal Intervals for Playback

The default behavior of bag play is to play back all events of the selected channels ordered by timestamp, from earliest to latest. However, it is possible to select a subset of these events by specifying temporal or index-based intervals. The --start-index, --end-index, --start-time and --end-time commandline options allow specifying these restrictions.

For example, to replay the events of a log file starting with the 1000-th event, the following command can be used.

$ bag play --start-index 1000 /tmp/nao.tide 'spread:?name=5555'

Controlling Replay Timing

Different strategies for controlling the replay timing can be used. Strategies and their parameters are specified using the --replay-strategy commandline option. The default replay strategy tries to mimic the original timing.

Other strategies include

  • Replay events as fast as possible
  • Replay events at a fixed rate

In addition, some strategies permit modification of their behavior via parameters. For example

-r 'fixed-rate :rate 20'
-r 'recorded-timing :speed 1.5'

Tip

Available strategies and their parameters are described in the output of bag play --help.

Interactive Playback

bag play can be controlled interactively using terminal-based interface. The interface is explained in the bag play manual page. An interactive sessions could look like this:

$ bag play -p none -r interactive test_trial_32.tide socket://localhost:5555
OHAI, type command; unambiguous prefix suffices. empty command repeats previous
one.
> length
12815        # File has 12815 events
> emit
             # Event 0 emitted; no reply
> next
1
> exmit
No such command: "EXMIT". Available commands: length, index, time, next,
  previous, seek, emit, emitandnext, quit.
> emit
             # Event 1 emitted; no reply
> emitandnext
2            # Event 1 emitted; cursor is now at 2
>            # Empty command repeats previous command
3            # Event 2 emitted; cursor is now at 3
>

Remote-controlled Playback

bag play can expose an RPC interface that allows navigating the log file being played back and controlling emission of events. An URI has to be supplied as the :uri argument of the --replay-strategy option to configure the scope and transport through which the RPC interface should be exposed. The full syntax of the commandline option therefore is

-r 'remote-controlled :uri "URI"'

The interface is explained in the bag play manual page.

A simple C++ program using the exposed interface could look like this:

#include <stdlib.h>
#include <iostream>
#include <rsb/Factory.h>
#include <rsb/patterns/RemoteServer.h>

int main(int argc, char** argv) {
  std::string scope(argv[1]);

  rsb::Factory& factory = rsb::Factory::getInstance();
  rsb::patterns::RemoteServerPtr bagplay = factory.createRemoteServer(scope);
  while (true) {
    std::cout << *bagplay->call<boost::uint64_t>("emitandnext") << std::endl;
    sleep(1);
  }

  return EXIT_SUCCESS;
}

Assuming this program has been compiled into an executable named remote-control and spread://localhost:4803/bagplay/control has been supplied as URI above, bag play could be controlled as follows:

$ ./remote-control /bagplay/control

Merging

Merging Multiple Log Files into One

Merge all log files matching the glob expression /vol/my-separate-logs/**/*.tide into a single log file named bla.tide.

$ bag merge -o bla.tide '/vol/my-separate-logs/**/*.tide'

Note

The quotes (') prevent the shell from interpreting the glob expression.