.. _tutorial:

==========
 Tutorial
==========

.. toctree::
   :maxdepth: 1

   tutorial-client-api

.. _recording:

Recording
=========

Dealing with Errors
-------------------

.. seealso::

   :ref:`error-options`
     Commandline options for controlling behavior in case of (serious)
     errors.

When recording data, an important decision to make is how to deal
with errors. In |project|, there are two high-level possibilities:

Abort the entire recording as soon as the first error is detected

  This approach has the advantage that error will not go unnoticed. On
  the downside, data may be lost unnecessarily when a recording is
  aborted due to a minor error.

Try to continue, ignoring all errors

  This obvious disadvantage of this approach is that errors may go
  unnoticed during recording and lead to unpleasant surprises
  later. The potential advantage consists in the fact that recording
  may continue and produce useful result despite some errors. This
  approach may make sense when recording data which is hard/expensive
  to produce.

These strategies are chosen via the :option:`on-error` commandline
options.

Recording using a local Spread daemon
-------------------------------------

This example assumes

* A Spread daemon running on the local machine and listening on port
  4803
* Image, Sound and Proprioception message are published (for example
  by running the `streaming components for Nao
  <https://code.humavips.eu/projects/naorsb/wiki/Components)>`_ on the
  :ref:`RSB <rsb>` :term:`channels <channel>`

  * ``/nao/vision/top``
  * ``/nao/audio/all``
  * ``/nao/proprioception``
* Image, Sound and Proprioception message are encoded using the `RST
  <https://code.cor-lab.org/projects/rst>`_ data types

  * ``rst.vision.Image``
  * ``rst.audition.SoundChunk``
  * ``rst.kinematics.ProprioceptionState``

.. code-block:: sh

   $ bag record -o /tmp/nao.tide
                'spread:/nao/vision/top?name=4803'
                'spread:/nao/audio/all?name=4803'
                'spread:/nao/proprioception?name=4803'

The resulting file will (unless a different :term:`channel` allocation
is specified) contain one :term:`channel` for each of the three RSB
:term:`channels <channel>`.

.. Note::

   The file extension ("tide" in this case) determines the used file
   format backend. Currently, only the `TIDELog
   <https://retf.info/svn/drafts/rd-0001.txt>`_ file format is
   supported.

Recording using a remote Spread daemon
--------------------------------------

This example assumes

* A Spread daemon running on the machine "remote" and listening on
  port 4803
* Same data types as above

.. code-block:: sh

   $ ./bag record -o /tmp/nao.tide
                  'spread://remote:4803/nao/vision/top'
                  'spread://remote:4803/nao/audio/all'
                  'spread://remote:4803/nao/proprioception'

Remote-controlled Recording
---------------------------

The remote control interface can be used, for example, as follows

.. code-block:: sh

   $ bag record -c 'spread:/control' -o /tmp/everything.tide spread:

Then, without terminating the above :program:`bag record` process, the
:ref:`call` program can be used to issue commands:

.. code-block:: sh

   $ call 'spread:/control/terminate()'

.. _inspection:

Inspection
==========

Inspecting a File
-----------------

When invoked with just a file name, :program:`bag info` will display
some basic information about the stored data. For example

.. code-block:: sh

  $ bag info 1314442144.tide

should produce output similar to the following::

  File "1314442144.tide"
    Events  : 7,620
    Start   : 2011-08-27T12:49:17.356679+02:00
    End     : 2011-08-27T12:56:41.333822+02:00
    Duration: 443.977143d0
    Channel "/KuHa/test/:UTF-8-STRING"
      Type    : (RSB-EVENT UTF-8-STRING)
      Events  : 28
      Start   : 2011-08-27T12:49:41.037310+02:00
      End     : 2011-08-27T12:56:10.710000+02:00
      Duration: 389.67269d0
      Rate    : 0.0718551767125379d0
    Channel "/isr/hyp/:UTF-8-STRING"
      Type    : (RSB-EVENT UTF-8-STRING)
      Events  : 30
      Start   : 2011-08-27T12:49:53.493381+02:00
      End     : 2011-08-27T12:54:31.182873+02:00
      Duration: 277.689492d0
      Rate    : 0.1080343364235043d0
    Channel "/isr/param/:UTF-8-STRING"
      Type    : (RSB-EVENT UTF-8-STRING)
      Events  : 22
      Start   : 2011-08-27T12:49:41.398174+02:00
      End     : 2011-08-27T12:56:11.095187+02:00
      Duration: 389.697013d0
      Rate    : 0.05645411503320914d0
    Channel "/nao/mss/:UTF-8-STRING"
      Type    : (RSB-EVENT UTF-8-STRING)
      Events  : 33
      Start   : 2011-08-27T12:49:41.370000+02:00
      End     : 2011-08-27T12:56:11.096949+02:00
      Duration: 389.726949d0
      Rate    : 0.08467466795579487d0
    Channel "/nao/proprioception/:.rst.ProprioceptionData"
      Type    : (RSB-EVENT .rst.ProprioceptionData)
      Events  : 2,298
      Start   : 2011-08-27T12:49:17.356679+02:00
      End     : 2011-08-27T12:56:32.623617+02:00
      Duration: 435.266938d0
      Rate    : 5.279518840918719d0
    Channel "/runner/statechanges/:UTF-8-STRING"
      Type    : (RSB-EVENT UTF-8-STRING)
      Events  : 33
      Start   : 2011-08-27T12:49:23.262380+02:00
      End     : 2011-08-27T12:56:10.688253+02:00
      Duration: 407.425873d0
      Rate    : 0.08099632887084714d0
    Channel "/nao/vision/top/:.rst.ImageMessage"
      Type    : (RSB-EVENT .rst.ImageMessage)
      Events  : 926
      Start   : 2011-08-27T12:49:17.614445+02:00
      End     : 2011-08-27T12:56:31.452848+02:00
      Duration: 433.838403d0
      Rate    : 2.1344352957153956d0
    Channel "/audio/mic1:.rst.SoundMessage"
      Type    : (RSB-EVENT .rst.SoundMessage)
      Events  : 4,250
      Start   : 2011-08-27T12:49:17.356679+02:00
      End     : 2011-08-27T12:56:41.333822+02:00
      Duration: 443.977143d0
      Rate    : 9.572564865124148d0

.. Note::

   The file extension ("tide" in this case) determines the used file
   format backend. Currently, only the `TIDELog
   <https://retf.info/svn/drafts/rd-0001.txt>`_ file format is
   supported.

.. _cat:

Cat
===

Printing the Contents of Log Files
----------------------------------

When invoked with a :term:`log file`, :program:`bag cat` prints the
contents of one or more :term:`channels <channel>` to its standard
output stream. This is probably most useful for :term:`channels
<channel>` containing textual data, since binary content usually
requires a form of decoding.

Example

.. code-block:: sh

   $ bag cat isr.tide | head
   <sr db_start="80" db_utt="80" />
   <sr db_start="45" db_utt="46" />
   <?xml version="1.0" encoding="UTF-8" standalone="no" ?><speech_hyp [...]
   <?xml version="1.0" encoding="UTF-8" standalone="no" ?><speech_hyp [...]
   <?xml version="1.0" encoding="UTF-8" standalone="no" ?><speech_hyp [...]
   <?xml version="1.0" encoding="UTF-8" standalone="no" ?><speech_hyp [...]
   <?xml version="1.0" encoding="UTF-8" standalone="no" ?><speech_hyp [...]
   <speech_hyp xmlns:bxml="http://www.sleepycat.com/2002/dbxml" [...]
   <TIMESTAMP>1290275288296</TIMESTAMP>

Selecting Channels for Processing
---------------------------------

The default behavior of :program:`bag cat` is to print the contents of
all :term:`channels <channel>` found in the given file. However, when
only a subset of the recorded :term:`channels <channel>` should be
printed, the :option:`--channel` commandline option can be used to
select :term:`channels <channel>`. The following example prints the
contents all :term:`channels <channel>` matching one of the regular
expressions :regexp:`hyp` and :regexp:`param`.

.. code-block:: sh

   $ bag cat -c 'hyp' -c 'param' /tmp/nao.tide

Controlling the Output Format
-----------------------------

Like the :ref:`logger <logger>`, :program:`bag cat` supports different
output formats. These are controlled using the :option:`--style`
option.

Example

.. code-block:: sh

   $ bag cat --style 'programmable/template :template "${create} ${data}\\n"' mydata.tide
   2011-12-13T17:03:25.533535+01:00 #(98 108 117 98)
   2011-12-13T17:03:25.534054+01:00 #(98 108 117 98)
   2011-12-13T17:03:25.534121+01:00 #(98 108 117 98)
   [...]

Loading Data Type Definitions
-----------------------------

For some :term:`log files <log file>`, :program:`bag cat` needs to
know about additional data type definitions to be able to decode
entries stored in the :term:`log file`. In order to load these data
type definitions, the :option:`common --idl-path` and :option:`common
--load-idl` command-line options are used.

Example with ``$RST-PROTO`` pointing to the root directory of the
rst-proto project:

.. code-block:: sh

   $ bag cat -I$RST-PROTO/proto/stable/ -I$RST-PROTO/proto/sandbox/ -l$RST-PROTO/proto/\*\*/\*.proto data-file.tide

.. _replay:

Replay
======

Playback of All Recorded Channels
---------------------------------

To play all recorded :term:`events <event>` of a :term:`log file` on
their original :term:`channels <channel>`, two commandline arguments
have to be specified:

1. The :term:`log file` from which :term:`events <event>` should be
   read
2. The "base" :term:`scope` to which :term:`events <event>` should be
   published

In the following example, the specified :term:`scope` is ``/`` which
causes all :term:`events <event>` to be played on their original
:term:`scopes <scope>` since recorded :term:`event` :term:`scopes
<scope>` are appended to the specified base :term:`scope`

.. code-block:: sh

   $ bag play /tmp/nao.tide 'spread://remote:4803'

For details regarding the URI syntax involved in :term:`transport` and
:term:`channel` specifications, see :ref:`uri-schema`.

Selecting Channels for Playback
-------------------------------

The default behavior of :program:`bag play` is to play back all
:term:`channels <channel>` found in the given file. However, when only
a subset of the recorded :term:`channels <channel>` should be played
back, the :option:`--channel` commandline option can be used to select
:term:`channels <channel>` for playback. The following example plays
all :term:`channels <channel>` matching one of the regular expressions
:regexp:`vision` and :regexp:`audio`.

.. code-block:: sh

   $ bag play -c 'vision' -c 'audio' /tmp/nao.tide 'spread:?name=5555'

Selecting Temporal Intervals for Playback
-----------------------------------------

The default behavior of :program:`bag play` is to play back all
:term:`events <event>` of the selected :term:`channels <channel>`
ordered by timestamp, from earliest to latest. However, it is possible
to select a subset of these :term:`events <event>` by specifying
temporal or index-based intervals.  The :option:`--start-index`,
:option:`--end-index`, :option:`--start-time` and :option:`--end-time`
commandline options allow specifying these restrictions.

For example, to replay the :term:`events <event>` of a :term:`log
file` starting with the 1000-th :term:`event`, the following command
can be used.

.. code-block:: sh

   $ bag play --start-index 1000 /tmp/nao.tide 'spread:?name=5555'

Controlling Replay Timing
-------------------------

Different strategies for controlling the replay timing can be
used. Strategies and their parameters are specified using the
:option:`--replay-strategy` commandline option. The default replay
strategy tries to mimic the original timing.

Other strategies include

* Replay :term:`events <event>` as fast as possible
* Replay :term:`events <event>` at a fixed rate

In addition, some strategies permit modification of their behavior via
parameters. For example

.. code-block:: sh

   -r 'fixed-rate :rate 20'
   -r 'recorded-timing :speed 1.5'

.. tip::

   Available strategies and their parameters are described in the
   output of ``bag play --help``.

Interactive Playback
--------------------

:program:`bag play` can be controlled interactively using
terminal-based interface. The interface is explained in the
:ref:`bag play manual page <interactive-playback>`. An interactive
sessions could look like this:

.. code-block:: sh

   $ bag play -p none -r interactive test_trial_32.tide socket://localhost:5555
   OHAI, type command; unambiguous prefix suffices. empty command repeats previous
   one.
   > length
   12815        # File has 12815 events
   > emit
                # Event 0 emitted; no reply
   > next
   1
   > exmit
   No such command: "EXMIT". Available commands: length, index, time, next,
     previous, seek, emit, emitandnext, quit.
   > emit
                # Event 1 emitted; no reply
   > emitandnext
   2            # Event 1 emitted; cursor is now at 2
   >            # Empty command repeats previous command
   3            # Event 2 emitted; cursor is now at 3
   >

Remote-controlled Playback
--------------------------

:program:`bag play` can expose an `RPC
<http://en.wikipedia.org/wiki/Remote_procedure_call>`_ interface that
allows navigating the :term:`log file` being played back and
controlling emission of :term:`events <event>`. An URI has to be
supplied as the ``:uri`` argument of the :option:`--replay-strategy`
option to configure the :term:`scope` and :term:`transport` through
which the RPC interface should be exposed. The full syntax of the
commandline option therefore is

.. code-block:: sh

   -r 'remote-controlled :uri "URI"'

The interface is explained in the :ref:`bag play manual page
<remote-controlled-playback>`.

A simple C++ program using the exposed interface could look like this:

.. code-block:: c++

   #include <stdlib.h>
   #include <iostream>
   #include <rsb/Factory.h>
   #include <rsb/patterns/RemoteServer.h>

   int main(int argc, char** argv) {
     std::string scope(argv[1]);

     rsb::Factory& factory = rsb::Factory::getInstance();
     rsb::patterns::RemoteServerPtr bagplay = factory.createRemoteServer(scope);
     while (true) {
       std::cout << *bagplay->call<boost::uint64_t>("emitandnext") << std::endl;
       sleep(1);
     }

     return EXIT_SUCCESS;
   }

Assuming this program has been compiled into an executable named
``remote-control`` and ``spread://localhost:4803/bagplay/control`` has
been supplied as :samp:`{URI}` above, :program:`bag play` could be
controlled as follows:

.. code-block:: sh

   $ ./remote-control /bagplay/control

.. _merging:

Merging
=======

Merging Multiple Log Files into One
-----------------------------------

Merge all :term:`log files <log file>` matching the glob expression
:file:`/vol/my-separate-logs/**/*.tide` into a single :term:`log file`
named :file:`bla.tide`.

.. code-block:: sh

   $ bag merge -o bla.tide '/vol/my-separate-logs/**/*.tide'

.. note::

   The quotes (``'``) prevent the shell from interpreting the glob
   expression.