Dikablis Mobile Eye-Tracking System

From REALab Wiki
Jump to navigation Jump to search

The Ergoneers Dikablis eye tracking system includes lightweight glasses and specialized software which are optimal for determining the gaze of a subject even when they are in motion. The analysis software is configured to detect pre-defined markers which the experimenter places in the environment, and to determine when the subject's gaze falls upon an area of interest within marker-defined coordinates regardless of whether or not the subject is moving.

We have 2 Dikablis "Essential" mobile eye tracking system, field camera resolution 768 x 576 px, tracking resolution 50 Hz. More specs can be found here: Dikablis_essential.pdf

Assembly

To start using the Dikablis, make sure the hardware is assembled as shown. Start up the laptop, and turn on the cable box (gently pull and switch the toggle). Make sure the recording license stick is plugged in, and open the recording software on the desktop.

Dikablis Assembly.png

Setting Up Markers

For tips on how to set up the markers in your testing environment, refer to the marker setup guide.

Note that with the D-Lab 2.5 software, you can actually use high-contrast areas in the field video as a substitute for the markers we usually would use. This is useful for specifying AOIs outside the lab setting or in a public place, where we are unable to set up our markers ahead of time.

Dikablis Software

The following programs are on the desktop of the Dell laptop: Dikablis Recorder 2.0, Control Center 1.0, Eye Controlled Interaction 2.5, Dikablis Analysis 2.0, Marker Detector 2.1, D-Lab Essential 2.0, and Dikablis Player 2.0. These are briefly described below, roughly in the order in which they would be run to test and then analyze one subject's data.

Quick summary (click the links for more detail):

  • Recorder- program to calibrate the eye tracker and manually record eye cam and field cam video and log file
  • Control Center- can send network communications and allows for e.g., matlab to automate creating a new subject file, start and stop of recording as well as sending event info
  • Eye Controlled Interaction- real-time marker and AOI detection
  • Dikablis Analysis- post-recording but pre-analysis processing, including re-calibration
  • Marker detector- run before using D-Lab Essential for frame by frame marker detection in field cam video which allows for AOI analyses
  • D-Lab Essential- Area of Interest analyses
  • Player- playback of eyecam and fieldcam videos with eye gaze overlay


Although these programs can be installed on any computer using the Dikablis installation CD, the Recorder should only be used with the Dikablis Dell laptops as they have been optimally configured for recording.

Note that when you save changes to data in the online folder of a subject, they are automatically saved to different files in that subject's offline folder.

Using Recorder

Main Page: Dikablis Recorder

This program is what you will be using to record video from the dikablis. Once the dikablis is properly setup and the participant is wearing the dikablis unit, calibration can be completed using the calibration wizard. After a new project and subject are opened/created, you are ready to start recording.


NOTE: If you are having problems with the calibration wizard in Dikablis Recorder, there are more tips and instructions in this Calibration guide.

Eye Controlled Interaction

Main Page: Eye Controlled Interaction

This software can be found on the newer 14" Dikablis Dell laptop. It allows you to calculate in real time where someone is looking in marker-based coordinates. You can even define AOI based on marker coordinates and determine whether someone is looking at that AOI or not in real time. To do this, you first have to virtualize the environment/room by telling the software where the AOIs are. Note that the markers must be completely in view of the field camera, e.g,. no cables interposed. You can then stream out real time marker-based coordinates and AOI data via tcp/ip. For example, you can define an AOI called screen referring to a computer screen, then the tcp/ip output for that AOI will output "Area screen value: 1" when the subject is looking at the screen, and "Area screen value: 0" when not looking at it.

In order to set up real-time marker detection and AOI marking:

1) Set up the environment in Eye Controlled Interaction software

2) Add the following code in matlab:


a) This code is to set up the UDP object over ethernet:

u = udp('127.0.0.1',65000,'LocalPort',65000, 'InputBufferSize', 409600); %buffer will fill with ~4000 bytes per second, adjust 'InputBufferSize' as needed.
u.Terminator = 'LF/CR';  %tell it about line endings
fopen(u);


b) this code will read in one line from the Eye Controlled Interaction software(~4000 bytes of information):

A = fscanf(u);
B =regexp(A, '\t', 'split');

Recording and Setting Events

Remote Recording and Event Setting with Control Center (for D-Lab Analysis)

If you will be using the D-Lab analysis software to create AOIs and output mean gaze statistics in a table, and you have different within-subject conditions, you can use matlab to control Control Center on one computer, which in turn will control recording and event labelling on the dikablis laptop running the Recorder software.

If you have a matlab script, in order to mark in the Dikablis recorder video the time interval in which a given condition is being displayed, follow these steps:


1) Set up your conditions in D-Lab and save that file for importing into Control Center (step 4)


2) Add code to your matlab script that will: (the following code was adapted from pgs 97-99 of the D-Lab manual)

a) create a TCP/IP connection,

c = tcpip('127.0.0.1',2008);%tcpip(IP,port) IP = ip address of computer running Control Centre(127.0.0.1 for local), port = 2008(default) 
c.Terminator = 'CR/LF';  %tell it about line endings
fopen(c)    %open the channel

b) create a new subject,

fprintf(d,['experiment ', subject, '\n']);pause(1) %subject variable contains Subject's name


c) start recording,

fprintf(d, 'start recording\n');pause(1); %send once to start recording

d) Mark The Current Event, (note: the event name should be the same as the alternative trigger of that condition as you specified in D-Lab)

fprintf(d, ['trigger1', '\n']);pause(1); %send once to mark the start of the condition, once more to mark the end of said condition 
%%(if it was set as an "Interval" condition in D-Lab)

e) stop recording

fprintf(d, 'stop recording\n');pause(1);%send once to stop recording


3) Open the project folder in the Recorder program (do not create new subject if you do that remotely in matlab, i.e., in 2b above)

Recorded project.png

4) Open Control Center and import the D-lab trial structure file

5) Run the matlab script

Make sure the Recorder indicates that there is a network connection, the new subject is created, and the recording is started and later stopped.


Now, the recorded videos and project structure can be imported into D-Lab, and quickly analyzed - refer to section xyz below on D-Lab analysis.

Note that the Recorder software must be running in order to communicate over ethernet cable with the Dikablis computer AND that the two computers must have LAN IP addresses with identical numbers in the first 3 parts: e.g., 150.100.100.10 and 150.100.100.21


Demo Experiment

Remote Recording without Event Setting (for journal.txt event output)

The following will send event info to the journal.txt file, which is not (as far as I can tell) easily imported into the d-lab software for analysis. If you have different conditions and would like to use the d-lab software for output of mean gaze statistics, it is best to do remote recording and event setting with control center. (these and more commands can be found on page 74 of the software manual):

  • To log in and start communicating with the Dikablis:
%note that c is just a variable name; port 2001 and "user,pass" are the Dikablis defaults
c=tcpip('150.100.100.20',2001); %replace 150.100.100.20 with LAN address of Dikablis computer if different
fopen(c);
fprintf(c, 'Luser,pass');pause(1);
  • To open a project (that you've already created):

e.g., to open the project "petertest"

fprintf(c,'Opetertest');pause(1);
  • To create and open a new subject in the project that is currently open:

e.g., to create the subject "s003"

fprintf(c,'CEs003');pause(1)
  • To start recording:
fprintf(c, 'Rstart');pause(1);
  • To mark an event:

e.g., "FixOn"

fprintf(c, 'EFixOn');pause(1);

or, e.g., "Stim1"

fprintf(c, 'EStim1');pause(1);
  • To stop recording:
fprintf(c, 'Rstop');pause(1);
  • To close communication with the Dikablis:
fclose(c);

Q: With our current software, can we have triggers, eyedata, and marker-defined coordinate info all in the same file?

A: This info has to be in 2 files because the marker detector has to be run offline. You have to use post-hoc scripting to combine. Alternative is to use the Eye Control software.

Note that the Recorder software must be running in order to communicate over ethernet cable with the Dikablis computer AND that the two computers must have LAN IP addresses with identical numbers in the first 3 parts: e.g., 150.100.100.10 and 150.100.100.21


Demo Experiment

For an example of how these commands may be used in a matlab script, take a look at the following demo script (please email Peter if you need the user name and password):

This very simple script will display text in place of stimuli ("stim1"), and can be adjusted or integrated into a full matlab experiment.

Other ways of Setting Events

Here is a list of additional ways to mark events in your experiment.

Note: only D-Lab friendly ways of event marking can be used for analysis in D-Lab. Other methods mark events in the 'journal.txt' file and can only be used for analysis in other programs

Marker Detection

When analyzing video with D-Lab, the video must first go through Marker Detection for D-Lab to recognize the markers placed in the video. This is done by opening the Marker Detector program and following the instructions found here.

Analysis with D-Lab

To analyse video with D-Lab, the video must by marked with events(use cases) either during video capture or after the fact in D-Lab itself.

Analysis in D-Lab is outlined in detail on the D-Lab wiki page.

PDF Manuals