View source for Optitrak + Projector touch table/surface

[[File:Touch table 2.png|640x310px]]   [[File:Opti touch table.JPG|319x240px]]   


==Calibration with Tracking Tools==

''note: if you are using the laptop setup in room 3108, make sure that the dongle for Tracking Tools is not plugged in when starting up the computer.''

To calibrate the optitrak for use with the touch table, make sure that the Tracking tools dongle is plugged in.  Then start Tracking Tools from the desktop.

In general, you will follow these steps in order to calibrate:

# Use the wand to register the area of interest
# Use the plane to set the ground plane
# If using trackables, create trackable objects
# Set the data to stream out (for reading into matlab)


===Step-by-step calibration guide===


====1. Open Tracking Tools====
Open Tracking Tools and select "Perform Camera Calibration".

If you are sure the setup has not changed, you can load a previous calibration that you have saved.  To do so, select "Open Camera Calibration", and select a recent file.  Then Proceed to [[#6. Set Ground Plane|Step 6]].


<Center>[[File:Start trackingtools.png|500x371px]]</Center>

====2. Change Illumination====
Change Illumination Type from "Strobe IR" to "Continuous".


<Center>[[File:Continuousillum trackingtools.png|600x445px]]</Center>

====3. Set Blocking Masks (if necessary)====
Clear the table of the reflective equipment (e.g., finger marker,  wand, ground plane) and make sure that there are no bright lights registered in the software.  If there are, find their source on or around the table and remove them so that the cameras images are all dark.  Once this is the case move on to [[#4. Wanding|Step 4]].


As a last resort if there are any lights visible in the cameras' videos that can't be removed, you can block out that area in the corresponding camera with a mask.  To do so, click the "Block Visible Markers" button, and it will automatically mask any markers currently registered by the cameras (as indicated by a red mark corresponding to the blocked area in the camera's field).  You can use the "Clear Blocking" button to remove masks if you need to adjust or start again.  Once all lights are masked, move on to [[#4. Wanding|Step 4]].


<Center>[[File:Blockingmasks1 trackingtools.png]]
[[File:Blockingmasks2 trackingtools.png]]</Center>

====4. Wanding====
'''''*Important: make sure the wand size selected corresponds to that indicated on the actual wand before proceeding.'''''  Click on "Start Wanding" [(1) in the picture below] and sweep the wand over the table's surface.  Your goal is to a) wand the 3D area you want to track in, e.g., most likely the area close to the table's surface; and b) "paint" all the regions as full of wand data as possible. e.g., in the picture below, camera 4 is more painted than camera 5.

When the indicated quality becomes "Very High" (2), you can click on "Calculate" (3).


<Center>[[File:Wandingcalibration trackingtools2.png|479x480px]]</Center>

====5. Apply Wanding Results====
The tracking tools software will now calculate the quality of your calibration.  If you wanded well, you should ultimately not see the distorted square grid shown in Cameras 2 and 3 below.  Bear in mind that the grid will become less distorted as the results of the calibration are being calculated, however if the distortion in the grid persists when the cameras read "Exceptional", you should calibrate again by clicking the "Reset" button at the top of the 3-Marker Calibration panel; then go back to [[#4. Wanding|Step 4]].  When all cameras achieve "Exceptional" results, and the overall result is "Exceptional", and there is no distortion in the grids for any of the cameras, click on "Apply Result" (see picture below).

Click on "Apply" again when the dialogue pops up, and save the results of your calibration (by default the file name will include the date and time- you can load the same calibration later in the same session if you are sure that the setup has not changed).


<Center>[[File:Applycal trackingtools.png|465x480px]]</Center>

====6. Set Ground Plane====
Set the ground plane on the lower left corner of the table top so that Z arrow points towards the top of the projected image.  Make sure that the markers of the ground plane are parallel with the left edge of the projected screen image and at the same time parallel with the bottom edge of the projected screen image.  Click on "Set Ground Plane", and save the results.


<Center>[[File:Setgroundplane1 trackingtools.JPG|319x240px]][[File:Setgroundplane2 trackingtools.png|500x352px]]</Center>

====7. Stream Out Data====
Now stream out the data so that it can be read into Matlab.  To do so, click on the "Streaming Pane" icon.  Put a check in the "Broadcast Frame Data" box, and change Type from "Multicast" to "Unicast".  You can now minimize Tracking Tools and it will output data continuously in the background.


<Center>[[File:Streamingicon trackingtools.png]]</Center>

<Center>[[File:Broadcastunicast trackingtools.png]]</Center>

== Matlab scripts ==

===Function: Setting the optitrack plane in matlab ===
In place of the matlab code for reading in optitrak data, the following matlab function can be used to:

# read in data sent from the optitrak system, and subsequently
# set the table top plane, as understood by the Optitrak, into variables for reference in matlab, and 
# transform the native coordinate system of the optitrak (x z y) into another, possibly more familiar coordinate system (x y z).


<!-- *[http://weblab1.psych.ubc.ca/wikiakje/uploads/akwikishr/opti_setup_touch_plane.m opti_setup_touch_plane.m] -->
*[http://real.psych.ubc.ca/images/b/bf/Opti_setup_touch_plane.m opti_setup_touch_plane.m]


This function should be called before opening a main window with psychtoolbox.  You can call the function with the following bit of code:

<source lang="matlab">
[touch_plane_info, client] = opti_setup_touch_plane;
optiData = client.GetLastFrameOfData;
distToTouch = -5/1000; %marker above this height will not record as touch (in mm)

%then call the window
[w, wRect]=Screen('OpenWindow',screenNumber, white);
mX=wRect(3); %max X of screen
mY=wRect(4); %max Y of screen
hz = frameRate(w);
</source>

This function will call on the experimenter to place the marker on the 4 corners of the projected screen, and to press OK on a dialogue box.  Pay attention to the coordinates returned in the dialogue boxes- if they are exactly [0 0 0], then you likely forgot to [[#7. Stream Out Data|stream out the data]] from Tracking Tools- be sure that the appropriate box is actually checked off.

Note also that the function initializes a client which will continuously update itself without the need for any additional matlab code.  However, this does mean that the client should be uninitialized at the end of your script when you are done with the optitrak.  To uninitialize the client, use this code:

<source lang="matlab">
client.Uninitialize;
</source>

'''''If you don't uninitialize the client, Matlab is prone to force close.'''''

====Setting or loading the optitrack plane====

If you don't need to set up the plane everytime and would rather have the option to either set the plane or load the last used plane, use the following code instead of the above:

<source lang="matlab">
ButtonName = questdlg('Perform calibraion?', ...
                     'New Calibration or load', ...
                     'New Calibration', 'Load Calibration', 'Quit', 'New Calibration');
switch ButtonName,
 case 'New Calibration',
    [touch_plane_info, client] = opti_setup_touch_plane;    
 case 'Load Calibration',
    [touch_plane_info, client] = use_last_touch_plane;
 case 'Quit',
    client.Uninitialize;
    screen('closeAll');
    return;      
end % switch

optiData = client.GetLastFrameOfData;
distToTouch = -5/1000; %marker above this height will not record as touch (in mm)
</source>

When using this for the first time, select New Calibration.  This will save a matlab data file called lastTouchInfo.mat into the same directory as the script, which contains the coordinates of the touch plane.  After this file is created, you can choose to use those same coordinates by selecting Load Calibration.

Here are the scripts needed for this code:

<!-- *[http://weblab1.psych.ubc.ca/wikiakje/uploads/akwikishr/opti_setup_touch_plane2.m opti_setup_touch_plane2.m] -->
*[http://real.psych.ubc.ca/images/a/a8/Opti_setup_touch_plane2.m opti_setup_touch_plane2.m]
(change line 215 to the appropriate directory for the computer you are working on.)
<!-- *[http://weblab1.psych.ubc.ca/wikiakje/uploads/akwikishr/use_last_touch_plane.m use_last_touch_plane.m] -->
*[http://real.psych.ubc.ca/images/6/69/Use_last_touch_plane.m use_last_touch_plane.m]

===Register a touch on the table ===

This bit of code returns the variables touchFlag, x, and y.  When the marker is very close to or touching the surface defined by the [[#6. Set Ground Plane|ground plane]] in calibration, touchFlag is returned with a value of 1, and you can use the x and y values to mark where the touch was registered on the plane. 


<!-- *[http://weblab1.psych.ubc.ca/wikiakje/uploads/akwikishr/detectTouch_opti.m detectTouch_opti.m] -->
*[http://real.psych.ubc.ca/images/6/6f/DetectTouch_opti.m detectTouch_opti.m]

====Registering a touch anywhere on the plane====
Insert this code into your script, and the script will advance to the next section of code when a touch is registered anywhere on the table, otherwise it will continuously loop:

<source lang="matlab">
touchFlag = 0;
while ~touchFlag
    [touchFlag,x,y] = detectTouch_opti(touch_plane_info,distToTouch,optiData);

    %[touch,secs,keyCode] = KbCheck;

    %if find(keyCode) == 27 %ESCAPE
    %    client.Uninitialize;
    %    Screen('CloseAll');
    %    return;
    %end
    waitsecs(1/hz);
end		
</source>

''(The commented code above is not necessary, but useful if you want to escape out of the program with the "Esc" key of the keyboard.)''

====Registering a touch only on certain areas of the plane====
If you only want to register touches on certain areas of the table, you can adapt the following bit of code.  This code will continuously loop until one of 4 rectangular areas register a touch with the detectTouch_opti.m function.  It will also store into variables how long it took to lift off or slide away from the square-shaped starting position ''(RxnTime)'', and how long it took to touch down on one of the 4 rectangular response areas ''(RspTime)'', as well as which of the 4 areas was touched ''(responsebin)''.

<source lang="matlab">
positions={[12.47 mY*58/100-96 262.47 mY*58/100+96], ...
[mX*38/100-125 mY*36/100-96 mX*38/100+125 mY*36/100+96], ...
[mX*62/100-125 mY*36/100-96 mX*62/100+125 mY*36/100+96], ...
[761.53 mY*58/100-96 1011.53 mY*58/100+96]};
%above are the positions of the compost, garbage, paper, and recycling bins, which are referenced below; 
%mX and mY are max x and y coordinates of the screen

%...

binResponseFlag=0;
initiateFlag=0;
tic;
while ~binResponseFlag
	[touchFlag,x,y] = detectTouch_opti(touch_plane_info,distToTouch,optiData);
	if (touchFlag==0 | y < mY*9/10-50 | y > mY*9/10+50 | x < mX/2-50 | x > mX/2+50) & initiateFlag==0
		RxnTime=toc;
		initiateFlag=1;
	end
	if touchFlag==1 & x > positions{1}(1) & y > positions{1}(2) & x < positions{1}(3) & y < positions{1}(4)
		RspTime=toc;
		responsebin='Compost';
		binResponseFlag=1;
	elseif touchFlag==1 & x > positions{2}(1) & y > positions{2}(2) & x < positions{2}(3) & y < positions{2}(4)
		RspTime=toc;
		responsebin='Garbage';
		binResponseFlag=1;
	elseif touchFlag==1 & x > positions{3}(1) & y > positions{3}(2) & x < positions{3}(3) & y < positions{3}(4)
		RspTime=toc;
		responsebin='Paper';
		binResponseFlag=1;            
	elseif touchFlag==1 & x > positions{4}(1) & y > positions{4}(2) & x < positions{4}(3) & y < positions{4}(4)
		RspTime=toc;
		responsebin='Recycling';
		binResponseFlag=1;  
	end
	waitsecs(1/hz);	
end
</source>

===Putting it all together: Example Script===
The following program displays recycling signs (garbage, compost, recycling, paper) which vary in design, and the subject's task is to reach from the home position and touch the appropriate sign for a variety of waste items (e.g., pop can, apple core).  The main script is called "signage reycling.m".  [http://real.psych.ubc.ca/images/d/da/Signage_recycling_opti.zip Signage_recycling_opti.zip]