Gesture recognition library for Corona SDK

Posted by gtatarkin, Posted on March 19, 2011

8 votes

Simple gesture recognition library for Corona SDK using Point Reduction Code and Levenshtein distance algorithm ( http://en.wikipedia.org/wiki/Levenshtein_distance ). Gesture pattern:

http://twitpic.com/4a4otc

Circle / O, square, triangle, swype up/down/left/right also included.

Usage sample:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
local Gesture = require("lib_gesture")
 
local pointsTable = {}
local line
 
local myText = display.newText("Result: ", 50, 50, native.systemFont, 32)
myText:setTextColor(255, 255, 255)
 
local function drawLine ()
 
        if line then 
                line:removeSelf() 
        end
        
        local numPoints = #pointsTable
        local nl = {}
        local  j, p
                 
        nl[1] = pointsTable[1]
                 
        j = 2
        p = 1
                 
        for  i = 2, numPoints, 1  do
                nl[j] = pointsTable[i]
                j = j+1
                p = i 
        end
        
        if ( p  < numPoints -1 ) then
                nl[j] = pointsTable[numPoints-1]
        end
        
        if #nl > 2 then
                        line = display.newLine(nl[1].x,nl[1].y,nl[2].x,nl[2].y)
                        for i = 3, #nl, 1 do 
                                line:append( nl[i].x,nl[i].y);
                        end
                        line:setColor(255,255,0)
                        line.width=5
        end
end
 
local function Update(event)            
        if "began" == event.phase then
                pointsTable = nil
                pointsTable = {}
                local pt = {}
                pt.x = event.x
                pt.y = event.y
                table.insert(pointsTable,pt)
        
        elseif "moved" == event.phase then
        
                local pt = {}
                pt.x = event.x
                pt.y = event.y
                table.insert(pointsTable,pt)
        
        elseif "ended" == event.phase or "cancelled" == event.phase then
                        drawLine ()
                        myText.text = Gesture.GestureResult()
        end
end
 
Runtime:addEventListener( "touch"               , Update )

Download library: http://www.concat.pl/corona.html


Replies

hima
User offline. Last seen 1 year 1 week ago. Offline
Joined: 8 Apr 2010

Awesome! Thank you so much for this :)

abwing
User offline. Last seen 15 weeks 5 days ago. Offline
Joined: 5 Dec 2010

THANKS SO MUCH!!!

henkbl
User offline. Last seen 22 weeks 1 day ago. Offline
Joined: 26 Oct 2010

Nice work! Any chance this can work to see if someone is drawing a letter correctly. Such as the letter A. Thinking a small child who may do it a number of different ways including lifting their finger multiple times while drawing the letter.

Thanks

gtatarkin's picture
gtatarkin
User offline. Last seen 2 hours 45 min ago. Offline
Joined: 16 Dec 2010

-----
Nice work! Any chance this can work to see if someone is drawing a letter correctly. Such as the letter A. Thinking a small child who may do it a number of different ways including lifting their finger multiple times while drawing the letter.
-----
You can always "teach" Your app to read new gesture. In line 420 of library You have code like this:
print (table_To_Str( anglesMap ))
This give You gesture map from screen. Repeat the gesture several times and add it to gesture table. Example:
"53", => A
"67612" => A
"53032" => A
"6212" => A
"7612" => A

BeyondtheTech
User offline. Last seen 13 hours 45 min ago. Offline
Joined: 14 Apr 2010

I don't know if it's a Corona bug, but line 11 should be

if line and line.removeSelf then

If you tap on the screen, it will crash the program, because there's no removeSelf method if there's no object. Changing the line to the above will prevent that.

gtatarkin's picture
gtatarkin
User offline. Last seen 2 hours 45 min ago. Offline
Joined: 16 Dec 2010

Fix bug patch:

in main lua - line 11:


if (line and #pointsTable > 2) then
line:removeSelf()
end

kennw's picture
kennw
User offline. Last seen 15 hours 50 min ago. Offline
Joined: 9 Jan 2011

This is awesome... I was looking for something like this a little while ago.

I'm wondering how comparable the algorithm is to the $1 unistroke one in terms of accuracy and stuff? http://depts.washington.edu/aimgroup/proj/dollar/

I'll play with this corona version, but just wondering if anyone had any knowledge / opinions on that aspect.

Best,
~~Kenn

gtatarkin's picture
gtatarkin
User offline. Last seen 2 hours 45 min ago. Offline
Joined: 16 Dec 2010

Algorithm is very similar - after reduction number of points on screen it compares the angle of line to virtual horizontal line and result of this operation is compared with predefined patterns so accuracy should be very close to $1 gesture recognition.
Dowload lib from my webpage run it in Corona simulator and compare with this pattern:
http://twitpic.com/4a4otc

Regards
Greg

luoruize
User offline. Last seen 10 weeks 2 days ago. Offline
Joined: 9 Mar 2011

I tried using this library, but found it to be extremely not accurate. Is there anything better than this for Corona, or maybe I'm doing something wrong?

flashfreakmx's picture
flashfreakmx
User offline. Last seen 2 weeks 3 days ago. Offline
Joined: 4 Jun 2011

worked great out of the box... when I incorporated it in my project the line drawings only produce squares. What could be causing that? Thanks! I thought maybe since I was using it with director?

gtatarkin's picture
gtatarkin
User offline. Last seen 2 hours 45 min ago. Offline
Joined: 16 Dec 2010

Can you send me your project? grzegorz(dot)tatarkin(at)concat(dot)pl
Regards

niku744
User offline. Last seen 46 weeks 1 day ago. Offline
Joined: 13 May 2011

Is there another link that I can download the library from? The one above apparently does not work.Thanks in advance.

edgar86m
User offline. Last seen 5 weeks 25 min ago. Offline
Joined: 7 Jan 2011

Link seems to work fine for me. Thanks for the library! Works really well :)

yikboontan
User offline. Last seen 30 weeks 4 days ago. Offline
Joined: 24 Apr 2011

Hi. This is great stuff, for people like me who doesn't know a thing on gesture recognition algorithm. I've been using your code for a game in development right now. For a simple 5 min integration, players get to draw patterns for the game character to cast spells.

Much appreciated, gtatarkin.

Honestly, I'm here to ask if you can perhaps improve the code? At the moment it isn't nearly perfect. Drawing an 'O' for example, closely relates to 'U', (inverse U) 'A' which are the 3 gestures in the game. It sometimes recognizes U or inverse U as O, or vice versa. Having zero knowledge of the algorithm, thought it be best to ask you first before attempting to improve it myself.

Regards

Boon

gtatarkin's picture
gtatarkin
User offline. Last seen 2 hours 45 min ago. Offline
Joined: 16 Dec 2010

If you can wait few weeks until I finish big project in my full time job... Right now I don't have enough time to deal with Corona :/
Best regards

hafiz.arshad9
User offline. Last seen 39 weeks 1 day ago. Offline
Joined: 22 Aug 2011

Hi,
i am a new comer in corona and wants to design a game that should look like this one,
http://www.gamebeearcade.com/games/1539/2-many-bugs.html

How i get start the main game play screen and lines drawing around moving objects, any help will be appreciated

yikboontan
User offline. Last seen 30 weeks 4 days ago. Offline
Joined: 24 Apr 2011

If you're new to Corona, recommend downloading many sample projects and try to understand/copy their code. It worked for me, so maybe it'll work for you.

Thought starters:
1) Main game play screen/menu objects -> use ui library available on code exchange, they are many out sample projects using this library, recommend you go for Monster Tilt on Corona code exchange
2) To draw a line on screen: This gesture recognition library has some code on that for reference
3) Moving objects, two ways I know of: using transition.to() function, or simply specify the position of object at every frame refresh. Recommend 2nd method, sample code is in Monster Tilt as well.

yikboontan
User offline. Last seen 30 weeks 4 days ago. Offline
Joined: 24 Apr 2011

Thanks gtatarkin :)

rickyd1
User offline. Last seen 10 hours 16 min ago. Offline
Joined: 22 Jan 2011

Thanks for this great class. I am in a bit of a pinch.

I am suing this class with the Director class. Usine Runtime wasn't working so well so I am using an object to listen.

However, I have to swype 2 or more time in order for the gesture to be recognized. Any idea why this is?

If someone could point me in the right direction, I would appreciate it.

Thanks

Rich

My Code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
local pageBackground = display.newRect( 0, 0, display.contentWidth, display.contentHeight)
 
pageBackground.alpha = 0.01
 
function movePage(event)
        if "began" == event.phase then
                pointsTable = nil
                pointsTable = {}
                local pt = {}
                pt.x = event.x
                pt.y = event.y
                table.insert(pointsTable,pt)
                print( currView )
        elseif "moved" == event.phase then
                local pt = {}
                pt.x = event.x
                pt.y = event.y
                table.insert(pointsTable,pt)
        elseif "ended" == event.phase or "cancelled" == event.phase then
                action = Gesture.GestureResult()                        
                if action == 'I' then
                        director:changeScene( moveup, "moveFromBottom" )
                end
                if action == 'SwipeD' then
                        director:changeScene( movedown, "moveFromTop" )
                end
                if action == 'SwipeL' then
                        director:changeScene( moveleft, "moveFromRight" )
                end
                if action == 'SwipeR' then
                        director:changeScene( moveright, "moveFromLeft" )
                end     
                print( action )
        end
end
 
pageBackground:addEventListener( "touch", movePage )

hawkwood
User offline. Last seen 5 days 23 hours ago. Offline
Joined: 19 Feb 2011

Is this just me:

If you tap and hold for like 5 seconds then release without moving your finger, the "swipeR" event is sent.

Anyone else?

jkrassman's picture
jkrassman
User offline. Last seen 4 hours 34 min ago. Offline
Joined: 4 Aug 2011

Wowww, just googled this and it is fantastic.

One question, if I want to remove some characters from the class, should I delete the entries in gestureSign and gestures? Or will it not work?

I just want to have some characters for recognition.

Regards, Joakim

jkrassman's picture
jkrassman
User offline. Last seen 4 hours 34 min ago. Offline
Joined: 4 Aug 2011

I rather answer myself, yes it is just to remove the entries you don't want :)

rubs
User offline. Last seen 6 days 21 hours ago. Offline
Joined: 12 Sep 2010

Yes m8, It seems that if you tap (no need to hold on for x seconds), this will return "SwipeR" string.

gtatarkin's picture
gtatarkin
User offline. Last seen 2 hours 45 min ago. Offline
Joined: 16 Dec 2010

You can always add function to recognition SwypeL and SwipeR. Something like this:
- if function GestureResult() return SwypeL or SwypeR AND math.abs(x0 - x1) >100 (or 50) it's swype else it's just tap.
Regards

Fufutime's picture
Fufutime
User offline. Last seen 3 weeks 5 days ago. Offline
Joined: 18 Feb 2012

This is just what I need in my project. Alas, the page the download link points to, gives me a 403 error (Access forbidden).
@gtatarkin is this a mistake?

Kind regards

gtatarkin's picture
gtatarkin
User offline. Last seen 2 hours 45 min ago. Offline
Joined: 16 Dec 2010

:/ Just start to rebuild my website. Should be back online in week or so. Can you try this link right now? http://dl.dropbox.com/u/2496823/recognition_lua.zip
Regards

Fufutime's picture
Fufutime
User offline. Last seen 3 weeks 5 days ago. Offline
Joined: 18 Feb 2012

Brilliant!
Thanks for sharing