technical architecture.

Multi-Touch and iOS/Android from the same project, Unity3D

by on Apr.06, 2012, under Uncategorized

There is a bit of a void out there in terms of recent information regarding multi touch in unity, most of the legwork of development looks to have been done in 09/10, looking around the forums there are many more questions than answers, and all out of date. SO! Lets record some of my findings from building a quick turnaround multi platform product config tool. this is the multi touch side of things, there was a slight bit more to keep it all happy with iOS/Android, but for the most part, I adopted their conventions where I could and forced the multi touch into that mold.

First interesting problem is how to begin dev and testing of a multi touch interface when you don’t have one to use! (wee!) Fortunately there is a few fun tools built by others in the CV world that can get you up and running quickly. There is much information to be had looking around in the NUIGROUP forums based around building your own touch tables. My immediate timeline didn’t let me play with full tables (sad!), fortunately Seth Sandler had put together a nice DIY project involving a webcam, some cardboard, glass, and a sheet of vellum, run on the Community Core Vision package . That got me multi touch input with output to the TUIO protocol fairly easily!

Next, looking into how to handle the TUIO inside of unity, there are a few main packages that work well, (Among others found at : tuio.org) I ended up with two that got serious consideration, the Mindstorm tuio, and the UniTUIO by the guys at xtuio, the code and overall base of the mindstorm system looked more solid and cohesive, but due to the schedule I ended up with the unituio for expediency sake, because it converts the TUIO touches to unitys base input system, by way of building bridged Input class and overriding the get touch functions. There is some editing needed to bring it back up to date with the 3.5 changes, but its minor and the console error log should get you there.

Now! The fun part, things are starting to work, now we can get down to biz. (Almost) There are a few other things to take into account, tuio systems are not taken into account with the built in unity gui. Alternate ui systems that work fairly easily with them are EZGUI and NGUI, I went with NGUI, and have been very happy with the support response time, and solid performance. I also adapted the truly awesome FingerGestures to use the tuio inputs, so now I’ve got a framework that works on all platforms, and with a loader for finger gestures that checks for the presence of the tuio inputs, and loads those, or else, reverts to base mouse input as needed. future: combine the two inputs so that there is no manual work on export!

One more: depending on the touch screens you’re using, you may need to convert windows 7 touches to tuio, I did this with the Touch2Tuio program. built in Windows 7 touches are functional for 1 touch, and the pinch to zoom gesture gets converted to the scroll wheel by default in unity 3.5 (And I believe it started working in 3.0 or 3.4.. the forum is vague on this.)

Performance wise, it still misses entirely too many touches for my peace of mind as a long term solution, but it worked quickly, and got us through the initial conference expo, and was a success! so work will continue, and it will get better and bigger and faster and more awesomeness. or something. I don’t know that I’d suggest this system for a long running exhibit, but it worked fine in the 3-5 day range for me.

Cheers.

 


8 Comments for this entry

  • PatHightree

    Nice writeup Dave, thanks for that.
    I’ve been doing much the same in the last month, ended up working with uniTUIO too.
    You sound really happy with FingerGestures, how difficult was that to integrate into the project ?

    greets, Patrick

  • Dave Buchhofer

    Hey Pat! How’s it going over there? Can i crash your place for Unite?! (Haha)

    It went pretty well, its not perfect by far, it still misses a fair amount of touches, and the uniTUIO doesn’t appear to close touches the same was as the unity Input and Fingergestures expects.

    Easiest way was to make a copy of the Plugins/FingerGestures/TouchScreenGestures.cs (I named my TUIOGestures) and edited the prefabs and finger gestures initializer accordingly.

    I detailed the minor code change in the fingergestures thread here

    If you have any trouble hit me up on gtalk.

    Cheers.

  • PatHightree

    Hey man, things are good here, still working with unity every day. Mostly on the drill ship simulator, but now also a touch table project.
    Please come and crash at our place! I’d be offended if you didn’t 🙂
    And bring that lovely wife of yours.
    You’re in luck because the Saturday after Unite we’ll be having our annual big party!

  • Karl

    Hi Dave,

    Thanks for the post, its just what I need….however I cant get it to work 🙁

    I have made the changes you suggested but I can see no Input.touches being generated by uniTUIO. I am using the latest uniTUIO and FingerGestures versions and Unity 3.5.1 on a Windows 7 touch device which uses PQLabs software. uniTUIO seems to work on its own but not with FG, I think it is not generating the touches?? Any help would be super 🙂

    Thanks

    Karl

  • Dave Buchhofer

    Hi Karl, you’re not the first person to say they didn’t get it all working, so I went ahead and added some more context to the explanation on the unity forum

    If you still have trouble let me know.

    cheers,
    Dave

  • Karl

    Thanks Dave, I had to add some extra code in to get it working but you got me 99% there 🙂

  • Derek

    Used your example to create a new prefabs for uniTUIO gesture, can control the example but can’t interact with the sampleui in the finger touch example. Was wondering if you experienced this problem.

    Thanks

  • Akil Senthil

    Hey Dave
    Thanks so much for giving us a direction on how to work with Touch screens!

Leave a Reply

Looking for something?

Use the form below to search the site:

Still not finding what you're looking for? Drop a comment on a post or contact us so we can take care of it!