+ OzTrekk Educational Services Home
 
 

Posts Tagged ‘Pokemon Go’

Friday, October 6th, 2017

University of Melbourne physiotherapy students use augmented reality

From Pokémon GO to the classroom—how a collaboration between the University of Melbourne and Microsoft in AR is taking students under the skin of their patients.

Story by Andrew Trounson, University of Melbourne

Pokémon GO pushed augmented reality, or AR, into the mainstream, sending 500 million people around the world chasing cartoon characters on their phones. But now, in a unique multi-disciplinary collaboration, it’s making the leap from entertainment to education.

A new fusion of augmented reality, gaming technology, and anatomy is giving physiotherapy students at the University of Melbourne access to cutting-edge technology to take a look inside the human body by projecting different layers of muscles and bones over the top of a volunteer “patient.” It provides an inside view of how the body works as it moves in real time.

Melbourne physiotherapy students use augmented reality

Learn more about the Melbourne physiotherapy program

The technology, called the Augmented Studio, is designed to radically enhance the teaching of physiotherapy where students currently use their knowledge of anatomy to understand how muscles work beneath the skin of a patient because they can’t see through them. But the Augmented Studio, developed by researchers at the University of Melbourne, bridges the gap between that theory and practice.

By using tracking sensors mounted on a scaffold it projects images of our muscles and skeleton directly onto a volunteer. The images automatically follow the shape and movement of the body, giving students in the studio space an interactive all-round view of how our bodies work. It can even allow them and their teachers to “draw” on the projected image to make information and action more explicit.

“What we are doing is overlaying virtual models of what we look like underneath our skin and synchronising that with real human action,” says Dr Thuong Hoang, who is a Research Fellow at the Microsoft Research Centre for Social Natural Users Interfaces at the University of Melbourne.

The Augmented Studio was built by Dr Hoang, computer engineer Zaher Joukhadar and Phd student Martin Reinoso, who adapted Microsoft’s Kinect body sensing and tracking device as well as “RoomAlive” projection technology, both of which were originally designed for computer gaming. Once a person steps into the projection space and forms a T-shape with their arms outstretched, the trackers lock on to them and the projected image conforms to their shape and movement.

At the moment the projected overlay doesn’t show how our muscles actually move when we contract and relax our muscles. Instead, it tracks the body and movement at the joints. But eventually Dr Hoang wants to add in animation that can show the actual movement of muscles as the model moves.

University of Melbourne Physiotherapy lecturer Dr David Kelly says the students quickly embraced the technology during pilot sessions in 2016, which are continuing in 2017. He says the combination of live movement and interaction, in which students could actually move and feel the model’s limbs, helps them to grasp the relationship between their learned anatomy and how it works dynamically.

“For first-year students it can be really hard to bring together anatomical knowledge with how the body actually works because it can be difficult to visualise. But when they see a real person who they can interact with, while also seeing the muscles and skeleton projected over the top, combined with the ability to draw and write on the body, it all becomes much easier for the students to learn about how the body moves,” says Dr Kelly, from the Melbourne School of Health Sciences.

The Augmented Studio also provides a more visual and intuitive way of learning that Dr Kelly says will benefit those students who naturally learn more easily by direct visualisation, rather than through reading and listening. “There has always been a group of students that struggle because the limited ways in which we have to teach may not conform to how they learn best,” he says.

Developments in AR, which seeks to use technology to enhance what we can already see, hear and feel in the real world, are far ahead of chasing GPS tracked Pokémon. There are viewing devices such as glasses that can overlay what we see with three-dimensional graphics, video and holograms, and we can generate projections like games that people manipulate by moving our hands.

The big advantage of the Augmented Studio over advances like 3D holograms is that the students can actually touch and move the body, making it a much more interactive experience. They also don’t have to wear headgear, which means it could potentially be used in bigger settings with larger numbers of students.

“It has always been hard to capture the dynamic side of how our anatomy works, so the difference here is the high level of interaction you can achieve. The student can, for example, ask the model to kick and they can then look at variations from different angles at what is happening as someone kicks,” Dr Kelly says.

The Augmented Studio is still in early-stage development and Dr Kelly would love to see it migrate to using muscle animations. Dr Hoang is also working to develop a system for the student interaction with the model to be automatically recorded onto their tablets so they can have a permanent record of what they were learning.

Another challenge is to find a way to make the studio more transportable and quicker to set up. At the moment the studio can work very effectively in a dedicated tutorial space where it could be permanently set up, but Dr Kelly says a more portable set up would increase its flexibility for teaching.

The Augmented Studio is an extension of Dr Hoang’s earlier work exploring how virtual reality and body tracking could be used to help guide body movement for dance and marital arts students. Arising from a collaboration between the physiotherapy department’s Teaching and Learning director, Associate Professor Louisa Remedios and Professor Frank Vetere, Director of Microsoft Social NUI, Dr Hoang started working with the physiotherapy department on developing a teaching aid. He then realised that virtual reality, in which you are immersed in an entirely created world, wasn’t suited to teaching physiotherapy that is very hands on.

“When we got into the class rooms we had to change our thinking. VR just wouldn’t work in the tactile environment in which they learn and practice,” Dr Hoang says. It was when he noticed that students kept referring back to anatomy charts when they were practicing on each other that he started thinking of using augmented reality to put the virtual muscles on the body

Dr Hoang is now working on extending the tracked projection technology to various health and fitness areas, and even in performance art. He says that using tracking sensors with projections it is possible to create guides that show people how to position their bodies for practicing fitness, sport and dance.

Using virtual reality headsets he and PhD student Martin Reinoso have already developed a prototype that allows a martial arts teacher to remotely instruct students on the right position to hold. By using body tracking and linked headsets student can match their movement to align with those of their teacher. There is also scope to project information on our own body’s performance, such as heart rate and breathing, so it is visible either on our projected selves or on a nearby surface.

“The innovation we have created isn’t just limited to the fixed information that we have been projecting so far. If can be used to project dynamic information onto yourself or any surface around you,” Dr Hoang says. “All of what I’m dreaming of is very possible.”

About the Melbourne Doctor of Physiotherapy

Eligible University of Melbourne Physiotherapy candidates for admission will have completed undergraduate studies in human anatomy and human physiology at the university level. Other subjects which may be helpful for physiotherapy applicants include psychology, physics, biomechanics, research methods, evidence-based practice, statistics, biochemistry, and additional units of anatomy.

Program: Doctor of Physiotherapy (DPT)
Location: Melbourne, Victoria
Next available intake: February 2019
Duration: 3 years
Application deadline: TBA. For the 2018 intake, the application rounds closed June 1 and July 27, 2017.

Apply to the Melbourne Doctor of Physiotherapy Program!

*

Would you like to learn more about the University of Melbourne Physiotherapy program ? Please contact OzTREKK’s Australian Physiotherapy Schools Admissions Officer Krista McVeigh at krista@oztrekk.com.

Friday, July 22nd, 2016

Sydney PhD candidate explains what Pokémon Go can tell us about augmented reality experiences

From university campuses to public parks and suburban spaces, Pokémon Go is seemingly everywhere—including the OzTREKK office: yesterday we caught a Zubat and a Kakuna!

Kyle Moore, a doctoral candidate in the University of Sydney’s Department of Media and Communications, is researching urban mobile gaming.

what can Pokémon Go tell us about augmented reality experiences

PhD candidate Kyle Moore outside the Museum of Contemporary Art in Circular Quay. Augmented reality is changing gamers’ experience of public spaces, says the urban gaming researcher (Photo credit: University of Sydney)

The research has focused largely on players of Ingress, a massively multiplayer augmented reality game developed by Niantic, and has in recent months included field and beta testers of Pokémon Go. Kyle’s thesis explores how gaming can influence players’ understandings of urban environments.

What does Pokémon Go reveal about augmented reality and the future of gaming?

“We’ve seen this happen with the success of mobile games like Angry Birds and Candy Crush, which both became hugely successful and symbolic of a new gaming audience. Similarly, Pokémon Go reveals gamers’ willingness to adopt new forms of technology, like augmented reality and location-awareness, which previously were very peripheral to everyday mobile gaming and more common amongst experimental, artistic, or even gimmicky mobile games,” says Kyle.

Your PhD looks at the emerging technological phenomenon of urban mobile gaming. Is Pokémon Go a breakthrough or a fad?

“In terms of the technology, Pokémon Go is similar to a range of other games that have been released. These were often experimental, playing around with the capabilities and limits of the devices. Both Nintendo and Niantic labs have used these technologies before. Nintendo, with their Nintendo 3DS portable console, has a number of built-in and downloadable augmented reality games, while Niantic are best known for their successful alternate-reality game Ingress. Pokémon Go has certainly reached a viral status—whether or not this will fade away remains be seen. Without a doubt there will be numerous clone games pushing the limits of these emerging technologies.”

How are augmented reality games changing our public spaces?

“Numerous stories about the dangers of playing Pokémon Go rightly signal that refusing to acknowledge the space we are in can in fact have consequences,” says Kyle, “and that games should no longer be considered outside of these parameters.”

“The popularity of the game means we need to rethink our engagement with traditional spaces of play and leisure, such as parks and playgrounds, as well as spaces where play has traditionally been seen as subversive—city spaces in general. It’s also important to consider the implications this has for spaces outside the city, for those in rural or suburban spaces, who will have difficulty playing in these familiar spaces, and the impact traveling to play will have on these groups.”

Pokémon Go is essentially a toy. What makes it so popular with adults?

“It’s difficult to avoid putting it all down to nostalgia and 20 years of fan engagement,” says Kyle.

“But then again, I can’t deny that this would have had a huge impact on the widespread adoption of the game. Those who would have played Pokémon in the late 1990s now have increased mobility—they are able to freely move across and through city spaces, generally they are able afford mobile devices, and manage their free time.

“The game also appears to be hugely popular with children, whose parents may have never played a Pokémon game growing up. I’ve seen parents teaching their children how to effectively use their device to catch Pokémon, and vice-versa. There is a cross generational element to the game, which links towards shared ownership and literacies of mobile devices—that parents and children learn from these shared experiences.”

What are the developments to look out for in augmented reality?

“For augmented reality, we can probably hope to see more sophisticated modes of layering. Generally, this technology can read from codes, similar to a QR code, to give grounding to an image. This technology may feasibly be able to read buildings or landmarks in a similar way.”

Kyle adds: “In terms of developments for Pokémon Go, there is talk of using near field communication (NFC) as a means of trading between mobile devices. For augmented reality and location-based games in general, we may see more integration of location-awareness, in both mobile devices and potentially in portable gaming devices too. Alongside this, portable devices like the PlayStation Vita allow for 3G networks, which may be useful in real-time networked game play and location-awareness.”

*

Would you like to learn more about the University of Sydney’s graduate arts degrees? Contact OzTREKK Admissions Officer Shannon Tilston at 1-866-698-7355 or shannon@oztrekk.com!