The ‘perks’ of augmented reality

Surgical training and evaluation more accessible and cost-effective with overlay technology-based system

The Perk Station first received funding in March 2008.
Image by: Tyler Ball
The Perk Station first received funding in March 2008.

A new surgical navigation tool, the first of its kind in Canada, has made simulated needle-based surgeries more accessible.

The Perk Station, an augmented reality surgical navigation suite, strives to make training in percutaneous surgery more accessible for the medical and scientific communities.

Associate computer science professor Gabor Fichtinger, the leader of the project, said the machine uses image overlay technology to help students plan and execute needle-based surgery simulators. Previously, to simulate percutaneous surgeries, students used surgical navigation, where tools were overlayed on a computer screen, Fichtinger said.

“The problem with that is you do surgery here and the representation is over there. So, you split your attention, turn your head and you need a great deal of hand-eye coordination between the surgical field and the plan.”

The Perk Station stems from Fichtinger’s experience at Johns Hopkins University in Baltimore, MD, where he taught a computer-integrated surgery course from 2003 to 2007. The course included a one-week clinical laboratory component using human cadavers.

Fichtinger said upon arriving at Queen’s in 2007 he wanted to design a device to simulate surgical procedures done via needle puncture of the skin, such as biopsies, without the use of human cadavers.

“What you want to do is take away the expensive part, which is the imaging device itself and the live patient because you don’t want to practice on live people. So you want to create a standardized simulated environment for training people in order to measure their performance in different surgical scenarios.”

The project received funding in March 2008 and was used in classes starting that fall. The machine is now in its third or fourth embodiment, Fichtinger.

Fichtinger said using human cadavers for surgical training purposes requires operating rooms and clinical quality medical imaging devices such as magnetic resonance imaging (MRI) and computerized tomography (CT) machines. Participating students and teachers also require biohazard blood-born pathogen safety credentials and ethics board approval.

The Perk Station doesn’t require any of those qualifications.

Fichtinger said using an MRI machine for training purposes costs between $600 and $700 per hour, while the use of CT machines can cost $300 per hour.

Fichtinger said the Perk Station allows students and surgeons to plan surgical routes using a computer interface by calculating target locations, tool trajectories and other critical parameters. The surgical path is then shown through both image and laser overlay, allowing the to simulate the surgery by following the path created using the planning software.

“Like some people need a GPS for driving, some people may need a map, while some people don’t need anything.”

Fichtinger said Perk Station uses “phantoms” to simulate parts of the human anatomy, using a standardized gel formula.

“In addition to the actual Perk Station, you have to fake your patient. In this case, we suspended a piece of human vertebrae in gel. This is artificial skin, it sort of feels like the human body,” he said, adding that the surgical path is more important than simulating the texture of real human skin. “It’s a real human bone which you can buy. It’s cut up human bone, gelatin and silicone, in a case.”

Fichtinger said the Perk Station is also an evaluation tool.

“This is also a way of finding out what some surgeons—or surgeons to be—what their limitations are. It just might be that in third-year medical school, you put some people on this kind of test branch and they find out they shouldn’t be surgeons to start with,” he said.

Fichtinger said the Perk Station will be included as part of the patient simulation lab in the Queen’s School of Medicine’s new building, which broke ground in October.

Fichtinger, who also acts as an adjunct professor at Johns Hopkins University said each suite costs about $2,000 to assemble, with one model at Queen’s and two stationed at a satellite lab at Johns Hopkins.

“We expect that the clinical version of this will be installed at KGH eventually, but right now it is more important for us to take to medical student and resident training.”

Fichtinger said with its lightweight aluminum frame, the Perk Station was made to be easily transported to be shown at other universities. He said he’s taken it across North America and as far as London, U.K.

“I had a mechanical engineering student on that and we came up with this design for the prototype,” he said. “It folds down, it goes into a suitcase. You can hop on the train or the plane with this thing.”

Fichtinger, who teaches COMP 230 and CISC 330, which are undergraduate computer-assisted integrated surgery courses, said there are plans to turn the current undergraduate lab on the second floor of Goodwin Hall into a laboratory for percutaneous surgery.

“We’ll populate this undergraduate lab,” he said. “Perk Stations will be up around the wall and back—maybe three, maybe, four and some other surgical navigational tools.” Fichtinger said he’s applying for more grant money to fund the project. The project is primarily funded by the Queen’s Teaching and Learning Enhancement Grant and the U.S. National Cancer Institute.

“The original seed money paid for the first embodiment, but of course these guys are expensive and need to be on pay roll, so the American tax dollars are paying for them until we decide to get independent funding in Canada. Right now, it’s more important to do the job and we’ll worry about money later.”

Fichtinger said the hardware and software plans for the Perk Station are available to the public.

“What we have is open source for both the hardware, as well as the software. Anybody who wants can have a list of the parts for the assembly. You can build your own Perk Station at home if you want. I don’t know if anyone would want to do that.”

Tamas Ungi is the project’s post-doctoral researcher, in charge of implementation and improvement of the Perk Station. He arrived in Kingston last November after receiving his PhD in radiology at the University of Szeged in Hungary.

Ungi said the future of biomedicine lies in providing doctors with appropriate information at the right times, not necessarily developing computer-run systems.

“There are some different ideas, like make a robot and let the robot do surgery. I don’t think most people will accept this idea,” he said. “When you are inserting a needle into a patient, what is the information you need? It is the image of the needle, that you can virtually see on the patient in the correct position.”

Ungi said the Perk Station is part of a movement towards using augmented reality technology within biomedical computing.

“People just don’t trust robotic arms,” he said. “What is really difficult is to convince doctors to use something new. If you show them this system, what they do here is really the same thing they do every day—they just have an additional image of the needle overlayed where it should be after the insertion. But if you allow a robot to do everything for the operators, they will not like it.”

School of Computing director Selim Akl said the Perk Station has become a “showcase piece” for its biomedical computing program—which makes up one-third of computer science students at Queen’s.

“We try to attract undergraduates and tell them how important it is for the future of medicine to have computer scientists involved,” he said. “That’s the future of medicine. Medicine as we know it will be changed by work that is done in this department and others like it.”

Akl said starting next academic year, computer-integrated surgery will be a mandatory credit for all biomedical computing students. It’s currently an optional course.

Akl said he encourages students from other faculties to take advantage of the opportunity to learn using the Perk Station.

“Anyone on campus can take it, it has very minimal requirements,” he said. “Any students who are in Life Sciences, chemistry or biology, this is a very nice course for them—especially if they have ambitions to become a doctor.”

Akl said the School of Computing also uses the Perk Station as a form of community outreach by taking it to local high schools.

“I’ll take the Perk Station to high schools, give a quick lecture to high school students and have them try their hands on it.”

Akl said so far the suite has visited Marie-Rivier High School and Kingston Collegiate and Vocational Institute, with plans to take it to more schools.

“It was designed and built using parts ordered over the Internet. It’s very inexpensive to build one, but of course, it’s all a question of relative expense. We’re talking a couple of thousand dollars, not a couple of hundred because it is a research tool,” he said. “I would like to have two or three of them. One for undergraduates, one for graduates and one to take on the road.”

All final editorial decisions are made by the Editor(s) in Chief and/or the Managing Editor. Authors should not be ed, targeted, or harassed under any circumstances. If you have any grievances with this article, please direct your comments to [email protected].

Leave a Reply

Your email address will not be published. Required fields are marked *