Bionic Vision Backpack

The challenge

The backpack is the body-worn host for that retinal implant, and the build that went into human trials after earlier generations had been validated in animal studies. Its job was to make the system usable away from fixed laboratory equipment — in a research setting, or at a recipient's home.

The work

The signal chain

Everything the system knows about the scene originates outside the body: an Intel RealSense camera mounted in a pair of glasses streams colour, monochrome and depth data back to an NVIDIA Jetson Xavier compute module in the pack. The pack then drives the implant: power and data pass inward through the skin to a subdermal interface, over a proprietary encode-decode scheme, to stimulate surviving neural structures. The camera is external and the stimulus travels in — not the other way about. The implant-side communications came from the Bionics Institute, out of the same cochlear implant lineage.

What to send

The interesting problem is what to send. The implant's electrode density is low, so there is no useful way to map a visual field directly onto it; the recipient would receive a smear. So the system interprets the scene rather than reproducing it. Depth and visual-field data are processed on the Jetson, objects and people are recognised, and the result is encoded as sequences of optical flashes — a symbology, in which a distinct pattern means your son or your partner rather than an image of them. What the recipient learns to read is not a picture but a vocabulary.

The wearable end

Signal integrity and processing across that chain — camera to compute to implant, in real time, on a person who is walking around — was the hardest part of the design. Tekt's scope was the wearable end: the power system, the communication path carrying data and power to the custom implant safely, and the vision-processing and data-storage sub-system behind it. Tekt designed the PCBs. It runs on battery, because a mains tether defeats the purpose, and it is packed into a ruggedised housing, because the requirement was safe long-term operation in the field rather than a bench session.

The vision processing is NVIDIA-accelerated. Tekt's NVIDIA work sits in the Jetson edge AI line: on 2 September 2022 the company announced it had "strategically aligned with NVIDIA as a new Jetson Ecosystem partner", NVIDIA's only edge AI ecosystem partner in the Australasian region. That partnership post-dates this project: Tekt was already listing "bionic implant processing systems" among its work at its twelfth anniversary in April 2021.

The RealSense headset

The camera end of the system is a pair of glasses, and it is a product in its own right rather than a mount for someone else's module.

An Intel RealSense camera is integrated into the frame, which is where the colour, monochrome and depth streams come from — three kinds of information about the same scene, from one sensor cluster sitting on the brow bar behind the lenses. Depth is the reason the system can describe a scene rather than merely photograph it: knowing how far away a person is standing is what makes "your son, two metres ahead" expressible in a language of flashes.

The frame carries the recipient's side of the link as well as the camera. A coil sits on a short stalk behind the ear, which is how power and data cross the skin to the subdermal interface — the transcutaneous half of the chain, in the one place on a head where a coil can sit against bone reliably. A USB-C tether runs to the pack, so the glasses carry sensing and coupling while the compute and the battery stay off the head where their weight does no harm.

Everything about it is built for a clinical programme rather than a shelf. The units are marked for clinical trial use only, and the surrounding control hardware is designed to match — sealed, buttoned, and legible to a researcher rather than a consumer.

Human trials

The system was used in a medical study with four pilot patients.

About the client

Tekt has worked with Bionic Vision Technologies and the associated University of Melbourne research group since Matt Adams was a sole trader — a programme to restore a sense of sight to people who have lost it to retinitis pigmentosa, using an electrode array implanted at the retina.