Surgeons could use a ChatGPT-like interface to instruct a robot to carry out small tasks, such as suturing wounds and dilating blood vessels.
Surgical robots have been in use for decades, but these are normally controlled entirely by a human. Researchers are now developing autonomous versions that can perform parts of an operation without human assistance, but these can be difficult for people to work with because of a lack of fine control.
To address this, at the University of Toronto, Canada, and his colleagues have developed a virtual assistant, called SuFIA, that can translate simple text prompts into commands for a surgical robot and defer to a human surgeon when it gets stuck.
Advertisement
SuFIA uses OpenAI鈥檚 GPT-4 large language model to break down requests from a surgeon, such as 鈥減ick up the needle and move it鈥�, into a sequence of smaller subtasks. These subtasks will then trigger a piece of software to run in another tool, such as a robotic surgeon or camera.
鈥淚t鈥檚 like when you play video games, you have a much lower dimensional controller even though the actual game you鈥檙e playing is much more complex,鈥� says Garg. 鈥淚f you鈥檙e playing soccer, you鈥檙e not really controlling the humanoid player, you just press kick and shoot and jump and pass.鈥�
Garg and his team tested four tasks in a simulated environment, including picking up and moving needles and dilating blood vessels. They then tested the needle tasks with a real surgeon.
Automating simpler surgical tasks, such as moving a needle around, could help surgeons better perform procedures, says at University College London. However, it might take some time before regulatory bodies are convinced that this should be used in clinical trials, he says.
鈥淭he question is, what real value does it add to the patient care?鈥� says Stoyanov. 鈥淭hat鈥檚 the bit that, until it鈥檚 in place safely, is very difficult to answer because sometimes we develop technology with one indication, and then you put it in the clinic and you realise that actually it doesn鈥檛 quite function as you wanted.鈥�
Using large language models in surgical assistants will also require very careful safety features to mitigate the risk of 鈥渉allucinations鈥�, where the AI convincingly makes up facts, says Stoyanov.
arXiv
