Powerful artificial intelligence based on large language models can be computationally split to run on several smartphones. That could enable people to use AIs locally without relying on a cloud service鈥檚 data centres 鈥� and without needing to share sensitive queries or personal information with a tech company.
鈥淥ur key motivation was privacy,鈥� says at the University of California, Irvine. 鈥淚f users want to run large-language-model queries without revealing their questions to providers, how can we do that?鈥�
Abdu Jyothi and her colleagues developed the 鈥淟inguaLinked鈥� system, which can split and assign a large language model鈥檚 computation to individual phones connected wirelessly over a mobile network running at 4G speeds or better. The system also monitors each networked phone鈥檚 current available memory and network connectivity when assigning computational work to avoid overloading devices and to minimise communication delays.
Advertisement
Their experiments showed that, by linking four commercial phones through the LinguaLinked system, it was possible to enable BLOOM large language models 鈥� developed by the open-source AI company 鈥� to either generate text or analyse the positive or negative sentiment of text. The phones used in the experiments included three Google Pixel 7 Pro phones and a less powerful Cubot X30 phone.
The average AI processing speed per 鈥渢oken鈥� 鈥� the basic unit of input and output text for AI models 鈥� is about 2 seconds on a small AI model with 1.1 billion parameters and 4 seconds on a larger model with 3 billion parameters.
That is slow compared with dedicated computer servers that process each token within hundredths of a millisecond but still usable, says Abdu Jyothi. She suggests that the system could also scale up to include many more phones to handle much larger AI models. For example, OpenAI鈥檚 GPT-4 has more than 1 trillion parameters.
Existing techniques can already compress AI models with a few billion parameters to run on a single phone without much sacrifice in performance, says at the University of California, Berkeley. Still, he says the new LinguaLinked system could be combined with those preexisting compression techniques 鈥渢o handle much larger models that cannot currently be hosted on a single device鈥�.
The new system represents 鈥渁 very clever way of balancing the assigned work to match the target device鈥� while 鈥減artitioning the work on multiple, less powerful devices鈥�, says at The Ohio State University. He also described the work as a 鈥渟tep in the right direction鈥� for enabling people to use large language models on local devices instead of the cloud for both security and privacy reasons.
But such developments could be a double-edged sword. Serving up a large language model 鈥済uerilla style from a whole bunch of mobile phones which you鈥檝e cleverly chained together鈥� could pose new challenges for regulators compared with overseeing large AI models hosted in centralised data centres and cloud services, wrote , co-founder of Anthropic, a San Francisco-based company and OpenAI rival that develops large language models, in a newsletter.
鈥淭his increases the chance of AI being functionally ungovernable because it makes it possible to deploy and use systems via broadly distributed, generic hardware,鈥� wrote Clark.
arXiv
