Australian AI company develops world-first method to improve speed and reduce computation power requirements of LLMs
CANBERRA, Australia, Nov. 11, 2024 (GLOBE NEWSWIRE) — Leading machine learning and AI specialist Trellis Data has today announced a groundbreaking innovation in large language model (LLM) decoders that enables outputs to be generated more than three times faster.
Trellis’s Dynamic Depth Decoding (D3) technique uses speculative decoding to achieve an average speed increase of 44% compared with the previous fastest decoder; it is the fastest decoder available in the world today.
LLMs are the architecture behind AI systems, and D3 enables them to generate text approximately 3.16 times faster than standard decoding without compromising on accuracy.
The World Economic Forum estimates the computational power required to sustain global growth in AI is doubling every 100 days. With AI systems creating more and more value for organisations around the world, methods that create efficiencies in cost and energy requirements and mitigate environmental impact have become highly sought after.
According to Trellis Data research submitted for global peer review, the improvement in speed delivered by D3 translates to an estimated 68.4% reduction in computational power requirements, which in turn lowers the costs and carbon emissions associated with running LLMs.
Trellis Data CEO Michael Gately said, “There are three components critical to the success of any AI system: trust, speed and performance. Speed is a key area because it has a direct correlation with cost and environmental impact.
“D3 enables us to address one of the key bottlenecks of speed – the decoder – offering customers a reduction in the cost of running AI servers and a lower carbon footprint.”
“Trellis Data is at the forefront of the rapidly developing AI field and we’re excited to make D3 available on our platform. D3 will provide customers using our speech management and knowledge management capabilities immediate benefits including an enhanced user experience and greater ability to scale.”
D3 is broadly applicable to generative AI use cases, including chatbots, text summarisation, translation, transcription, and content generation. Additionally, the reduced computational costs and carbon emissions deliver a competitive advantage for customers, particularly as LLM size and usage continues to grow.
Trellis Data uses a broad spectrum of AI technologies, including generative AI, transcription, translation, and computer vision, specifically designed to save time and reduce costs. The development of D3 is another key milestone in Trellis Data’s strategy to address the important concerns relating to the adoption of new AI technologies, as they deliver secure, disconnected AI applications that seamlessly integrate with existing business systems.
Headquartered in Canberra, Trellis Data has a growing team of over 40 staff with annual recurring revenue (ARR) growth set to exceed 100%. The company also has offices in Adelaide, Sydney and Arlington (Virginia, USA), with imminent plans to expand into the Middle East and South East Asia.
Trellis Data provides AI solutions to a diverse range of customers, including government agencies, defence and law enforcement, training and supply chain enterprises, and companies in high-security sectors such as finance and legal.
Michael Gately is speaking at MilCIS 2024 in Canberra on 12 November and is a finalist in the Artificial Intelligence category of the 2024 AmCham Alliance Awards, with winners to be announced at a Gala in Sydney on 14 November.
