A research team at the School of Business and Management of The Hong Kong University of Science and Technology (HKUST Business School) has developed InvestLM – Hong Kong’s first open-source large language model (LLM) for financial generative AI (GenAI) applications, capable of generating investment-related, human-like responses comparable to those of well-known commercial chatbots, including OpenAI’s ChatGPT. InvestLM’s model parameters[i] and the insights from its development process have been made publicly available to support industry practitioners and researchers in deploying LLM-related technology.
AI-powered natural-language chatbots based on LLMs with billions or even tens of billions of parameters are known for their proficiency in handling a wide range of real-time text-generation tasks. Developing such chat services used to require abundant computing power that is exclusive to very large corporations. This changed when open-source general-purpose LLMs became available earlier this year, which has allowed those with moderate computing resources to train LLMs for their own needs.
By adapting from LLaMA-65B[ii], an open-source general-purpose LLM, with a high-quality and diverse set of finance and investment-related texts[iii] using a technique called instruction fine-tuning[iv], the HKUST research team has developed InvestLM, a state-of-the-art[v] LLM for the financial domain. InvestLM’s responses have been rated as comparable to those of state-of-the-art commercial LLMs, including GPT-3.5, GPT-4, and Claude-2,[vi] by financial experts, such as hedge fund managers and research analysts. This demonstrates InvestLM’s strong capabilities in understanding financial texts, which can potentially enhance the work efficiency of finance and investment professionals in tasks such as providing investment insights, extracting information from and summarizing financial news and reports, according to the research team. Moreover, compared to the foundation model LLaMA-65B, from which InvestLM is adapted, InvestLM exhibits better control in producing responses without hallucinations.
Prof. TAM Kan-Yan, Dean of HKUST Business School, said, “developing LLMs in-house can help financial firms gain competitive edge through the application of generative AI, while retaining better control over proprietary information and customers’ data. Reflecting HKUST’s lead in embracing generative AI among tertiary education sector in Hong Kong, this project has provided valuable insights for the financial sector on leveraging the fast-growing field of generative AI, in addition to making a powerful financial LLM accessible to the public.”
Prof. YANG Yi, Associate Professor of HKUST’s Department of Information Systems, Business Statistics and Operations Management, and a member of the research project team, said, “financial LLMs are either inaccessible due to their proprietary nature, or of low quality. To our knowledge, InvestLM is the first open-source financial domain LLM that can provide insightful responses to investment-related questions, as affirmed by financial professionals. By offering our insights into fine-tuning a foundation model for financial text generation, we hope that InvestLM can serve as a useful reference for industry practitioners in the financial sector and beyond who want to unlock the power of generative AI.”
The research team has discovered that applying a diverse set of high-quality, domain-specific instructions to train an LLM is more effective in enhancing its capabilities for handling domain-specific tasks than using a large volume of general-purpose instructions. In cases where computational resources are limited, LLM developers often use smaller LLMs instead of larger LLMs. The team has found that instruction tuning is especially effective in improving the performance of smaller LLMs than larger LLMs.[vii]
