AI Training Costs Continue To Plummet

High AI training costs have been a significant barrier to AI adoption, preventing many companies from implementing AI technology. According to a 2017 Forrester Consulting Report, 48% of companies highlighted high technology costs as one of the primary reasons for not implementing AI-driven solutions.

This article was written by Haziqa Sajid and originally published by Unite.AI.

However, recent developments have shown that AI training costs are rapidly declining, and this trend is expected to continue in the future. According to the ARK Invest Big Ideas 2023 report, training costs of a large language model similar to GPT-3 level performance have plummeted from $4.6 million in 2020 to $450,000 in 2022, a decline of 70% per year.

Let’s explore this trend of declining AI training costs further and discuss the factors contributing to this decline.

How Have AI Training Costs Changed Over Time?

According to the recent ARK Invest 2020 research, the cost of training deep learning models is improving 50 times faster than Moore’s Law. In fact, the expense associated with running an AI inference system has drastically reduced to almost negligible levels for numerous use cases.

Moreover, training costs have decreased ten times yearly over the past few years. For instance, in 2017, training an image classifier like ResNet-50 on a public cloud cost around $1,000, but by 2019, the cost had decreased significantly to approximately $10.

These findings align with a 2020 report by OpenAI, which found that the amount of computing power needed to train an AI model to perform the same task has been decreasing by a factor of two every 16 months since 2012.

Furthermore, the ARK report highlights the declining AI training costs. The report forecasts that by 2030 the training cost of a GPT-3 level model will come down to $30, compared to $450,000 in 2022.

Factors That Contribute to Declining AI Training Costs

Training AI models become cheaper and easier as AI technologies continue to improve, making them more accessible to a wider range of businesses. Several factors, including hardware and software costs and cloud-based AI, have contributed to declining AI training costs.

Let’s explore these factors below.

1. Hardware

AI requires specialized high-end costly hardware to process high volumes of data and computations. Organizations like NVIDIA, IBM, and Google provide GPUs and TPUs to execute high-performance computing (HPC) workloads. High hardware costs make it difficult to democratize AI on a large scale.

However, as technology advances, hardware costs are decreasing. According to the ARK Invest 2023 report, Wright’s Law predicts that AI-relative compute unit (RCU) production costs, i.e., AI training hardware costs, should decrease by 57% annually, leading to a 70% reduction in AI training costs by 2030, as shown in the graph below.

2. Software

AI software training costs can be lowered by 47% annually through increased efficiency and scalability. Software frameworks like TensorFlow and PyTorch enable developers to train complex deep learning models on distributed systems with high performance, saving time and resources.

Furthermore, large pre-trained models like Inceptionv3 or ResNet and transfer learning techniques also help reduce costs by allowing developers to fine-tune existing models rather than training them from scratch.

3. Cloud-Based Artificial Intelligence

Cloud-based AI training reduces costs by providing scalable computing resources on demand. With the pay-as-you-go model, businesses only pay for their computing resources. Also, cloud providers offer pre-built AI services that accelerate AI training.

For instance, Azure Machine Learning is a cloud-based service for predictive analytics that allows rapid model development and implementation. It offers flexible computing resources and memory. Users can scale up to thousands of GPUs quickly to increase their computing performance. It allows users to work through their web browsers on pre-configured AI environments, eliminating setup and installation overhead.

The Impact of Declining AI Training Costs

The decreasing costs of AI training have significant implications for various industries and fields, resulting in improved innovation and competitiveness.

Let’s discuss a few of them below.

1. Mass Adoption of Sophisticated AI Chatbots

AI chatbots are on the rise due to declining AI costs. Especially after the development of OpenAI’s ChatGPT and GPT-4 (Generative Pre-trained Transformer), there has been a noticeable surge in the number of companies looking to develop AI chatbots with similar or better capabilities.

For instance, five days after its release in November 2022, ChatGPT amassed 1 million users. Although today, the cost to run the model at scale is approximately $.01 per query, Wright’s Law predicts that by 2030, chatbot applications similar to ChatGPT will be deployable on a massive scale much cheaper (estimated $650 to run a billion queries), with the potential to process 8.5 billion searches per day, equivalent to Google Search.

2. Increased Use of Generative AI

The declining costs of AI training have led to a surge in the development and implementation of generative AI technologies. In 2022, there was a significant increase in the use of generative AI, driven by the introduction of innovative generative AI tools, such as DALL-E 2, Meta Make-A-Video, and Stable Diffusion. In 2023, we have already witnessed a ground-breaking model in the form of GPT-4.

Apart from image and text generation, generative AI is helping developers write code. Programs like GitHub Copilot can help complete a coding task in half the time.

3. Better Usage of Training Data

Reduced AI training costs are expected to allow better utilization of machine learning training data. For instance, ARK Invest 2023 report suggests that by 2030, the cost of training a model with 57 times more parameters and 720 times more tokens than GPT-3 (175B parameters) is projected to decrease from $17 billion to $600,000.

Data availability and quality will be the primary limiting factor for developing advanced machine learning models in this low-cost computing world. However, training models would develop the capacity to process an estimated 162 trillion words or 216 trillion tokens.