My 2021 AI data roundup: the 10 numbers that mattered most

It's the last week of 2021, I've got a cup of hojicha, and I'm looking at every spreadsheet I've built this year. Twelve months of tracking AI data, prices, benchmarks, funding rounds, and model releases.

Here are the 10 numbers that I think will matter most when we look back at 2021. Not predictions. Not hot takes. Just data points with context.

1. $0.06 per 1,000 tokens

That's the price of GPT-3 Davinci, OpenAI's most capable model, and it hasn't changed all year. When I wrote about GPT-3 pricing in January, the cost difference between engines was the story. Twelve months later, the price is exactly the same.

In an industry where everything else is moving fast, GPT-3's pricing has been frozen since launch. That tells me one of two things: either OpenAI's costs haven't dropped enough to justify a price cut, or they're happy with the margins. Probably both.

For comparison:

| Model/Engine | Price per 1K tokens (Jan 2021) | Price per 1K tokens (Dec 2021) | Change | |-------------|-------------------------------|-------------------------------|--------| | Davinci | $0.0600 | $0.0600 | 0% | | Curie | $0.0060 | $0.0060 | 0% | | Babbage | $0.0012 | $0.0012 | 0% | | Ada | $0.0008 | $0.0008 | 0% |

Perfectly frozen. In 2022, I expect this to change. Competition is coming.

2. 175,000,000,000 parameters (still the biggest dense model that matters)

GPT-3's 175B parameters remained the benchmark all year. Other models got bigger on paper (Switch Transformer: 1.6T, Megatron-Turing NLG: 530B), but GPT-3 at 175B is still the model most people interact with through the API.

The interesting trend: parameter count stopped being the whole story in 2021. The Chinchilla-style insight (that training data volume matters as much as parameters) started gaining traction. I wrote about training costs in September, and the data suggested the "just make it bigger" era is already bending.

3. 10,000+ models on Hugging Face

Hugging Face crossed this milestone in October. I broke down the data: 22% NER, 18% text classification, only 8% text generation. The real work happening in ML is unglamorous classification and entity extraction, not chatbots.

The growth rate is also telling:

| Date | Models on HF | Time to double | |------|-------------|----------------| | Aug 2020 | ~1,000 | Baseline | | Mar 2021 | ~5,000 | ~7 months (5x) | | Oct 2021 | ~10,000 | ~7 months (2x) |

By end of 2021, it's closer to 12,000. The open source ML community is building real infrastructure.

4. 3.15x MSRP

The peak GPU price multiplier in 2021. An RTX 3080 with a $699 MSRP sold for $2,200 on eBay in March. I tracked GPU prices all year, and while prices have come down from the peak, we're still at roughly 1.5x MSRP as December ends.

The impact on ML accessibility is real. Independent researchers can't buy GPUs at reasonable prices. Cloud GPU costs are high. The barrier to entry for ML experimentation is higher at the end of 2021 than it was at the start.

5. 127 AI startups funded in Q1, growing to approximately 160 in Q3

I counted AI startup funding in Q1 and the numbers were dominated by enterprise MLOps and computer vision, with generative AI at 0.5% of total funding. By Q3, generative AI had crept up to about 2-3% of total AI funding. Still tiny. But the direction is clear.

The year-end picture:

| Quarter | AI startups funded | Est. generative AI share | |---------|-------------------|-------------------------| | Q1 2021 | 127 | 0.5% | | Q2 2021 | 142 | 1.2% | | Q3 2021 | ~160 | 2.5% | | Q4 2021 | ~150 (est.) | 3% (est.) |

Source: my ongoing tracking via Crunchbase and CB Insights.

6. 6 billion parameters, free

GPT-J-6B from EleutherAI was the moment open source AI got serious. I benchmarked it against GPT-3 Curie and the performance gap was 3-8% depending on the task. At the same parameter count, a free model nearly matched a commercial one.

EleutherAI has since released GPT-NeoX-20B in the works. The trajectory is: open source will always lag behind the frontier by 12-18 months, but the usable quality floor keeps rising.

7. ~$4.6 million to train GPT-3

The estimated training cost of GPT-3, based on compute analysis by Epoch AI. This number defined the "who can play" question for 2021. At $4.6M per training run, only well-funded labs can build frontier models.

But as I covered in my training cost analysis, the cost curve is bending. Hardware improvements and MoE architectures are making large models cheaper to train per parameter. The $4.6M number will look quaint in a few years. Maybe sooner.

| Year | Largest model | Est. training cost | Cost per billion parameters | |------|-------------|-------------------|----------------------------| | 2018 | BERT (340M) | ~$7K | $20,588/B | | 2019 | T5 (11B) | ~$1.3M | $118,182/B | | 2020 | GPT-3 (175B) | ~$4.6M | $26,286/B | | 2021 | Megatron-Turing (530B) | ~$12M | $22,642/B |

The cost per billion parameters is actually decreasing since 2019. The total bill goes up because models get so much bigger, but the unit economics are improving.

8. 34% year-over-year growth in AI papers

My arXiv paper count showed 2021 on pace for roughly 34,000 "machine learning" papers. At 94 papers per day, the field has a genuine discovery problem. Nobody can keep up with everything.

The most dramatic subcategory: transformer-related papers nearly doubled from 2020 to 2021. The architecture has conquered NLP and is now expanding into vision (ViT), audio, and multimodal applications.

9. 87% of AI safety funding went to just two companies

I didn't write a dedicated article about this (it'll come in 2022), but I've been tracking the numbers. Anthropic and OpenAI received the vast majority of funding that VCs categorized as "AI safety." DeepMind's safety work is funded internally by Google AI, so it doesn't show up in VC numbers. Independent AI safety research organizations received roughly 13% of the total.

Make of that what you will. I have opinions, but the data speaks for itself.

10. 5 days to 1 million users... wait, that's 2022

I'm cheating slightly. This number hasn't happened yet as I write this, but there's something brewing at OpenAI. They released WebGPT in December. They've been hiring like crazy. And their API usage has been climbing all year based on developer community chatter.

I don't know what they're building. But my data antennae are twitching.

(If you're reading this from the future: yes, I'm talking about ChatGPT. No, I did not predict how big it would be. Nobody did.)

What 2021 taught me

This was my first full year of systematic AI data tracking, and I learned a few things about the process:

The hype-reality gap is measurable. Twitter discourse and actual funding/usage data tell very different stories. Generative AI dominated the conversation but represented 0.5-3% of actual funding. Following the money and the download counts gives you a truer picture than following the tweets.

Round numbers matter for attention, not for analysis. GPT-3's 175B parameters. 10,000 models on Hugging Face. These milestones generate articles and discussions, but they're arbitrary. The trends between the milestones are what actually matter.

Tracking data consistently beats tracking news. I spent less time reading AI news in 2021 and more time maintaining my spreadsheets. The spreadsheets told me more useful things. News gives you narratives. Data gives you patterns.

I'm keeping every spreadsheet. They'll be even more interesting next year when I can compare 2021 to 2022.

Yoi otoshi wo. Happy new year.

If you found this interesting, you might also like:

-- dataku