|
Meta’s tech leadership wants employees to get on board with using AI tools, which makes sense to me. What company wouldn’t want its developers using tools to make them more productive?
 |
So to incentivize AI usage, they started measuring token consumption and including it in performance reviews.
More tokens = better performance
I’ll give you one guess as to what started happening.
Employees started competing on who could use the most tokens. If it sounds reasonable to you, take it to the extreme. Some folks are literally just burning tokens for the sake of their numbers, not even for productive purposes.
This Meta analyst reports that they’re calling it “tokenmaxxing” and they even have an internal leader board for it:
 |
Today’s newsletter is sponsored by my friends at Augment Code.
Augment was kind enough to give me early access to Intent, their next-generation tool for developing software.
 |
All signs are pointing to the fact that the IDE is no longer the ideal place to create software.
Augment Code has jumped on this opportunity, and my first impressions are great.
Meta isn’t the only one counting tokens for performance
I like to look at big tech companies like Meta as a nice leading indicator, but they’re not the only one doing this already. If you’ve been reading the newsletter for a while, you’ll already know that other companies are incentivizing and measuring AI usage.
 |
|
 |
|
 |
|
 |
|
Of course, measuring tokens for performance with a “more is better” attitude is out there. Still, it seems others are following suit per Gergely Orosz:
 |
Goodhart’s law
It does surprise me that Meta would go with something this easily gameable. Goodhart’s law says:
When a measure becomes a target, it ceases to be a good measure
Do you think token usage is a good metric to measure performance?
Expect to see more weird metrics like this
Measuring engineering performance is hard. Like, really hard. Measuring AI proficiency is harder.
On one hand, should it matter how much an employee is using AI if their overall performance is great? No, it shouldn’t matter. The only reason a company wants to push AI usage would be because they believe AI makes people more valuable, even if those people aren’t bought-in.
So they seek to measure things like total token counts and incentivize higher usage. Which creates a perverse incentive, and you end up with tokenmaxxing.
My hope is that they see the flaw with this metric pretty quickly, but honestly it wouldn’t surprise me if we saw more like it pop up soon.
I want to live in a world where engineers are measured on output rather than process, but we’ll see where the next year takes us.
Invite your friends and earn rewards
| ||||||||||
| ||||||||||
