I have been using the iPhone for something like 10 years now. I am getting the urge to de-apple since I fell down the slippery slope and now have an iPad, watch, apple TV, MBP, etc.
Exercising less dependence on a single company for everything seems like a good thing, but I'm struggling to really ground that principle against the convenience of just buying "what works." Does it really matter at the end of the day?
I assume all the companies have zero respect for any of their users now. At least Apple can maintain a tablet and laptop line for more than a few years without the urge to restart it with "brand new ideas".
I wonder if they meant to draw a link between cybersecurity coding and debugging specifically or this really will apply to all coding and debugging. If it really is a more general restriction, then this is practically the same as it still being restricted.
While I appreciate, they publish this information, it's increasingly hard to keep track of it all. I've lost the mental model of how different models at different effort levels perform and what tasks they are good at.
In practice, I tend to just use the default on Claude Code that works well enough. But I wonder to what degree other users really play around with these settings to optimize for their project.
I always use Opus 4.8 at max effort for everything. The $20 subscription didn't have enough tokens, but the $100 one had too many of them. So now I just max out Opus in order to maintain 100% weekly utilization.
I'm a senior skilled developer and I find Anthropic $20 + Open AI $20 + OpenCode Go $10 offers more value than $100 on any particular service.
Juggling between all different models/agents is quite simple with Zed.
A caution about OpenCode Go though, the entire company seems to be run by AI so there's lot of billing related issues with zero support. I subscribe new every month as I lost money due to double payment with automatic subscription.
For non coding related tasks I use local models.
P.S. If anyone is interested to read more about my setup, let me know I'll publish a blog post.
I've been running about the same stack for well over a year now, Anthropic cheapest + OpenAI cheapest + z.ai coder (black friday offer).
The Z.AI is a bit wonky, so now I'm moving to Openrouter for Qwen+Kimi+Deepseek?GLM
My summer project is to figure out a proper agentic system where a "big" model does the planning, but automatically uses a cheaper one for the grunt work. Having Opus to config edits is just stupid :)
> My summer project is to figure out a proper agentic system where a "big" model does the planning, but automatically uses a cheaper one for the grunt work. Having Opus to config edits is just stupid :)
My company pays for the tokens so I don’t care. Biggest model and max everything. The slight risk of a smaller model making a mistake is more expensive than just running the bigger model all the time.
Would love to read that blog post. I'm toying with running local AI model with Claude and GLM as well depending on a task. Pretty decent success but it could be better.
I'm a heavy enough user that I have both the OAI and Anth $200 plans. I always use at least 50% of my weekly Opus quota at Extra setting (meaning I use double the limit of the $100 plan, at minimum). Max I rarely touch because it is twice as slow and the incremental capability gain is minimal. Usually if Opus can't sort something well at Extra, the answer isn't to use Max but to hand the issue off to GPT-5.5 at XHigh.
I too have settled into a kind of dual Claude/GPT model setup. I will often use one to review the other's work, or critique the other's plan in some way. Sometimes I'll have Claude implement a feature one way, then have GPT do it the other way, then have them both review each other's implementation. Then synthesize a final plan from the previous implementations+reviews.
I might just be having fun with models, but I have actually noticed their capabilities vary somewhat, and so my (perhaps vain) hope is that by using both, one can catch each the other's blindspots. It's still unclear to me if that's consistently happening, but I am making substantial progress in my personal and professional projects, so something seems to be working.
> Sometimes I'll have Claude implement a feature one way, then have GPT do it the other way, then have them both review each other's implementation. Then synthesize a final plan from the previous implementations+reviews.
I've done variants of this a number of times, but feel like it was a generally waste of my time to then have to compare them and write up which parts I liked or disliked: if the output is something substantial, each will have its pros and cons. Clear-cut wins aren't very common. Of course it could work well if we automated the whole thing with an orchestrator; you just need a model with actual good taste (according to your own preferences) ... so we'll have to compare all the models to find that one
Yes, same, between the two of them I feel like results are just better because they have different priorities.
At the same time, I’ve invested in tooling that prints and lints architecture I want, so which model is less of an interesting decision, because the results tend to be very close.
This is actually very counterproductive with Opus 4.8 - you are wasting a lot of time.
For Opus 4.8 training with overblown internal dialogue and second opinions - Max effort burns just tokens and wastes time without much value. Spinning wheels.
Why are you optimising for token use rather than getting whatever tasks you're trying to do done? Max effort Opus will happily sit there talking to itself for five minutes fixating on every tiny detail while Sonnet or even Haiku will just blast through the problem and be ready for the next one in seconds.
Because I'm paying $100/month for this, so I absolutely want to use all of those dollars. Any leftover weekly usage is a gift to Anthropic.
I am getting things done. I've made major progress on my projects, and even started new ones. My most requested tasks are: code review, brainstorming and research. The fixation on tiny details is exactly what I'm paying for.
I've been plugging this perhaps too many times now, but I am trying to bootstrap a user-sourced corpus of exactly "what model is good at task X". So, not benchmarks, but high-level tasks. There's a bit of a ordering problem in that nobody wants to bother commenting on a site that has few comments - so PTAL and contribute if you can. https://model.reviews
In practice I don't think any harness (happy to be corrected here!) uses the lesser capability models for writing code. The cost trade-offs are rarely worth it.
They are often used for reading code though.
To expand on this, while the "big model to write a plan, small model to write the specific code" idea is quite common it trips up on edge cases.
In theory the flow works like this:
- small fast models read lots of code, and pass details to the large model to write a plan
- large model takes those details and writes a detailed plan
- medium models write the code
The issue happens when the medium model hits something that the plan didn't take into account (which happens a lot - the big model didn't actually read the code). Then it has to either guess, or pass back to the large model.
If it guesses, the plan usually starts to fall to bits.
If it passes back to the large model, inevitable the large model has to start reading lots of code. In that case you are paying the expensive tokens to read so you might as well have it write the code too (many less tokens are written than are read)
It might be possible to get this to work, but I haven't seen anyone who has tried agentic work with frontier models be satisfied with this hybrid setup.
I'd note that Amp (mentioned above) is probably the leader in using multiple providers in a coding agent but still uses frontier models to write code.
> In practice I don't think any harness (happy to be corrected here!) uses the lesser capability models for writing code. The cost trade-offs are rarely worth it.
That's not something I understand very well. The less expensive models will quite happily chug away at tasks, if the codebase is well-structured (small files help a lot) and your instructions are clear. In contrast, I've never seen a large model turn bad instructions (instructions that would cause a human to think before starting) into a result I liked. You can run small models almost 10-100x as long for the same price in dollars, which covers a lot of correction and adjustment.
Why does everyone say the trade-offs are rarely worth it?
> In contrast, I've never seen a large model turn bad instructions (instructions that would cause a human to think before starting) into a result I liked
I think the distinction is here.
I expect my agent to build from product level descriptions. This might include specific special cases that I call out, but will rarely highlight existing special cases or edge cases - they already exist in the code, and I'd expect a programmer to make sure that behavior continues to work.
If a feature hits lots of these edge cases, the weaker model that is reading the code (aka Haiku) won't understand their significance, and will report back to the planning model incomplete or incorrect information.
The planning model (Opus - which hasn't actually seen the code remember!) will build a plan that is incorrect or incomplete and delegate coding to the mid level model (Sonnet) which will do it's best to make things work, without understanding the overall picture.
This is how you end up with slop - for example Sonnet reimplements things that already exist because it found one of the edge cases, but Opus had never known about it because Haiku didn't understand it.
It's possible that the new "agent teams" feature in Claude code can help with this. That keeps each agent alive with its context so they can ask each other things, but I haven't tried that enough to be sure - let alone with the specific model mix like this.
In your case, you are giving the Sonnet model specific instructions for what to implement mindlessly. I'd expect that to work well!
But that's not the same as the agentic workflow many other are using.
I appreciate the suggestion! But it isn't clear to me, from reading their marketing site, what they bring to the table from this perspective. Can you give me a more targeted pitch?
I haven't used them in a while so my info may be out of date, but they tended to track whatever models were the best and auto-use them for each task (eg, one for planning, subagent for a code search, other frontier for implementing). Their CLI seemed very well thought out to make you do things "the correct way" -- for instance, `/handoff` instead of `/clear`.
Sorry for the late answer and the missing context. usef- is right, the manual is probably the better page to share. Amp tries to give you a plug and play experience, where you can always see the actual costs and models/effort are autoselected for you. Some of my colleagues are big fans and use it a lot. I also like it, but prefer OpenCode.
I don't demand a customized compiler for my code even if such a compiler could outperform gcc. There is a lot of value in focusing on correctness to an extreme degree even if the outcome might be suboptimal to something more tailored - a tool with a large customer base can justify more resources going into its maintenance.
Ah I see! Yes, I was talking about a coding harness, not an enterprise agent. I entirely agree with you that your suggestion of driving it via evals is the right thing for that use case!
Same boat as you, and my answer is "... Except when I ask and overall or checkup task that is specifically heavy or overseeing in which case I use the maximum level" which lately meant ultracode.
I'm not going to play around with thinking level every request because the goal is to make me save time not spend it in a different setting menu.
I tend to run it on High and then step it up for problems where I'm noticing it struggles, bump it back down after. Sometimes I accidentally leave a session in Ultracode for a day and wonder why things are taking so long, but generally happy with the results.
Exactly this is my problem with all AI tools. I want someone else to create working tools for me so I can focus on my product. It is the same with other tools. I do not want to spent huge amounts of energy and time to setup my IDE, operating system or desk layout. I guess it is too early to have that now.
It's really not that much. It's a bit hard to make sense of it not because it's hard to keep track of, but because they are being deceptive and opaque about what you're actually buying, and the thing you're paying for is different from one day to the next, as they fuck around with the parameters to boost subjective performance during a launch, then quietly degrade the service to cut costs.
Just because it’s hard to keep track of doesn’t mean it’s not relevant.
Playing around with learning the differences is incredibly helpful to schedule on ones calendar weekly for an hour or two, while saving links throughout the week to try out.
Same advice as ever? We call it context engineering now, but prompt engineering still matters a lot. Most of the failures I run into are unspecified assumptions made by the model that derails the conversation, but usually updating the first prompt fixes it. Opus in my experience is a bit better about checking assumptions, while Sonnet will plow on ahead. An example is mentioning a file that doesn't exist: Sonnet will go ahead and try to grep your entire hard drive for it. Opus will say it's not local and request the path.
I trust neither for general knowledge and I still find Opus giving me answers that are completely BS. But the token spend for Q&A is nothing compared to coding, so I always use Opus + a lot of thinking. For coding, I find Opus to be better value/token but I haven't done any sort of rigorous test.
From today through June 22, Fable 5 is included on Pro, Max, Team, and seat-based Enterprise plans at no extra cost.
On June 23, we’ll remove Fable 5 from those plans. Using it after that will require usage credits. If capacity allows, we’ll extend the included window.
After this point—when sufficient capacity allows us to do so—we aim to restore Fable 5 as a standard part of subscription plans. We intend to do this as quickly as we can.
This seems like the pharmaceutical method of get them hooked on the drug with free samples, then once they can't live without it, raise the price. I'm not sure I want to start using Claude Fable on a max plan if it's just going to go away on June 23rd.
But maybe the more charitable reading is that they didn't have to offer this model at all on those plans and they are giving the standard free trial.
This is the entire business model of all AI companies. It costs far more to run the datacenters and build more capacity than they could ever hope to make back at current pricing models. I'm looking forward to pricing to catch up with reality and the resulting chaos that ensues.
Kind of how DeepSeek v4 dropped their pricing? I sense a shift which will hopefully bring lower and lower cost. Then again Qwen3.6 coding has been all I've needed for my projects and I'm perfectly fine with free.
Are you paying attention? These companies are trying to get market share without being anywhere close to making a profit - they are heavily subsidized. Many hundreds of billions have already been spent and will continue to be spent until the stupid fucking investors realize they will never get their money back. I have no doubt that day is coming.
The people actually working in this space will tell you that the cost of all of this continues to crater. Anthropic is already profitable minus its intentional forward-looking investments into R&D.
Serious investors look 10 to 20 years in the future. Everyone used google and youtube in 2006, but youtube wasn't profitable til 2016. How could a business burning money by hosting video ever be profitable? Costs come down BUT THEY JUST ADDED 720p, cost comes down, BUT THEY ADDED 1080p, cost comes down, 4k! cost comes down.
IMO the data from chats alone is worth $200B to Google.
Not really a comparison when the spend on YouTube was x10 smaller, and Googles core business has always been profitable beyond any hobby spending on YouTube.
I was just thinking this reminds me of the scene in The Wire where Avon admits to D'Angelo that the new heroin is in fact just the old heroin with different baby powder cutting it.
I was just saying last week: If Opus 4.8 max is as good as we get, and we plateau there, I think I'd be fine with it.
For the stuff I've thrown at it, that configuration has done a really great job. Including 70+KLOC go proxy with extensive test suite, some retro games, and more.
Or maybe its all about compute availability like they say. It could be that they plan to start training a new model on the 22ed, so the amount of compute available for inference will be greatly reduced.
Where is the Ferrari in this at all? I completely agree that they missed the mark in design. While the interior is 100% Jony Ive, the exterior screams "design by committee."
An electric Roma successor would have been much better received and possibly cheaper for them to develop (who knows?).
The silver lining in all this is that it means that the EV arm will not cannibalize their ICE cars.
The exterior screams asian-EV design langauge to me - which may not be an accident. Ferrari have made no secret of their hopes this car will succeed for them in China.
It looks like the EV version of Apple widgets and the iPhone home screen. There's so much rounded squares /rounded rectangle bullshit...it looks like something that was designed in 2010 and is about to get the shit sued out of it by Apple.
Every automaker is desperately trying to chase Chinese buyers. Most of them are too stupid to realize the Chinese can just....buy better Chinese EVs, and if they're not buying a chinese EV, it's because they don't want a Chinese EV, they want the foreign company's design and cachet.
Peopel don't buy Ferraris because they look like Chinese EVs. People buy them because they look like Ferraris and are exclusive.
Audi is doing stupid shit, too. They recently started making cars under the "AUDI" brand. Yeah. "AUDI". Versus "Audi" with rings.
If Ferrari wanted to sell more cars in China they could just stop be absurd dicks about a)who can buy their cars b)what people can do with them.
Things like "prohibit people from lending them to reviewers so Ferrari can game the review by putting on different tires and tuning the suspension for the specific track the reviewer will be using." Although might actually impress Chinese buyers since it aligns with them so well, culturally.
But that's not because Asian EVs have a specific identity, but because the Luce's design has NO identity. It has no heritage, like a sports car from a company that didn't exist 15 years ago.
At the moment I don't even see alot in it to BUILD a design-heritage upon, not many accents you could carry onwards to other cars.
The Hyundai Ioniq 5 is also an Asian EV. But it has character, it has accents, it has "rough edges". I can see aspects of it carrying onwards to the point that I see a van on the street and instantly know "it's a Ioniq". I don't see much of that in the Luce right now...
It's been a little while since I cared all that much about the models because they work well enough already. It's the tooling and the service around the model that affects my day-to-day more.
I would guess a lot of the enterprise customers would be willing to pay a larger subscription price (1.5x or 2x) if it means that they would have significantly higher stability and uptime. 5% more uptime would gain more trust than 5% more on a gamified model metrics.
Anthropic used to position itself as more of the enterprise option and still does, but their issues recently seems like they are watering down the experience to appease the $20 dollar customer rather than the $200 dollar one. As painful as it is personally, I'd expect that they'd get more benefit long term from raising prices and gaining trust than short term gaining customers seeking utility at a $20 dollar price point.
I am appreciative of your work on this piece. I'd love to see one that goes deeper into Dario Amodei. Perhaps even a series of profiles on the central figures of this AI era.
I somewhat understand Anthropic's position. However, thinking tokens are useful even if they don't show the internal logic of the LLM. I often realize I left out some instruction or clarification in my prompt while reading through the chain of reasoning. Overall, this makes the results more effective.
It's certainly getting frustrating having to remind it that I want all tests to pass even if it thinks it's not responsible for having broken some of them.
The stripe conference focused more than I would have liked on crypto.
I completely understand that there are markets and customers that can find real utility in it, but I wonder how many businesses will really ever benefit from stablecoins.
We're in higher education, and potentially our international clients could avoid hiccups with regulation, delays, compliance, and more using stablecoins, but it's really a guess. In the meantime, the pricing model of stripe seems to prioritize bigger and bigger clients.
That being said from Stripe's perspective stablecoins an easy bet to make. They win by building payment infrastructure within the traditional payment ecosystem and win by providing an alternative completely outside of it.
I think that's why I find his later works more compelling, particularly "Dependent Rational Animals," in which he grounds traditions in human telos. There are aims or goods which are common and transcend social constructs. After Virtue suffers from a great premise but doesn't quite stick the landing.
Exercising less dependence on a single company for everything seems like a good thing, but I'm struggling to really ground that principle against the convenience of just buying "what works." Does it really matter at the end of the day?