Probably worth not getting too comfortable. It's only a matter of time before a new generation of product people can do this. Also, I have said this before, instead of 1 product person and 10 engineers on a team, you may have 2-3 product+eng people and 3 engineers, etc.
You have it mixed up. This guy is “the new generation of product people”. The old generation had to be technical enough to be able to grok the systems that they ‘producted’ over. It’s exactly the ‘new generation’ that LLM themselves right into Dunning-Kruger.
> The "let's make money by selling/renting out TPUs" faction has won and the "let's make money by training and selling a frontier model" faction has lost.
It's a variation of opportunity cost. A company that has an opportunity to take $1 and make $1.50 on it can't justify an opportunity to spend $1 and make $1.25, even though a less profitable company may make a good living on that. When considering capital allocation, Google has to consider the opportunity cost of investing more into their highly lucrative ads business. Another company that has no access to such a lucrative business uses different opportunity cost when it comes to allocating capital. It can easily be the case that Google could end up justify being in the business of renting out shovels and end up chased out of the business of using the shovels to create AIs entirely because that turns out not to be where the money is. I'm not saying that's obviously inevitable; I'm saying it's a possible and reasonable outcome.
That's why even though the industry produces giants, these giants can never just eat everything. Even though it seems like they have all the money, it isn't practical for them to try to do everything and in fact limits get hit very quickly for anything other than the primary, lucrative business.
Apparently there is no snappy term for this in the business space, according to such AI searches as I have run.
Of course the three short paragraph description of a general principle posted to Hacker News to give a partial answer to someone's 8-word question is simplified. What's the alternative? HN's text box is too short to contain a 4-year business course.
With all due respect, did you read my comment beyond the first paragraph? It addresses both points, TPU economics/pivot to sales + internal shortages making it hard to train models, to the extent they can be addressed based on public sources.
There are other factors at play, but they're more recent/second-order.
> AMD, in partnership with Intel and the x86 EAG (Ecosystem Advisory
Group) [EAG24], is readying ACE as the standard matrix acceleration architecture for x86, further enhancing the already
vibrant x86 ecosystem.
Please define new. Also, I think AMD uses very similar cores in server and client. So, disabling AVX512 may be an Intel thing (my guess is that so they can easily move threads between E & P cores).
They didn't disable it at first on their client CPUs, and it resulted in code randomly crashing depending on whether ifunc resolvers first ran on a big core or a little core.
It's pretty surprising that multiple CPU vendors have run into issues like this (some more than once, fucking Samsung), when it's pretty much the first thing that anyone on the toolchain side of thing asks when they hear about heterogenous cores on a CPU.
It would be possible, but Microsoft has already accumulated many decades of being extremely reluctant to make even the slightest changes in Windows in order to accommodate new CPU features.
Since 1990 and until now, the Intel-AMD ISA and the hardware architecture of the Intel-AMD CPUs have accumulated a large number of very undesirable features, which increase the cost of hardware and reduce the security of computers, for which the only justification was the fact that Microsoft refused to update Windows so they expected that the hardware vendors must do whatever it takes to keep their CPUs compatible with the legacy operating systems.
These workarounds have moved inside the CPUs many functions that properly belong to the operating system, like remote management and voltage-frequency control.
And do you know about Linux? After cursory search I couldn't find anything as well. In this case it seems very expensive, if not impossible without the huge penalty, to implement such a workaround in the hardware.
Heterogeneous compute aware kernel scheduler seems like the most natural approach to me.
The kernel can't do anything here, unless your answer is "threads can't ever migrate between types of cores, so cross your fingers on which core you spawn on". The kernel doesn't know what flavor of ISA userspace is expecting, because userspace doesn't know what flavor it wants until it actually gets to run: symbol multiversioning lets you compile a function for multiple versions of an instruction set and pick the best one at runtime.
You could imagine the kernel being clever and handling the first SIGILL in hardware and then marking the process as big-core-only, but you can't handle this generally since you can have new instructions get encoded as nop (I believe the AArch64 PAC instructions do that?). You could also have an ifunc resolver intentionally call the instruction and expect and handle the signal.
The solution is to make your hardware actually work, and not claim to support instructions that do not actually exist on all of your cores.
When an invalid instruction exception occurs, it is very easy to identify whether the invalid instruction was an AVX-512 instruction, because those start with a prefix different from any other instructions.
After that happens, the thread can be marked as requiring AVX-512 and then it can be scheduled only on P-cores.
Moreover, one can define a new flag for the ELF header of executable files, specifying the AVX-512 requirement, which will be set by the compiler, depending on the compilation options, and then the OS knows from the beginning how to schedule a process.
Of course, the best solution is still to implement a uniform ISA. If a lower performance is accepted, the execution of the AVX-512 instructions can be implemented at a negligible additional cost over the previous x86_64 ISA.
What has scared the E-cores designers into not implementing AVX-512 was not the execution part, but the instruction decoding part. The instruction decoder for x86_64 is already horrendously complicated when many instructions must be decoded concurrently in a single clock cycle and adding the need to handle the completely distinct encoding of the AVX-512 makes it even more complicated.
Nonetheless, the competition from AMD has eventually forced Intel to give up in their attempt of reducing production costs by saving die area with a simpler instruction decoder in the E-cores, so the E-cores of Nova Lake will have the same ISA as its P-cores.
And what happens when E-core decoder encounters the AVX-512 instruction set? Does this mean that E-cores will have to implement some sort of (downsized) vectorized execution engine too? Otherwise, I don't see how this would work. Migrating on the HW core level would be too complicated.
> When an invalid instruction exception occurs, it is very easy to identify whether the invalid instruction was an AVX-512 instruction, because those start with a prefix different from any other instructions.
You're missing the point: the processes executing AVX-512 do not have any particularly strong requirement for AVX-512 instructions, they're just optimistically using them to improve performance because CPUID claimed they were supported. It would be pathologically bad behavior to control scheduling behavior with this, because in practice it would result in things like glibc deciding to use AVX-512 for string functions resulting in every program that uses memcpy (i.e. every program) not being allowed to be scheduled on the little cores.
> if the virus and its DNA are undetectable then you can't spread it
The devil may be in the details. E.g. if a COVID test shows negative, it doesn't mean that you can't spread it. This is partly because different tests have different sensitivities.
> I'm pretty sure
FYI, without citations, it is hard to distinguish credible experts vs people on the internet saying "trust me bro".
Out of curiosity: what is your input to the model? A CAD file or a drawing?
I find it does a good job at isometric views from floor plans. However, I needed Gemini 3.1 Pro to be able to have a chance at rendering 3D human point of view images from floor plans.
Here is a repro using GCE's C4A Axion instances (c4A-highcpu-72). Seems to beat Graviton? Maybe the title of the thread can be updated to a larger number :) ? I used the largest instance to avoid noisy neighbor issues.
Oh nice! Axion C4A and Graviton4 use the same core (Neoverse V2), so the performance difference is due to factors like clock speed and power management.
I used a geometric mean to calculate the top-line "86 GB/s" for NEON pack/unpack; so that's 91 GB/s for the C4A repro. Probably going to leave the title unmodified.
Aren't we way way past that already? QPS to any of the frontier models for a given point in time is most likely (far) greater than zero.
reply