I'm going to make a prediction here: Including memory on a multi-chip module/building systems around an MCM SoC is going to become the standard for a large fraction of the personal computer market.
The performance gain from doing so in CPU<->core memory and core memory<->GPU transfers is huge, and the manufacturer can match RAM timing and performance precisely to their processor or even implement non standard ram types as they like. There are other benefits too like simplified motherboard design.
Now that Apple has taken the risk, other manufacturers will look at doing the same. Not all computers will use the SoC model, but for laptops and many desktops this will be a big win.
I don’t know how this “memory on the SOC” thing became a narrative. That isn’t responsible for any of the M1’s performance. Apple isn’t using anything fancy like HMC or HBM that can’t be done with off-package memory. It’s a regular 128 bit memory bus and standard LPDDR4x (at the top standard frequency), with slightly higher memory latency than Intel and AMD. Pretty much any HEDT x86 system has a more impressive memory system.
>I don’t know how this “memory on the SOC” thing became a narrative.
Yes it has been spreading like plague. And I had to post something similar [1] not long ago. And many more before that.
The M1 could have an Off package Quad Channel DDR4 Memory and still be as fast. The performance improvement ( from a memory perspective ) is coming from Same Memory Address Space and other similar optimisation.
Going off package will at the very least increase power dissipation. Exiting a package, going across a PCB, and entering additional packages will increase capacitance significantly as well as increase resistance and inductance. This will impact performance. If the increased capacitance does not change actual operating speed, then the buffers are supplying more current to overcome the capacitance, not to mention the potential ringing and other undesired effects from the additional parasitics. There is a penalty for going off the SOC. It is not just a narrative, it is physics.
The same memory address space choice is of course important, but its performance and power envelope is impacted by the SOC vs. separate package choice.
The combination of M1 performance and low power has happened due to a series of choices made by Apple. Forgoing user configurability and fixing memory choices at manufacture while using SOC tech made mainstream by the phone industry is one of those impactful choices. There are of course several other important choices, but it is incorrect to discard this choice as non-impactful.
If I understood the parent comment, isn't it more about shared CPU/GPU memory? This is something different which M1 has in common with game consoles and smartphones but not traditional PC's, isn't it?
no it's not the same. "Shared" means different things.
In M1, the GPU reads directly from memory written by the CPU.
In Intel/AMD, the data has to be copied from the CPU's address space to the GPU's. "Shared" only means there aren't separate main and graphics memory chips/banks. But said shared memory is segregated.
Uhm, as a games developer working on consoles.....no it isn't. You can do it like this if you wish, but generally the entire address space is accessible from either CPU or GPU. Maybe it's implemented like this on PC, but at least the architecture design on X1/XBS/PS4/PS5 allows both reads and writes from any area of memory by either cpu or gpu.
On pc by default the memory for the iGPU is a dedicated segment of RAM. There are probably tricks to read from each other RAM but not integrated like Apple has done.
The i9’s slow supported memory is a product of that being Intel’s 14nm line which uses the same architecture released in 2015. The 10nm core chips use LPDDR4x, just like the M1. The $999 Surface Laptop 3 uses it at 3733 MT/s, just a bit slower than the M1: https://www.anandtech.com/show/14933/microsoft-announces-sur.... That speed of memory is supported on the lowest end 10nm mobile i3, which appears in sub-$300 NUCs available since 2019.
The M1 is a great chip, but that has nothing to do with the location of the memory. LPDDR4x-4267 is a standard memory type. Kudos to Apple for using the highest commonly available speed bin, but it’s a standard speed bin for that type of memory.
I'd say that having massive low latency caches on die plays a larger role.
>On the cache hierarchy side of things, we’ve known for a long time that Apple’s designs are monstrous, and the A14 Firestorm cores continue this trend. Last year we had speculated that the A13 had 128KB L1 Instruction cache, similar to the 128KB L1 Data cache for which we can test for, however following Darwin kernel source dumps Apple has confirmed that it’s actually a massive 192KB instruction cache.
That’s absolutely enormous and is 3x larger than the competing Arm designs, and 6x larger than current x86 designs, which yet again might explain why Apple does extremely well in very high instruction pressure workloads, such as the popular JavaScript benchmarks.
The huge caches also appear to be extremely fast – the L1D lands in at a 3-cycle load-use latency. AMD has a 32KB 4-cycle cache, whilst Intel’s latest Sunny Cove saw a regression to 5 cycles when they grew the size to 48KB.
That cache is that big because the decode and ROB are so wide. If AMD or Intel's current designs widened the L1, it wouldn't make a difference. In fact, AMD reduced L1 cache size from 64k to 32k from Zen 1 to Zen 3.
x86 needs to find a way to scale decoders without blowing the power budget. Given that the decoders are already bigger than the integer units, I suspect that will be a hard thing to do.
I get crashes with XMP, usually right at worst possible time on a zoom call where I am presenting! And saying users should overclock just seems weird, apple works out of the box.
For whatever reason, the overall memory system on the M1 systems just seems better than intel. I really wish I could follow more details from on-die cache to how memory is actually loaded / unloaded to speeds, but every time I've looked at it a little it just seems the M1 / Apple are doing it better across the whole stack.
This may have changed since then (Mid-2020). Gamers Nexus have done an undercover sting where they found it was possible for a support agent to reject a warranty request on the basis of XMP: https://www.youtube.com/watch?v=I2gQ_bOnDx8&t=1155
Performance per Watt. Reducing the trace length makes it possible to get high frequency RAM working with acceptable power consumption. How many other ultrabooks clock their RAM at 4266MHz?
That's some weird semantics.
It is very much soldered separately, they are separate components soldered next to each other [1].
(Yeah, on a own sub-PCB, but still)
If that's not "soldered separately", then we might as well zoom out and apply that statement for the whole PC ("all components are in the same package, only the charger is separate")
Why not both? A little DRAM on SoC and the rest of the DRAM on the motherboard. Kernel in charge of "swapping". Maintains expandability while keeping most of the performance benefit.
How? You're basically describing RAM caching. Putting all the ram physically close to the processor gives a giant performance gain that's mostly lost if any of the system's RAM is "remote" on the motherboard.
Is there any concrete numbers to back the claim that "Putting all the ram physically close to the processor gives a giant performance gain"? Will be interesting to see what is the memory latency of M1 compared with a regular Intel/AMD processor.
It doesn't. As has been corrected time and time again.
The M1 has pretty high memory latency at around 100 ns [1], which is significantly higher than either AMD or Intel for typical systems. Note that physical distance between CPU and memory is rather less important for latency, as DRAM is high latency in itself, so adding a few ns at most due to wiring is not going to matter.
Just for context, the M1's latency is fairly close to other LPDDR4(x) systems (Tigerlake and Zen3 SoCs).
This is fairly typical of LPDDR4 compared to normal DDR4, part of the compromise.
Not the most scientific, but userbenchmark is useful because it has latency graphs available for millions of systems.
Yes. Thank You. If M1 discussion continues to be like this we have a possibly of stamping out M1 misinformation on HN.
But sometimes we are just lazy to provide the context or to spell out everything. These information is so readily available with a simple Google. And yet the past dozens of M1 thread this "memory" advantage thing keeps popping up.
The performance gain from doing so in CPU<->core memory and core memory<->GPU transfers is huge, and the manufacturer can match RAM timing and performance precisely to their processor or even implement non standard ram types as they like. There are other benefits too like simplified motherboard design.
Now that Apple has taken the risk, other manufacturers will look at doing the same. Not all computers will use the SoC model, but for laptops and many desktops this will be a big win.