The iPhone 5s Review
by Anand Lal Shimpi on September 17, 2013 9:01 PM EST- Posted in
- Smartphones
- Apple
- Mobile
- iPhone
- iPhone 5S
CPU Performance
For our cross-platform CPU performance tests we turn to the usual collection of Javascript and HTML5 based browser tests. Most of our comparison targets here are smartphones with two exceptions: Intel's Bay Trail FFRD and Qualcomm's MSM8974 Snapdragon 800 MDP/T. Both of those platforms are test tablets, leveraging higher TDP silicon in a tablet form factor. The gap between the TDP of Apple's A7 and those two SoCs isn't huge, but there is a gap. I only include those platforms as a reference point. As you're about to see, the work that Apple has put into the A7 makes the iPhone 5s performance competitive with both. In many cases the A7 delivers better performance than one or both of them. A truly competitive A7 here also gives an early indication of the baseline to expect from the next-generation iPad.
We start with SunSpider's latest iteration, measuring the performance of the browser's js engine as well as the underlying hardware. It's possible to get good performance gains by exploiting advantages in both hardware and software here. As of late SunSpider has turned into a bit of a serious optimization target for all browser and hardware vendors, but it can be a good measure of an improving memory subsystem assuming the software doesn't get in the way of the hardware.
Bay Trail's performance crown lasted all of a week, and even less than that if you count when we actually ran this benchmark. The dual-core A7 is now the fastest SoC we've tested under SunSpider, even outpacing Qualcomm's Snapdragon 800 and ARM's Cortex A15. Apple doesn't quite hit the 2x increase in CPU performance here, but it's very close at a 75% perf increase compared to the iPhone 5. Update: Intel responded with a Bay Trail run under IE11, which comes in at 329.6 ms.
Next up is Kraken, a heavier js benchmark designed to stress more forward looking algorithms. Once again we run the risk of the benchmark becoming an optimization target, but in the case of Kraken I haven't seen too much attention paid to it. I hope it continues to fly under the radar as I've liked it as a benchmark thus far.
The A7 falls second only to Intel's Atom Z3770. Although I haven't yet published these results, the 5s performs very similarly to an Atom Z3740 - a more modestly clocked Bay Trail SKU from Intel. Given the relatively low CPU frequency I'm not at all surprised that the A7 can't compete with the fastest Bay Trail but instead is better matched for a middle of the road SKU. Either way, A7's performance here is downright amazing. Once again there's a performance advantage over Snapdragon 800 and Cortex A15, both running at much higher peak frequencies (and likely higher power levels too, although that's speculation until we can tear down an S800 platform and a 5s to compare).
Compared to the iPhone 5, the 5s shows up at over 2.3x the speed of last year's flagship.
Next up is Google's Octane benchmark, yet another js test but this time really used as a design target for Google's own V8 js engine. Devices that can run Chrome tend to do the best here, potentially putting the 5s at a disadvantage.
Bay Trail takes the lead here once again, but again I expect the Z3740 to be a closer match for the A7 in the 5s at least (it remains to be seen how high the iPad 5 version of Cyclone will be clocked). The performance advantage over the iPhone 5 is a staggering 92%, and obviously there are big gains over all of the competing ARM based CPU architectures. Apple is benefitting slightly from Mobile Safari being a 64-bit binary, however I don't know if it's actually getting any benefit other than access to increased register space.
Our final browser test is arguably the most interesting. Rather than focusing on js code snippets, Browsermark 2.0 attempts to be a more holistic browser benchmark. The result is much less peaky performance and a better view at the sort of moderate gains you'd see in actual usage.
There's a fair amount of clustering around 2500 with very little differentiation between a lot of the devices. The unique standouts are the Snapdragon 800 based G2 from LG, and of course the iPhone 5s. Here we see the most modest example of the A7's performance superiority at roughly 25% better than the iPhone 5. Not to understate the performance of the iPhone 5s, but depending on workload you'll see a wide range of performance improvements.
Facebook
Twitter
RSS
464 Comments
View All Comments
Dug - Wednesday, September 18, 2013 - link
"maybe you should hire a developer to write native cross platform benchmark tools"WHY? It is not going to make any difference. Developers aren't writing native cross platform programs. If they can take advantage of anything that's in the system, then show it off.
That would be like telling car manufacturers to redesign a hybrid to gas only to compare with all the other gas only cars.
ddriver - Wednesday, September 18, 2013 - link
"Developers aren't writing native cross platform programs"Maybe it is about time you crawl from under the rock you are living under... Any even remotely concerned with performance and efficiency application pretty much mandates it is a native application. It would be incredibly stupid to not do it, considering the "closest" to native language Java is like 2-3 times slower and users 10-20 times as much memory.
Dug - Wednesday, September 18, 2013 - link
Exactly my point! "native cross platform" Each cross-platform solution can only support a subset of the functionality included in each native platform.It doesn't get you anywhere to produce a native cross platform benchmark tool.
Again you have to mitigate to names and snide comments because you are wrong.
ddriver - Wednesday, September 18, 2013 - link
What you talk about is I/O, events and stuff like that. When it comes to pure number crunching the same code can execute perfectly well for every platform it is complied against. Actually, some modern frameworks go even further than that and provide ample abstractions. For example, the same GUI application can run on Windows, Linux, MacOS, iOS and Android, apart from a few other minor platforms.Anand Lal Shimpi - Wednesday, September 18, 2013 - link
Ultimately the benchmarking problem is being fixed, just not on the time scale that we want it to. I figured we'd be better off by now, and in many ways we are (WebXPRT, Browsermark are both steps in the right direction, we have more native tools under Android now) but part of the problem is there was a long period of uncertainty around what OSes would prevail. Now that question is finally being answered and we're seeing some real investment in benchmarks. Trust me, I tried to do a lot behind the scenes over the past 4 years (some of which Brian and I did recently) but this stuff takes time. I remember going through this in the early days of the PC industry too though, I know how it all ends - it'll just take a little time to get there.Actually I think 128-bit registers might've been optional on v7.
The only reason encryption results are in that table is because that's how Geekbench groups them. There's no nefarious purpose there (note that it's how we've always reported the Geekbench results, as they are reported in the test themselves).
In my experience with the 5s I haven't noticed any performance regressions compared to the 5/5c. I'm not saying they don't exist and I'll continue to hunt, it's just that they aren't there now. I believe I established the reasoning for why you'd want to do this early, and again we're talking about at most 12 months before they should start the move to 64-bit anyways. Apple tends to like its ISA transitions to be as quick and painless as possible, and moving early to ARMv8 makes a lot of sense in that light. Sure they are benefiting from the marketing benefits of having a feature that no one else does, but what company doesn't do that?
I don't believe the move to 64-bit with Cyclone was driven first and foremost by marketing. Keep in mind that this architecture was designed when a bunch of certain ex-AMDers were over there too...
Take care,
Anand
BrooksT - Wednesday, September 18, 2013 - link
Why would Anand write cross-platform benchmarks that have no connection to real world usage? Especially when you then complain that the 64 bit coverage isn't real world enough?ddriver - Wednesday, September 18, 2013 - link
For starters, putting the encryption results in their own graph, like every other review before that, and side to side comparison between geekbench ST/MT scores for A7 and competing v7 chips would be a good start toward a more objective and less biased article.And I know I am asking a lot, but an edit feature in the comment section is long overdue...
TheBretz - Wednesday, September 18, 2013 - link
For what it's worth this is NOT a case of LITERALLY comparing "Apples" and "Oranges" - it is a case of comparing "Apple" and many other manufacturers, but there was no fruit involved in the comparison, only smarthphones and tablets.ddriver - Wednesday, September 18, 2013 - link
Apples to oranges is a figure of speech, it has nothing to do with the company apple... It concerns comparing incomparable objects which is the case of completely different JS implementations on iOS and Android.Arbee - Wednesday, September 18, 2013 - link
Please name any case when AT's benchmarks and reviews have been proven to be biased or inaccurate. There's a reason the writers at other sites consider AT the gold standard for solid technical commentary (Engadget, Gizmodo, and the Verge all regularly credit AT on technical stories). As far as bias, have you *heard* Brian cooing about practically wanting to marry the Nexus 5? ;-)I think what actually happened here is that apparently Apple engineers listen to the AT podcast, because aside from 802.11ac and the screen size the 5S is designed almost perfectly to AT's well-known and often-stated specifications. It hits all of Anand's chip architecture geekery hot buttons in a way that Samsung's mashups of off-the-shelf parts never will, and they used Brian's exact line "Bigger pixels means better pictures" in the presentation. And naturally, if someone gives you what you want, you're likely to be happy with it. This is why people have Amazon gift lists ;-)
Krait's 128 bit SIMD definitely helps, but it won't match true v8 architecture designs. I've written commercially shipping ARM assembly, and there's a *lot* of cruft in the older ISA that v8 cleans right up. And it lets compilers generate *much* more favorable code. I'll be surprised if the next Snapdragons aren't at least 32-bit v8. Qualcomm has been pretty forward-looking aside from their refusal to cooperate with the open-source community (Freedreno FTW).
As far as 64 bit on less than 4 GB of RAM, it enables applications to more freely operate on files in NAND without taking up huge amounts of RAM (via mmap(), which the Linux kernel in Android of course also has). Apps like Loopy HD and MultiTrack DAW (not to mention Apple's own iMovie and GarageBand) will definitely be able to take advantage.