Our smartphone benchmarks are centered around browser-based tests, and this is one area that really shows how far behind Internet Explorer 10 Mobile is relative to the mobile versions of Safari and Chrome. The other part of this is looking at Krait 200 at 1GHz versus the higher-clocked parts we see in the more expensive Windows Phones, which allows us to look at relative performance on the same software platform.

SunSpider Javascript Benchmark 0.9.1—Stock Browser

Google Octane Benchmark v1

Mozilla Kraken Benchmark Sunspider performance is actually quite good, especially compared to other Krait 200-based devices like the One X and AT&T Galaxy S3. Where it falls off is in everything else—Kraken and Octane performance is just horrendous in comparison to the modern smartphone platforms. I didn’t think it was actually possible to see numbers that high in the Kraken benchmark until I ran it a half-dozen times on my 521 review unit as well as a friend’s Lumia 920. Microsoft clearly optimized for Sunspider, as we’ve seen over the years, and while that lets them stay competitive in that one benchmark, it doesn’t really mean anything as far as having decent or even acceptable browser performance. It’s just sad.

Between the 920 (MSM8960, so dual-core Krait 200 @ 1.5GHz and Adreno 225) and the 521, there’s a definite performance delta. Obviously, this comes through in roughly 50% faster benchmark numbers across the board, but in day to day use, it mostly makes itself felt in slower application loading and multitasking/task switching, which is also related to having just 512MB RAM instead of the 1GB of the 920. Multitasking in Windows Phone has never really sat well with me, because it’s not really multitasking in the traditional sense—the need to completely close and rehydrate tasks makes resuming applications exceedingly slow. Microsoft has always seemed to have an issue with app load times with ARM devices—this has been true since day one with Windows Phone, and is a problem with Windows RT tablets too—so the resulting combination can really kill any desire to run more than one task on your phone at a time.

I couldn’t run GFXBench (DXBenchmark) on the 521 because it didn’t meet the 1GB RAM requirement necessary to run the benchmark. I actually wasn’t even aware that GFXBench had such a requirement in place until I tried to install it on the 521. In the grand scheme of things, I suppose it doesn’t matter too much—we’re pretty familiar with Adreno 305 by now, and it seems to be the go-to GPU for Qualcomm’s new series of dual-core Krait 200 SoCs. In a platform that deemphasizes silicon performance in the end user experience as much as Windows Phone seems to, I’m not sure that the performance delta between this and Adreno 225 makes too much of a difference, particularly with a far lower-res screen. We’re starting to see the more 3D intensive games require 1GB of memory to run, so this isn’t the device for the hardcore gamers. It does fine with the typical slate of casual games though—Fruit Ninja, Angry Birds, and the like.

The air interface is probably something you could point to and say is a bit lacking—in this day and age, dual carrier HSPA+ is almost an expectation on T-Mobile and international non-LTE devices. But honestly, you don’t end up missing it much. T-Mobile’s HSPA+ network is quite robust both in Seattle as well as in Silicon Valley, the two places I spent time testing the 521, so 10+ Mb/s was basically the norm.

I couldn’t figure out how to get the Windows Phone SpeedTest app to email a CSV of the test results, so no graphs with binned speed tests like we have on Android reviews. We’ve had a lot of experience with T-Mobile’s HSPA+ network over the years though, and it tends to be very good in most metropolitan areas. Obviously, there’s a significant jump in speed between HSPA+ and LTE, but the real-world difference between single and dual-carrier HSPA+, at least on T-Mobile, seems to be much harder to discern perceptively. The lack of DC-HSPA+ is still a bit of a disappointment, but I’m not nearly as cut up over it as I was when I saw it on the specsheet.

Battery Life The IPS Display
POST A COMMENT

116 Comments

View All Comments

  • notposting - Thursday, August 8, 2013 - link

    Are you guys planning to do a review of the 1020 (well, basically the camera)?

    I switched to WP7 in 2011, fwiw, and haven't looked back. Just went to WP8 this summer with my Lumia 928. Very happy with it.
    Reply
  • Crono - Thursday, August 8, 2013 - link

    According the podcast (can't remember if it was the AT one or from Vertex on iMore) Brian Klug is working on it. Reply
  • PHlipMoD3 - Thursday, August 8, 2013 - link

    Still waiting on that Lumia 920 review... Suppose it wont happen now with the 1020 release - albeit a very similar device internally... Reply
  • Myrandex - Thursday, August 8, 2013 - link

    I've waited for this too. I've really enjoyed my 920 and my wife enjoys hers as well. And speaking of price, it is nice that it is free with contract at Amazon for those of us who don't mind an AT&T Contract. Reply
  • zepi - Thursday, August 8, 2013 - link

    Instant messaging? What is actually missing there? I thought Whatsapp and Skype are both available? Problems with Google talk not working in the background properly etc? Reply
  • JPForums - Thursday, August 8, 2013 - link

    A little elaboration on the Instant Messaging issue would be helpful. Perhaps it doesn't have the IM you want or maybe there is some killer feature in an IM you like that isn't available on any of the WP8 IMs.

    Also, I'm a little confused as to the problem with multitasking. IIRC you can pull up Skype and still have it receive messages when you are doing something else as long as you don't shut it down. Didn't seem like it took very long to resume or load tasks to me, though it was a higher end WP8 and an HTC One is still perceptibly, though not significantly faster at loading in most situations.

    Your concerns on minimalism are quite valid, though I think many people have less issue than you might think. In my experience, people who have used WP for more than a few minutes tend to line up with Crono's classification. They find it easier to use than Apple. For many people the genuinely useful features and/or information that is hidden by the WP minimalist interface is still a distraction that gets in their way.

    Note: though I disagree/don't understand some of your points, I still think this was an excellent write up and hope to see more. Non-flagship devices are seriously underrepresented. Thanks Vivek.
    Reply
  • VivekGowri - Thursday, August 8, 2013 - link

    I don't know, I rely on IMs as part of my workflow and I've literally never missed so many messages in my life as the time I spent on Windows Phone 8. Happened late last year with the 810, happened now with the 521. IM+ is garbage and if that's the best multiprotocol IM client on your platform, your platform will be woefully inadequate for IM. That's it. I'm not looking for a killer feature, I'm literally just looking for working IM.

    Multitasking in WP involves completely closing and rehydrating the apps, it's super slow and not multitasking at all. iOS 4/5/6 (7 is real multitasking) does something similar but doesn't need to completely rehydrate each time I think, it's much more seamless than on WP. This is all exaggerated by the still-slow WP app load times. It's been 3 years, I don't know why this is still an issue.

    Windows Phone is frustrating for me, because I've been a fan of the platform for a very long time, I really want it to succeed as a third major smartphone OS and not just because it's the default option. It's been sad to watch it stagnate on so many levels since launch. Really, that's all I can say.
    Reply
  • Myrandex - Thursday, August 8, 2013 - link

    There are apps that you can switch quickly between without a complete refresh. I do it all the time and it is called Fast App Switching I believe.

    The only IMs I use on there are facebook messaging as I kind of got out of IMing when I started txting all of the time (nice universal standard), so I haven't actually used IM+ but knew of its existence. FB chat on the phone is quite nice, and hopefully Skype Instant Messaging integrates as nicely someday (what a pathetic joke that MS has purchased Skype so long ago but failed to really take advantage of integrating it into the OS yet like Facetime on iOS).
    Reply
  • notposting - Thursday, August 8, 2013 - link

    I've had good success with jiTalk (Gchat client).

    As for multitasking...are you doing it right? (serious question, not trolling) If you back out of the apps and/or if you launch them from the Start screen/app list, it reloads them. Otherwise if you are just hopping between apps with the long press on the back button, they come up with no delay and at the same state they were, in general (as Myrandex mentions). I can pop between a web site, a search, maybe a photo and a photo editing app, and email with no problems.
    Reply
  • cheshirster - Friday, August 9, 2013 - link

    So it all comes dows to poor IM+ for WP application?
    It was last updated on April and it is really week in terms of quality (large time to connect on 2g, crashes while starting). I don't use it.
    And what you say about multitasking is strange too, most core apps support fast resuming (the last to add it was Skype on 7.7.2013)
    Reply

Log in

Don't have an account? Sign up now