Shortly after that I discovered SimCity 2000. The first SimCity title, which was released back in 1989, was before my time so I never played or laid eyes on the original. At the time SimCity 2000 was incredible, it was extremely detailed and offered what seemed like endless hours of gameplay. Some five years later SimCity 3000 was released (1999) and again much of my childhood was spent playing it. For reasons that I cannot recall I never got into SimCity 4 (2003). I know I played it but for some reason it just didn’t speak to me like the previous two titles. Then along came SimCity Societies and at that point I thought my days of enjoying the SimCity games were over and for the better part of a decade they were. But when Maxis announced last year that a sixth installment in the SimCity franchise was coming the hair on the back of my neck stood on end. From the announcement, it looked to be a dramatic overhaul from previous titles featuring full 3d graphics, online multiplayer gameplay, a new engine as well as several new features and gameplay changes.
One year of waiting later, like so many others I pre-ordered the game and sat waiting for it to become available for download. Unfortunately like everyone else, once the game became available and I finally managed to download it, I wasn’t actually able to play. As you’ve probably heard for the past couple of weeks, the game requires an internet connection to play, meaning there is no offline mode. That in itself is extremely annoying but it’s much worse when the servers you are meant to play on cannot cope with demand and shut you out.
It took me several days of trying, as did the thousands of outraged fans. Since we planned to test SimCity I really needed to get in and work out how we were going to test the game. Thankfully by Sunday things improved and for the next three days I set about building our test environment.
Testing Methodology
Normally when we benchmark a first person shooter, finding a good portion of the game to test with is simply a matter of playing through the game until we find a section that is rather demanding. This generally requires an hour or two of gameplay and then we get to test in full. It’s a similar process when we test real-time strategy games such as StarCraft II, for example. In that instance we chose to play a 4v4 game, record the replay and use that for benchmarking. But with SimCity things were considerably more complex and time consuming. Because the game’s progress is stored on EA servers it’s not possible to just download and use someone else’s saved game of a massive city. While it is possible to load up the leaderboard within SimCity, see who has the biggest city, and check it out, we couldn’t use that for testing either since it’s a live city being played, thus forever changing and hardly a controlled-enough test environment.
There are a few pre-built cities, such as the one used in the tutorial “Summer Shoals” but with a population of less than 4000 it doesn’t exactly make for the most demanding test environment. Therefore we created a city that has a population of half a million sims with three more cities just like it on the map. When testing StarCraft II some readers were upset that we tested using a large 8-player map, claiming that they only play 1v1 and therefore get better performance. That is fine, but we wanted to show what it took to play the game in its most demanding state so that you’d never run into performance issues. Getting back to SimCity, it’s a slightly different situation as all the regions are the same size. Some maps have more regions than others, but they are all 2x2 kilometers (comparable to SimCity 4’s medium size). For testing we loaded one of our custom created cities (the same one each time) and increased the game speed to maximum, as this is how I always play anyway. Once that was done, we started a 60 second test using Fraps and in that time zoomed in and out multiple times while scrolling around the city. As usual we tested at three different resolutions: 1680x1050, 1920x1200 and 2560x1600. The game was tested using two quality configurations, which we are calling maximum and medium. Normally we would test three different quality settings, but there was virtually no difference between ‘max’ and ‘high’ so we scrapped the latter.
