Posts Tagged ‘caching’

iOS Location Caching Round-up – Conspiracy Theories: 0, Smart Location Caching: 1

More a meta post, or what Kuro5hin would have called MLP (meaningless link propagation), this post started out as a comment to one of my previous posts on the iOS location caching controversy but soon expanded way beyond a comment into a full blown post.

Firstly, let’s get the conspiracy theory out of the way; this theory has been presented in a variety of ways but all of them seem to think that your iOS device is tracking your location and that the reason for this is some shadowy request from government or intelligence agencies. Perhaps the most eloquent case for this was on Frank Reiger’s blog.

Now I love a good conspiracy theory as much as the next person and Frank’s blog post was a great read. But I have to take issue with the two main points he raises. Firstly there’s “if it was a bug then it would have been fixed … it hasn’t been fixed so it can’t be a bug and must therefore be deliberate“. Secondly there’s “not only has the bug not been fixed but the file even moved location without being fixed so it must be (even more) deliberate“.

Encyclopedia of Conspiracy Theories

I’ve worked in the software industry for almost 25 years, many of those cutting code, and can say with hand on heart that bugs, oddities and plain wrong behaviour stay in code bases not because they don’t need to be fixed but because other factors push them down in the priority list, factors such as hard release dates, new features taking precedence and the ill defined side effects of complex software systems not being able to be fully QA’d. Just because a bug or an unforeseen side effect remains in a production code base does not make a conspiracy theory of government or intelligence agency intervention.

We also live in a world of distributed software development teams. It’s enough of a challenge to keep teams in different floors of the same building in synch; it’s even more difficult when language, time zones and different countries get into the mix. Just because the consolidated.db cache moved location again, does not make a conspiracy theory.

So all in all, nice post, great conspiracy theory but, sadly, very little to back up the assertions.

But if your iOS device is tracking or caching your location, why is the data so inaccurate in places, showing places you’re pretty sure you haven’t been or have visited only fleetingly, yet not showing places you’d think would show up, such as where you live or work?

For the answer to these questions, I’d recommend a thorough reading of Peter Batty’s excellent three posts on the topic, which actually digs into the data that is present on iOS devices, rather than making shrill conspiracy theories based on other, equally shrill, media headlines.

Peter’s posts, “So actually, Apple isn’t recording your (accurate) iPhone location“, “More on Apple recording your iPhone location history” and “The scoop: Apple’s iPhone is NOT storing your accurate location and NOT storing history” go into great detail about what the consolidated.db location data cache does contain and, more importantly, what it doesn’t.

An anonymous comment on one of Peter’s posts points to a document submitted by Apple to US Congress in July 2010, which includes the following

When a customer requests current location information … Apple will retrieve known locations for nearby cell towers and Wi-Fi access points from its proprietary database and transmit the data back to the device … The device uses the information, along with GPS coordinates (if available), to determine its actual location. Information about the device’s location is not transmitted to Apple, Skyhook or Google. Nor is it transmitted to any third-party application provider, unless the customer expressly consents

Another comment from Jude on one of Peter’s posts makes this observation …

My Guess?

It’s not a list of cell phone locations that you’ve been to, but the opposite, a list of cell phone locations near you downloaded to the iPhone from Apple in case you move into range of one of them. i.e. At a guess what is happening is location services identifies a cell tower and asks for its location, and is replied to with the list of locations that contains that cell tower, that list is then cached so that it does not need to be requested again.

Of course, this is only a guess based on the wide range of addresses people are seeing and how its near to, but not exactly where, the people have traveled.

So rather than iOS actively and accurately tracking you and reporting this information to some, unspecified, intelligence agency it’s actually the complete opposite; your device is actively downloading the next cell tower and, in some cases, wifi information that is near you and where you might be going to provide a better location experience. Which explains the inaccuracy of the locations people have been seeing in their version of the cache data and explains why there’s some places they haven’t been showing up in the data and why places they have been aren’t showing up.

hat Fool Columbus Hasn't Got GPS

Of course, this information still has personal value and should really be secured by iOS and not by an individual having to secure their handset and encrypt their backups but if anyone still thinks they see the black helicopters circling, it looks more and more unlikely and, as Ed Parsons pointed out, a smartphone without location just isn’t … smart.

Photo Credits: Álvaro Ibáñez and Tom Jervis on Flickr.
Written at home (51.427051, -0.333344) and posted from the Nokia gate5 office in Schönhauser Allee, Berlin (52.5308072, 13.4108176)

Location’s “Ick Factor”; First iOS And Now Android

Two days ago I wrote about the “discovery” of a cache file on iOS devices that stores the position of cell towers and the associated media coverage surrounding this. Note that I use “discovery” in inverted commas here. As Sally Applin pointed out in a comment on my previous post, this “discovery” is not new and a paper on this by Alex Levinson, Bill Stackpole and Daryl Johnson was published in January 2011 as part of the Hawaii International Conference on System Sciences. Maybe sometimes researchers don’t read other, existing, research on a subject before publishing.

No matter where you go, there you are. - Buckaroo Bonzai

I’m not the only one to question the media coverage and the conclusions the media presents. Longtime Apple commentator and author Andy Ihnatko neatly sums the entire topic up thus

A few reality checks, lest I inadvertently do a Glenn Beck number on all of you, here:

  • This database isn’t storing GPS data. It’s just making a rough location fix based on nearby cell towers. The database can’t reveal where you were…only that you were in a certain vicinity. Sometimes it’s miles and miles off. This implies that the logfile’s purpose is to track the performance of the phone and the network, and not the movements of the user.
  • A third party couldn’t get access to this file without physical access to your computer or your iPhone. Not unless you’ve jailbroken your iPhone and didn’t bother resetting its remote-access password…or there’s an unpatched exploit that would give Random Person On The Internet root access to your phone.
  • It’s pretty much a non-issue if you’ve clicked the “Encrypt iPhone Backup” option in iTunes. Even with physical access to your desktop, a no-goodnik wouldn’t be able to access the logfile.

But still! What a nervous can of worms. This is an open, unlocked file in a known location in a standard database format that anybody can read. If someone has physical access to your Mac — or remote access to your user account — it’s a simple matter of copying a file and opening it. And while the logfile can’t tell someone that you were at a specific house, it can obviously tell your boss that you went to the Cape on the day you called in sick.

And it’s not as though Apple and these two developers are the only people who know that this file exists and that it’s so easy to access. By the time the Good Guys blow the whistle, the Bad Guys have had it for months. Lord only knows what they’ve been doing with this information.

It’s also, frankly, another reason why I value my iPhone’s “remote nuke” feature and wish it were possible to nuke all data directly from the handset. I can’t think of any circumstance under which my location data would possibly be damaging, incriminating, or even just embarrassing. That’s not the point: if I can’t control the data that my phone is collecting, I should at least have the power to destroy it utterly.

Another well known Apple commentator, John Gruber also comments that

The big question, of course, is why Apple is storing this information. I don’t have a definitive answer, but the best at least somewhat-informed theory I’ve heard is that consolidated.db acts as a cache for location data, and that historical data should be getting culled but isn’t, either due to a bug or, more likely, an oversight. I.e. someone wrote the code to cache location data but never wrote code to cull non-recent entries from the cache, so that a database that’s meant to serve as a cache of your recent location data is instead a persistent log of your location history. I’d wager this gets fixed in the next iOS update.

In my previous post I wrote that “caching is a common technique used to speed up network systems and it’s not surprising, at least to me, that iOS is using caching techniques” and it turns out that iOS is not alone and that, unsurprisingly, Google’s Android is doing pretty much the same thing, caching cell tower and wifi location information, again presumably for the purposes of speeding up the location systems on Android mobile devices. The one difference between the iOS and Android approach is that Android overwrites the cache data when the cache file fills up whereas iOS doesn’t. Rather than a dark location tracking conspiracy this looks more like a bug or an oversight on the part of iOS and as John Gruber notes, this will probably be fixed in an upcoming release of Apple’s mobile operating system. I would also hope that the visibility of this cache data on Android will also be secured too, and soon.

Taking a step back for a moment, caching of any information to reduce the need to make time costly network calls seems to be mobile’s Kobayashi Maru … you’re damned if you do and damned if you don’t. If you do cache information which is perceived, rightly or wrongly, to be sensitive then media outrage will result. If you don’t cache such information, then a mobile device will be reliant on network access every time the un-cached information is needed and that mobile device will be perceived as being “too slow“.

Probably the only way to prevent a recurrence of this sort of media event is for more transparency on how such information is being stored and used and, as John Abbott pointed out on my previous post, the provision of a setting which says “Switch this setting on for a super-quick location fix, we’ll keep your location private“.


As is so often the case, this is much less about the technical side of the issue and much more about what Ihnatko calls the “Ick Factor” … about how the public, led by the media, sees things.

Photo Credits: Stefan Andrej Shambora and Trevin Chow on Flickr.
Written and posted from home (51.427051, -0.333344)

iOS Location “Tracking”; Gross Invasion Of Privacy Or Media Sensationalism?

Oh dear. For a few years now I’ve been talking about how the privacy aspect of today’s location technologies is something that may just catapult location into the mainstream, and possibly tabloid, media and probably for the wrong reasons. I envisaged this as being something salacious and potentially titillating, such as two Z List celebrities involved in a high profile divorce case, where they claimed to be in two separate places but their phone’s A-GPS showed the complete opposite. If you were at Where 2.0 in San Jose this week or reading the headlines on the web sites of the BBC, The Guardian or BoingBoing, you’d be forgiven for thinking that just such a location media event had happened. But has it? The headlines certainly seem to think so …

iPhone tracks users’ movements … says the BBC

iPhone keeps record of everywhere you go … says the Guardian

Got an iPhone or 3G iPad? Apple is recording your moves …. says O’Reilly Radar

iOS devices secretly log and retain record of every place you go … says BoingBoing

… and when I use the word “says” in reality “screams” would be more accurate.

Privacy Area

But as is often the case, the headlines don’t tell the whole story. At Where 2.0 two security researchers have discovered a cache file in iOS which contains cell tower ids and corresponding longitude and latitude coordinates. This cache file isn’t accessible if your iOS device has a passcode lock enabled, which it should be, and while it is backed up to any computer you synch your iOS device with, if your backups are encrypted, which they should be, this cache file isn’t accessible is anyone, especially not “a jealous spouse or private detective” as the researchers claim.

So why is your iOS device caching your cell tower ids and their locations? A reasonable supposition would be to speed up the A-GPS subsystem in your device, so that time consuming network calls don’t always need to be made and so your iOS device seems to be faster. Caching is a common technique used to speed up network systems and it’s not surprising, at least to me, that iOS is using caching techniques.

Massachusetts Ave with iPhone Google Maps

So if you’re an iOS user, should you be worried? It’s true that the iTunes terms of service does say that “We may collect information such as occupation, language, zip code, area code, unique device identifier, location, and the time zone where an Apple product is used so that we can better understand customer behavior and improve our products, services, and advertising” but there’s currently no evidence that location information is actually being sent to Apple as a result of this caching. That’s not to say that it isn’t or that it won’t be in the future, but for now, it looks unlikely. Take some basic security precautions such as a phone passcode lock and encrypt your synchronised backups and this looks less like a gross invasion of privacy and much more like any other use of caching techniques.

Cell Tower Mast

I think it’s right and good that researchers are probing into the the depths of a popular mobile operating system and looking for vulnerabilities. Your location, regardless of whether it’s your current position or where you’ve been is valuable and above all private information and is ripe for abuse as last year’s news on how free Android apps are sharing people’s location without their knowledge shows. But I take issue with the conclusions drawn from such research as this and how it’s being publicised. To put this in context, consider the following, totally imaginary on my part, media headlines about caching …

Your web browser records every web page you visit!

Well yes, but … without this your browser would far less usable and far slower and caching is a fundamental part of how the web works.

Your Internet Service Provider stores copies of every email you send and receive!

Well yes, but … it’s part of the IMAP protocol that most email accounts use today.

Your mobile phone provider tracks your mobile phone’s location!

Well yes, but … it’s the way that cellular networks work. It’s how you can make and receive calls. Disable this and mobile networks stop working.

Finally I’m reminded of the other, media fueled furores, that have appeared and subsequently disappeared, around the launch of Google’s Latitude and Facebook’s Places. Much media coverage ensued, many sensationalistic headlines, much wringing of hands from industry pundits and then the rest of the world got on with using location technologies and didn’t give it a second thought … until the next time the media wants some headline attentions.

Photo Credits: Mark Barkaway, Steve Garfield and Happychopper on Flickr.
Written and posted from home (51.427051, -0.333344)