Posts about theory

iOS Location Caching Round-up - Conspiracy Theories: 0, Smart Location Caching: 1

2011-04-26 13:21

meta post, or what Kuro5hin would have called MLP (meaningless link propagation), this post started out as a comment to one of my previous posts on the iOS location caching controversy but soon expanded way beyond a comment into a full blown post.

Firstly, let's get the conspiracy theory out of the way; this theory has been presented in a variety of ways but all of them seem to think that your iOS device is tracking your location and that the reason for this is some shadowy request from government or intelligence agencies. Perhaps the most eloquent case for this was on Frank Reiger's blog.

Now I love a good conspiracy theory as much as the next person and Frank’s blog post was a great read. But I have to take issue with the two main points he raises. Firstly there’s “if it was a bug then it would have been fixed … it hasn’t been fixed so it can’t be a bug and must therefore be deliberate“. Secondly there’s “not only has the bug not been fixed but the file even moved location without being fixed so it must be (even more) deliberate“.

More a meta post, or what Kuro5hin would have called MLP (meaningless link propagation), this post started out as a comment to one of my previous posts on the iOS location caching controversy but soon expanded way beyond a comment into a full blown post.

I’ve worked in the software industry for almost 25 years, many of those cutting code, and can say with hand on heart that bugs, oddities and plain wrong behaviour stay in code bases not because they don’t need to be fixed but because other factors push them down in the priority list, factors such as hard release dates, new features taking precedence and the ill defined side effects of complex software systems not being able to be fully QA’d. Just because a bug or an unforeseen side effect remains in a production code base does not make a conspiracy theory of government or intelligence agency intervention.

We also live in a world of distributed software development teams. It’s enough of a challenge to keep teams in different floors of the same building in synch; it’s even more difficult when language, time zones and different countries get into the mix. Just because the consolidated.db cache moved location again, does not make a conspiracy theory.

So all in all, nice post, great conspiracy theory but, sadly, very little to back up the assertions.

But if your iOS device is tracking or caching your location, why is the data so inaccurate in places, showing places you're pretty sure you haven't been or have visited only fleetingly, yet not showing places you'd think would show up, such as where you live or work?

For the answer to these questions, I'd recommend a thorough reading of Peter Batty's excellent three posts on the topic, which actually digs into the data that is present on iOS devices, rather than making shrill conspiracy theories based on other, equally shrill, media headlines.

Peter's posts, "So actually, Apple isn't recording your (accurate) iPhone location", "More on Apple recording your iPhone location history" and "The scoop: Apple's iPhone is NOT storing your accurate location and NOT storing history" go into great detail about what the consolidated.db location data cache does contain and, more importantly, what it doesn't.

An anonymous comment on one of Peter's posts points to a document submitted by Apple to US Congress in July 2010, which includes the following

When a customer requests current location information ... Apple will retrieve known locations for nearby cell towers and Wi-Fi access points from its proprietary database and transmit the data back to the device ... The device uses the information, along with GPS coordinates (if available), to determine its actual location. Information about the device's location is not transmitted to Apple, Skyhook or Google. Nor is it transmitted to any third-party application provider, unless the customer expressly consents

Another comment from Jude on one of Peter's posts makes this observation ...

My Guess?

It's not a list of cell phone locations that you've been to, but the opposite, a list of cell phone locations near you downloaded to the iPhone from Apple in case you move into range of one of them. i.e. At a guess what is happening is location services identifies a cell tower and asks for its location, and is replied to with the list of locations that contains that cell tower, that list is then cached so that it does not need to be requested again.

Of course, this is only a guess based on the wide range of addresses people are seeing and how its near to, but not exactly where, the people have traveled.

So rather than iOS actively and accurately tracking you and reporting this information to some, unspecified, intelligence agency it's actually the complete opposite; your device is actively downloading the next cell tower and, in some cases, wifi information that is near you and where you might be going to provide a better location experience. Which explains the inaccuracy of the locations people have been seeing in their version of the cache data and explains why there's some places they haven't been showing up in the data and why places they have been aren't showing up.

Of course, this information still has personal value and should really be secured by iOS and not by an individual having to secure their handset and encrypt their backups but if anyone still thinks they see the black helicopters circling, it looks more and more unlikely and, as Ed Parsons pointed out, a smartphone without location just isn't ... smart.

Photo Credits: Álvaro Ibáñez and Tom Jervis on Flickr. Written at home (51.427051, -0.333344) and posted from the Nokia gate5 office in Schönhauser Allee, Berlin (52.5308072, 13.4108176)

Retiring The Theory of Stuff; But First, A Corollary

2010-05-04 16:52

Theory of Stuff out to pasture. It's had a good life. It's appeared in 5 of my talk decks (or so Spotlight tells me), in 3 of my blog posts and continues to generate hits on my blog (or so my analytics tells me).

When I tell people I'm going to talk about my theory, a Mexican wave of shoulder slumping passes through the room, coupled with a prolonged sigh from an audience who've just resigned themselves to a slow painful death over the coming minutes. Luckily things perk up when my introductory slide of Anne Elk (Miss) and her Theory appears but even so, it's time to quit whilst you're ahead.

But before I do ...

It's time to put the Theory of Stuff out to pasture. It's had a good life. It's appeared in 5 of my talk decks (or so Spotlight tells me), in 3 of my blog posts and continues to generate hits on my blog (or so my analytics tells me).

But before I do ...

One of the great thing's about O'Reilly's Where 2.0 conference is the vast number of people you meet who just fizz with ideas and intelligence in this somewhat nebulous space that we call location, place or geo. One such person is Sally Applin; she owns the domain sally.com so that's got her off to a good start. After Where 2.0 she pointed me to her own theory that voyeurism and narcissism sell software.

People like to look at themselves and at other people. If they can do it at the same time–then the application will succeed! Look at Facebook, LinkedIn, Twitter, MySpace, Skype–basically any software that allows for both looking at others and self viewing, self reading, self posting etc…will sell. We’re on the chimp ladder. We like to compare ourselves and compete.

If you generalise software out to the slightly more generic terms ofservice or product; you'll see that Sally's theory complements the Theory of Stuff quite nicely and even provides an exemplar of those businesses and ventures that prove the theory.

This is especially interesting when you look at the success (to date at least) of ventures in the social space, such as Facebook, Twitter and Foursquare. What else are these is not an online way of saying "look at me, here I am, this is what I'm doing" and in doing so generating a vast sea of highly localised and personalised data into the bargain? Photo Credits: wili_hybrid on Flickr.

Contextual Location (and Echoecho Redux)

2010-02-22 13:29

recently wrote about echoecho, an SMS based location sharing service and rather dismissed it as another PlayTxt or DodgeBall, both of which are now shuttered, and argued that EchoEcho fails my Theory of Stuff.

Nick Bicanic, the CEO of Purpose Wireless, the company behind echoecho was good enough to look me up and drop me a long email commenting on my blog post and -- very politely -- pointed out that I might want to revisit my opinion of the service. An edited version of that email to me formed the basis of his latest blog post on the topic of location as a context.

I recently wrote about echoecho, an SMS based location sharing service and rather dismissed it as another PlayTxt or DodgeBall, both of which are now shuttered, and argued that EchoEcho fails my Theory of Stuff.

So have I done echoecho a disservice? Quite possibly ... to find out I (re)installed it on my iPhone and onto my BlackBerry.

(intriguing aside 1: it's a novel experience to have to install onto two devices to test out a service. Not a bad thing. Just different).

As Nick pointed out "it's not all that fair to describe a new service by saying what it isn't - so let me tell you what it is. echoecho allows you to ask and answer the question where are you? as easily and simply as possible ... that's it ... think of it as a cross between a permission based SMS and a tweet - the idea is that it becomes as easy and ubiquitous as SMS."

After playing with echoecho (and according to Nick it is all lowercase and not WikiWord style) I really like the service. It's simple, it's elegant, it's very easy to use and I can see myself using this with friends and family. Heck, if my Mum actually remembered to turn her mobile on then she could use this and use it easily. Yes, it's restricted to a range of smart phones (iPhone, Android, BlackBerry and so on) but the same applies to a whole plethora of LBMS.

(intriguing aside 2: the installation on my BlackBerry kept on repeatedly prompting me to view permissions and once viewed and saved prompted me to view and save permissions. Repeat until bored. A hard reboot of the handset fixed this finally. I don't envy people doing BlackBerry development). But let's go back to the Theory of Stuff for a moment; where does the money come from? It's a free service so you can't (directly) monetize the People. You're not tracking your audience's location (and Nick assures me they're not) and there's no additional data to derive, such as local business listings or a set of geotagged POIs, which is a (mostly hidden) side effect of FourSquare and Gowalla who seem to find themselves the poster-child(ren) of LBMS at the moment.

So at face value, much as I admire the simplicity of echoecho, I initially came to the conclusion that the service fails the Theory of Stuff but with a caveat. If there's something clever going on under the hood that's not immediately apparent to the casual observer or if there's a way of getting People to make Stuff through the service then echoecho might pass the Theory.

Nick agreed with me, "Clearly if the app is free then the money can't come from the app. But that's a failure only in the most immediate literal sense. By that logic every freemium model is a failure during its free stage".

All of the above has shown that there's a need for at least one caveat to the Theory of Stuff, which should state that the Theory should only be applied if there's an attempt to monetize. echoecho isn't and should, for the time being at least, be exempt.

But there is definitely something clever going on under the hood, a bi-directional open API location sharing service. It's that platform that echoecho is built on top of and it's that platform that I'm going to be watching very closely indeed to see what comes out of Purpose Wireless. And of course I'll be looking to apply the Theory of Stuff to that offering.

Photo credits: katachthonios and sayzey on Flickr. Written and posted from home (51.427051, -0.333344)

Like a lot of people, I get most of the information I use, both personally and professionally, from the web; from RSS feeds, from keyword search alerts and from Twitter. The genesis of my recent Theory of Stuff slowly accumulated out of this mishmash of feeds, alerts and status updates.Firstly I read about EchoEcho, a new location based service which promises all manner of good stuff by showing you where your friends are regardless of which location based service they currently use. Let's leave aside for one moment that the service independence of this app seems to be based around the concept of getting all your friends to use EchoEcho and then consistently getting them to report their location. Let's look at something far more fundamental than that, the strong sense of location deja vu harking back over two years ago. Haven't we been here before?

Hindsight seems to have proven that concepts such as "who's nearby" and "show me where my friends are" aren't, on their own, enough to build a business around. The brief flare of enthusiasm over services which tried this approach such as PlayTxt and DodgeBall were soon extinguished as users, fickle as they are, got bored and moved onto the next big thing.Then there were two articles looking at "checking in", both looking at FourSquare and Gowalla but each one coming at it from wildly differing ends of the experience. On the one hand, there was Business Week quoting the eye watering "I don't feel complete unless I check in" from FourSquare, Gowalla and Yelp addict Diane Bisgeier. Though the article focuses on this as a San Francisco and the Bay Area phenomenon, this has crossed the Atlantic with vigorous checking in going on in the UK and in mainland Europe. I may even have contributed to this, from time to time.A totally contrasting view was shown by Andrew Hyde who was fed up of "the needless ego boost" of saying where he was and "committed location based suicide" by deleting his accounts from FourSquare and Gowalla. We'll leave to one side the irony that this was done very publicly and with an accompanying blog post.All of the above moved Thierry Gregorius to lament that "if 'normal' people don't see the point of location-based services, how can the geo-industry claim being mainstream?". A valid point but one which confuses the very visible front end view of location, as seen in LBMS and the less visible back end view of location. Ed Parsons summed this up succinctly by comparing back end location with the DNS system, which "normal people don't see the value of but use every day".It was these three themes, "who's nearby" as a raison d'etre alone, maintaining an audience by check-ins alone and selling location based services to a wide audience that made me sit down and write up my Theory of Stuff. The full text of this is in a previous post, but the short version of the theory states that in order for a business to succeed you need three things, some Stuff, be it data, inventory or something else, some People, your audience and some Secret Sauce which allows you to connect the audience to the stuff in a bidirectional manner. So how do these three themes fare against the theory of stuff? Surprisingly and thankfully, they all seem to validate it.The concepts of "who's nearby" and "where are my friends" on their own, fail the theory of stuff. You have People, and in some cases a very large and quickly growing audience. You have some Secret Sauce which connects those People via their locations. But because there's no Stuff to start with and the secret sauce isn't bidirectional, no Stuff is created. The effect of this is that monetization opportunities are non existent or severely limited and the service isn't sustainable. Both PlayText and DodgeBall are no more and the omens aren't looking good for EchoEcho as a result.Then there's FourSquare and Gowalla, both of whom seem to have been inspired by Google. Cast your mind back to when Google announced the concept of Street View which was met with sneers and derision from some. Before Street View even went live it was written off as a loss leader, a waste of time and money and it would be Google's white elephant.Others of us in the location industry took one look at a Street View car and noted that the cameras weren't just pointing parallel to the road surface to take photos of surrounding buildings. They were also pointing at the road and up at the road signage which, when combined with the fact that the (GPS, cell tower and wifi triangulation equipped) StreetView cars actually had to drive down the streets in question, would provide Google with their own mapping data that was also capable of powering routing and direction algorithms. A short while later and Google completes enough of North America to remove the need for TeleAtlas mapping data and makes massive savings on data licensing into the bargain.Street View passes the Theory of Stuff by providing new Stuff to be connected and monetized by their existing Secret Sauce and the People who make up their substantial audience.It would be easy to dismiss FourSquare and Gowalla as more up to date versions of the "where are my friends" service. While they seem to have created the current cultural phenomenon of checking in, which may well be their lasting legacy, both services have their own quirks (FourSquare's Mayors and Badges and Gowalla's items) and need to show they're capable of holding onto their existing audience and growing it, substantially. So this surely means that both FourSquare and Gowalla fail the Theory of Stuff? Not necessarily. Just as StreetView generated valuable Stuff for Google, so both FourSquare and Gowalla are also generating a detailed set of local business listings and points of interest, all of them neatly categorised and geotagged as a bonus. That's a lot of very valuable Stuff. This doesn't seem to have been something that's been noticed or commented on as much as it should be. If both these services can retain their audience and if they connect them with all the Stuff that is being captured and generated via Secret Sauce then they can most definitely pass the Theory of Stuff.The idea that location is analogous to the Domain Name System is slightly more challenging to fit into the Theory of Stuff's model but it's still possible.In the previous two themes, location has been the dominant factor in the provision of a service (PlayText, Dodgeball, FourSquare and Gowalla) or location data has been generated in order to create Stuff (FourSquare and Gowalla). In the DNS theme, location is not the prime reason for a service to exist, it's a context, part of the Secret Sauce, that helps the service provide its users with relevant information. This was highlighted by Kevin Marks and JP Rangaswami in last year's excellent The Impact of Context on the Mobile User Experience discussion at the Heroes of the Mobile Screen conference in London. Of course, you still need Stuff and People in order for this to work; Secret Sauce on its own is not a recipe for success.As nomadic devices have proliferated, the difference between The Web and The Mobile Web have vanished; it's just the web, regardless of how you experience it. A parallel can be drawn here with location. As location becomes more and more ubiquitous so the whole concept of a Location Based (Mobile) Service will also vanish, at least as a label. Location will just be a context. And there's nothing wrong with that; quite the reverse, as the location industry will have achieved their aim of ubiquity, of providing a service and information that everyone uses but which no one actually bothers to think about it being there.Photo Credits: Angelsk, dpstyles and leff on FlickrWritten and posted from Yahoo! campus, Sunnyvale, California (51.5143913, -0.1287317) Posted via email from Gary's Posterous

The Theory of Stuff

2010-02-01 12:33

Anne Elk (Miss), I have a theory. I call it my Theory of Stuff. I'm sure that other people, far more learned and erudite than I, have articulated such a theory but I've yet to come across any evidence for this and for now at least, it remains mine and it contains three buckets, looking something like this:

Once again, this is not the post I set out to write. The one I set out to write was called "In Search of Location's Sweet Spot" and it's sitting in draft and not yet posted. That's because before I can submit that post I need to write this one as a warm up act. Just like Anne Elk (Miss), I have a theory. I call it my Theory of Stuff. I'm sure that other people, far more learned and erudite than I, have articulated such a theory but I've yet to come across any evidence for this and for now at least, it remains mine and it contains three buckets, looking something like this:

On the far left hand side we have the stuff bucket. Whilst stuff may sound vague, it's entirely intentional. Stuff is defined as a collection or set of items, things or matter. Though I was focussing primarily on location data and location based mobile services, this applies equally well to other businesses and markets. It could be stock, inventory, left handed widgets or a plethora of other things. On the far right hand side we have people bucket. The exact number of people doesn't matter, for small businesses the number will probably be small and for large businesses the number will be, err, larger. These people are your customers, your audience. Hopefully they have money as well. And then in the middle we have the secret sauce bucket. Again, it doesn't matter what this is but it's very important to look at what the secret sauce actually does.

The secret sauce is a bidirectional pipe that connects stuff to people.
It allows you to expose your business's stuff to the people who are your customers, hopefully adding value along the way.
It also allows you to extract money from the people in exchange for access to your business's stuff. In the Internet industry we call this monetizing your audience.

In order for your business to succeed, you need to have all three of these buckets in place. Have people and secret sauce but no stuff? Fail. Have stuff and secret sauce but no people? Fail. You get the idea. Take a look at every business that is succeeding, especially those that are online and where the stuff bucket contains data, and you'll see that they have all three buckets in place. Take a look at those businesses which have failed or are failing, especially those that are online, and you either see one bucket missing or there's just not enough of it. Written and posted from home (51.427051, -0.333344) Posted via email from Gary's Posterous

Gary Gale

iOS Location Caching Round-up - Conspiracy Theories: 0, Smart Location Caching: 1

Retiring The Theory of Stuff; But First, A Corollary

Contextual Location (and Echoecho Redux)

Location is a Key Context, But Most People Don't Know This

The Theory of Stuff