Like most geeks I pride myself on being able to maintain the computers I use on a daily basis. Just like real men don’t eat quiche and real programmers don’t use Pascal, real geeks don’t call for professional help or technical support.

But then the day comes when one of your hard drives goes crunk, you go through all the tricks of the trade you know, you exhaust searching for possible solutions on the web and you realise that maybe, just maybe, while it’s not time to eat quiche or starting coding in pascal, it’s probably time to call for some professional help.

Like a lot of people, I’ve amassed a not inconsiderable amount of digital media over the years, in the form of apps, songs, movies and photos. Most of these live on my laptop and are religiously backed up with SuperDuper! and with Time Machine to external drives, with one of these drives holding the overspill. This aforementioned external drive had given solid, reliable service over the years but had started to act … quirkily. Fearing a critical mass of bad sectors I decided now was a good time to backup my backups.

And then it happened. Crunk. The disk died. So I fired up OS X’s Disk Utility and verified the disk. It had … issues. Time to repair the disk. So it chugged and it whirred and the progress bar progressed with glacial slowness until finally, several hours later, I saw the message I dreaded.

Disk Utility can’t repair this disk. Back up as many of your files as possible, reformat the disk and restore your backed-up files.

Of course, it was probably my fault. Despite the number of bad sectors and other magnetic media glitches that accumulate over time on a disk drive, the drive itself had still been functioning; probably because I’d never actually tried to read from one of those bad patches recently. But in trying to backup the drive, I was pretty much accessing every sector on the drive with the resulting crunk being pretty inevitable.

So what to do? Most of my photos were already hosted on Flickr. A lot, but by no means all, of my music could theoretically be re-ripped from CD. But my backups of my iPhone and iPad were gone and let’s not even begin to talk about the movies. It may only have been under 500 GB’s worth of data, which is a drop in the ocean compared to today’s multiple terabyte drives, but it was a lot of data to me and it represented a lot of time, effort and memories.

Maybe data recovery was possible? A quick online search for “mac data recovery” had my bank balance wincing in shock. This was going to be expensive, if it was possible at all. Most recovery firms charged to look at the drive and then charged to extract the data from the drive, with pricing being based on the number of files, not the capacity of the drive. Then I found Tierra Data Recovery. Fixed pricing, free analysis of whether the data could be recovered, free courier collection and payment only on successful recovery.

It seemed too good to be true. But a quick phone call, explaining the situation and Tom from Tierra, as he will now be known, calmly laid out my options. So the following day a courier collected my drive and took it to Scotland and a couple of days later I got an email from Tierra with the news that all of my data could be recovered for a little over £300.00, and after shipping a new drive to them, all of my data made its way from Scotland back to London.

Here in the UK we’ve become accustomed to being gouged by companies, to expecting poor or no customer service and to be treated like a cash cow. Which makes the speed and quality of the service provided by Tom and Gill at Tierra all the more unexpected and pleasing. I hope I never need the services of a data recovery company again, but if I ever do, Tom from Tierra will be getting my business again without a second thought. If you find yourself in this unenviable position, you should give Tom a call too.

But the strangest example of this sort of behaviour was in the first reports of recent events on the border of Costa Rica and Nicaragua that seemed to implicate Google Maps as justification for Nicaraguan troops crossing the border into Costa Rica and raising the Nicaraguan flag on Costa Rican territory. The dispute seems to hark back to the 1850′s where the contested border between the two countries followed the course of the San Juan River, the course of which has since moved somewhat, as rivers are wont to do. Costa Rica asserts their sovereignty on the disputed land based on the 1850′s arbitrated border which follows the course of the river and Nicaragua asserts theirs based on the fact that the river has moved so some land must be theirs.

The reference to Google Maps turns out to be a bit of a red herring as well, originating from an opportunistic sound bite rather than fact. Granted Google have based their data set on admittedly sparse data, some of it originating from the US State Department, which had got it wrong. But other mapping data vendors, who should know better and who at the time were having a great laugh at Google’s expense on various forms of social media, turn out to be just as incorrect as Google’s.

While this is probably the most extreme example of “but I found it on the internet so it must be true“, the whole story is less about whose map is right, less about blaming map error on an online map and more about how some parts of the world are less well mapped than others. Not all map data is created equal.

The twists and turns of the story are best followed on the original post from Jonathan Crowe’s excellent The Map Room blog and its follow up as well as an in-depth article on the subject from Ogle Earth.

But a conference audience is an odd beast; a percentage of which will be “the usual suspects“. They’ve seen you talk before, maybe a few times. The usual suspects also tend to hang out on the conference Twitter back channel. Woe betide if you recycle a talk or even some slides too many times; comments such as “I’m sure I’ve seen that slide before” start to crop up. Far better to come up with new and fresh material each time.

But sometimes you can get away with it and so it was with my theme of GeoBabel. Three conferences: the Society of Cartographers Summer School, The Location Business Summit USA, AGI GeoCommunity 2010. Three cities: Manchester, San Jose, Stratford-upon-Avon. Three audiences: cartographers, Silicon Valley geo-location business types, UK GIS business types.

I’ve written about GeoBabel before; it’s the problem the location industry faces as we build more and more data sets which are fundamentally incompatible with each other. This incompatibility arises either due to differing unique geographic identifiers, where Heathrow Airport, for example, is found in each data set, with differing metadata and a different identifier, or due to different licensing schemes which don’t allow data to be co-mingled. We now have more geographic data than before but each data set is locked away in its own silo, either intentionally or through misguided attempts to be open.

The slide deck, embedded above, is the one I used in San Jose. The ones for Manchester and for Stratford-upon-Avon are pretty much identical but are on SlideShare as well.

As another way of illustrating the problems of GeoBabel, I came up with what I’ve termed The Four Horseman Of The Geopocalypse. All very fin de siecle but it seemed to be understood and liked by the audience at each talk.

The first Horseman is not Pestilence but Data Silos. All of the different types of geographic data we have, international and national commercial data, national and crowd sourced open data, specialist and niche data and social network crowd sourced data each live in isolation to each other with the only common denominator being the geo-coordinates each data set’s idea of a place has.

The second Horseman is not War but Licensing. Nowadays in the Web 2.0 community we’re used to having access to data but we’re not willing to pay for it. Licenses vary between closed commercial licenses and open licensing. But even in the open license world there are silos, with well meaning licenses becoming viral and attaching themselves to any derived work.

Which segues neatly to the third Horseman, who’s not Famine but Derivation. Each time you create something from data, you’re deriving a new work in the eyes of most licenses and that means the derived work often has the original license still attached to it. You do the work, but you don’t own the work.

Finally, the fourth Horseman is not Death but Co-Mingling. There is no one single authoritative geographic data set, you need to find the ones which work for you and for your business or use case. That means you need to mingle the data sets and frequently the licenses you have for those data sets explicitly prohibit this.

But now after three outings, it’s time to retire GeoBabel, for now at least, just as I retired my Theory Of Stuff earlier this year. That means I had to find a new theme to talk about at my next event, the Geospatial Specialist Group at the British Computer Society. But that’s in my next post.

Ovi Places: Mobile Navigation needs to knock down its brick walls

When Ovi Maps launched at the start of this year, it really shook up the navigation industry. The free software gave everyone with access to Nokia’s Ovi Store a perfectly serviceable Personal Navigation Device (PND), completely for free. But Ovi Maps is just the first exposure of the Nokia branch called Ovi Places. Recently appointed Director of Ovi Places, Gary Gale, took some time to talk to GoMo News about the state of mobile navigation ahead of his appearance at the Location Business Summit, USA, 14-15 September, San Jose.

Most people know about Ovi Maps, but a lot won’t have heard about Ovi Places. What is it, exactly?

It’s the slightly unglamorous name for a set of back-end systems that understand what people are looking for. Within the Ovi Maps client, on both mobile and internet, there’s the ability to look for what the industry calls Points Of Interest – or POIs. But we prefer the term “places” – because POIs comes laden with preconceived baggage. Our colleagues in Japan consider anything that isn’t nailed down as a POI, including bus stops, park benches or traffic lights. That can lead to too-much data, an overflow that can’t be easily consumed. People tend to think of these kind of location and navigation services as a yellow pages business listings – which is certainly important for the classic LBS model of “where am I, and what’s around me”. But Ovi Places takes into account local information, colloquial information, landmarks and places you’d want to go to as a tourist. For example, where I am in the Nokia office in the middle of Berlin, we’ve got the really common tourist POIs showing up – like the Brandenburg Gate, for example – but Places also refers to an excellent restaurant in the courtyard below me, and a local coffee shop.

Where do you source that info? Are there Places fact finders or do you buy the info?

It comes from a variety of sources. Some of it comes from commercial data providers – this is actually one of the main reasons we acquired NAVTEQ, and why TomTom bought TeleAtlas. Digital mapping companies have a rich set of data above and beyond the normal PND stuff. But there are also a whole variety of specialist premium partners that we do deals with; we’re talking about regional specialists that we talk to on a country-by-country basis in order to gain their local insight.

There is no “one true” source of data – you need to make a lot of partnerships to get the best local data available.

At the moment, Ovi Places really only powers the Ovi Maps application. Are there plans for more services to exist under a Places umbrella?

At the moment, it’s exposed only through Ovi Maps. For the future… I can’t say anything specific, but watch this space!

How do you plan to make mobile location more personal to the mobile user?

Actually, the mobile user is probably the easiest use case for navigation. Your device has lot of options available to it to determine your location. From there, services like Places can provide rich experiences. The key problem is whilst all of this is pretty much mainstream now, there is a “Bay Area bubble” where a lot of the products and services coming out seem to think your user will always have a smartphone, and will always have a GPS lock with an excellent data connection. That may be fine for San Francisco, and even Western Europe. Sometimes even areas you think would be well served are awful. I recently went on a trip to Calais – when I got off the ferry and the GPS took 15 mins to pick up a lock. So you have to realise that there can be patchy 3G data coverage in even highly developed countries, and then look at areas which have growing economies and even worse connections. There are places in Africa and Asia that won’t have 3G data in the next 5 or 10 years.

You mentioned that mobile users are the easy use cases – what would you consider to be a challenging case?

The challenges arise when you’ve got infrastructure problems. Consider some of the poster child location services, like Foursquare, Gowalla and Yelp. Lack of 3G data infrastructure doesn’t appear to be factored into the business models for these companies. Try using one of them in Africa, or India, or Asia. The infrastructure isn’t there to address these needs. The populace simply don’t have access to these services.

Is Places doing anything to address that problem?

We’re looking at potential handsets that don’t need a dedicated on-board GPS or AGPS. They don’t need the typical app store economy. We’re able to tap into cell tower triangulation, where local laws and legislation permits it. It may not be as accurate as a GPS lock, but it’s better than nothing.

Is that really important for a developing country? How worried is a resident really going to be about their location services.

I think the best answer to that is from an article by Dr. Tero Ojanperä (Executive Vice President of Services, Mobile Solutions, Nokia). He said that the target is less about producing a device that runs apps than it is about creating a really useful platform – it’s more about producing a context-aware device, that gives you the best relevancy depending on the services available to it.  ”It’s about devices that offer truly connected services and learn your habits so well that they can give you what you want“. That means you have a service that will provide good services to every customer, no matter what the state of their local infrastructure is.

Last month I was at the GeoLoco conference in San Francisco, talking on a panel about the challenges the industry is facing. An audience member asked “what advice would the panellists give to someone who is trying to establish a foothold in location?” I felt my answer got the most responses, at least on the Twitter back-channel. which was “I come from Europe – don’t forget that we exist! There is a market outside of North America that is different in its needs and infrastructure“.

Services like TeleAtlas and OpenStreetMap (OSM) make a lot of use of crowd-sourced info. Does Ovi Places allow for that?

Very much so. We already have this kind of functionality built into the newer handsets, allowing you to add corrections and updates while you are on location. Crowd sourcing is very much a part of this industry’s future – but I don’t think it’s the panacea that people think it might be. It’s a vital additional source, but not the best thing since sliced bread until; at least, not until the industry gets together and comes up with a way to verify and editorialise new info. It’s a benevolent technological anarchy – because there’s no formalised control over how you tag a place, a consumer has to accept that finding out how to use the data will take significant time and revenue investment. If your local authority is trying to map its assets, you want to make sure those assets are exactly where you claim – because taxation and revenue streams can be assessed on that. If you get that wrong, it will lead to the kind of bad press a local authority doesn’t want. Especially if emergency services are trying to get to a specific street address – you need that data to be 100% accurate.

What do you think the main challenges facing mobile navigation are?

I think there two main challenges.

First is the privacy angle. People don’t quite understand what it is that they’re giving up to use the latest LBS app. You need to make sure that people understand the value proposition on the table when they’re giving up their location to gain relevance in their local search. The public as a whole needs to understand this. And it will probably be driven by tabloid headlines – some celebrity who gets divorced because a location service proves they weren’t where they said they were. And it would be better if it didn’t happen that way. I hope the Industry is open and transparent about it as much as possible. It will be to our detriment if we don’t expose this kind of information, and something sensationalist does happen.

Second, there’s a need for people to talk to one another. We’re all building loads of very rich data sets – OSM is doing it, Facebook, Foursquare, government services, NAVTEQ – but at the moment, to unlock their potential, they need to talk to each other. The current licensing set up means location data is still stored in a series of vertical silos which aren’t allowed to work with each other. And the actual industry moves so fast that even those who are involved in it find it hard to keep up with developments. So keeping the legal and licensing system up-to-date with it must be nightmarish. It’s getting increasingly more difficult to get solid patents in this area – and patents being wielded by the patent troll houses are being used in a way they were never intended. In order to work around this, I think the future will have to be less about aggregating these data silos, and more about synchronising the end-point exposure. If you have an identifier in one data set that corresponds to an identifier in another data set, they can sync up and present a united service to the end user… without having to share protected data.

Gary Gale will be speaking at the Location Business Summit, 14-15 September, San Jose, where he’ll be further addressing the issues surrounding the “silo problem” and licensing issues.

Owing to overwhelming demand by apps that use the service, the London Underground feed has had to be temporarily suspended. We hope to restore the service as soon as possible but this may take some days. We will keep everyone informed of progress towards a resolution.

In the meantime, if you want to see how it does looks when the API is up and running there’s a video clip of Matthew Somerville’s recent Science Day hack visualisation over on my Flickr photo and video stream.

I assert that train operators know where their assets are; it would be irresponsible if they didn’t

Whilst the plethora of train operators that fragmented from the ashes of the old British Rail network haven’t answered this challenge yet, Transport for London has, opening up just such data as part of the London Datastore API. In today’s age of talented web mashup developers, if you release an API people will build things with it if the information is useful and interesting and that’s just what Matthew Somerville of MySociety did at the recent Science Hack Day … a (near) realtime map of the London Underground showing the movement of trains of all of the Tube lines. A screen grab wouldn’t do it justice and it takes a while to load, so a video grab might help here.

Coming down the escalators at Waterloo and want to know whether to head for the Bakerloo or the Northern Line to take you north of the river? Now you can tell which line has a northbound train closest to Waterloo.

Want to see just how close the gap is between Leicester Square and Covent Garden on the Piccadilly Line really is? Now you can.

Of course, this doesn’t solve every problem …

1. If you’re on the escalators at Waterloo how do you get 3G data coverage to view this mashup on your phone as Transport for London still haven’t manage to achieve cellular coverage underground, unlike Amsterdam, Berlin and other cities?
2. The site will probably be the target of a tutting campaign from the Health and Safely police insisting that such a visualisation will cause people to run for the train and of course, they might trip and hurt themselves.
3. If you’re at the top of the escalator and the train is in the station, now, right this very minute now, how do you get down to the platforms quickly?

Whilst I can’t answer the first two of these questions, this publicity stunt from Volkswagon at Berlin’s Alexanderplatz U-Bahn station might just hold the solution for the third question … a slide!

Sometimes a map becomes more than just a spatial representation and becomes something else.

Sometimes a data visualisation becomes more than just the underlying data and almost takes on a life of its own.

When these two things meet or collide the results can be spectacularly compelling and produce, unintentionally … art? Look at the image below … filigree lace work? Crochet for the deranged of mind? Silk for the sociopath? Macrame for the mad? Sadly none of the above.

It’s instead an image from the Geotagger’s World Atlas but it’s still unintentionally beautiful.

The maps are ordered by the number of pictures taken in the central cluster of each one. This is a little unfair to aggressively polycentric cities like Tokyo and Los Angeles, which probably get lower placement than they really deserve because there are gaps where no one took any pictures. The central cluster of each map is not necessarily in the center of each image, because the image bounds are chosen to include as many geotagged locations as possible near the central cluster. All the maps are to the same scale, chosen to be just large enough for the central New York cluster to fit. The photo locations come from the public Flickr and Picasa search APIs.

I could look and stare at the all the images in Eric’s Flickr set for hours. Correction, I have stared at the images for hours.

1. not limited; unrestricted; unconfined
2. boundless; infinite; vast
3. without any qualification or exception; unconditional

Except in the world of mobile data or mobile broadband, where unlimited means, in a vaguely disturbing twisted, inverted, doublespeak sort of way, the exact opposite.

Vodafone, my current UK mobile provider, helpful tells me that I have unlimited data, subject to their fair use policy which promptly redefines unlimited as very much limited indeed and your limit is 5GB per month. That’s a lot of data. Even being the compulsive photo uploader, web browser, Foursquare and Gowalla check in, Twitter and Facebook poster and checker that I am, I’m hard pressed to go above 500MB per month let alone 5GB.

So I was both vastly amused and somewhat shocked when this text arrived on my iPhone on the way home from work last night.

A quick call to Vodafone luckily cleared this up as being a glitch in their billing systems and I would not, as stated be charged, nor had I gotten anywhere near the 5GB limit of unlimited.

I found the whole process rather amusing in hindsight but shouldn’t the mobile companies either come clean about what unlimited really means or just don’t sell unlimited data as a concept at all and just sell a, 5GB in my case, data limit?

They increase link sharing; the vast majority of social networking sites use 140 characters as the maximum size for an update, using the full version of a URL you’re sharing reduces the amount of space for you to put your own thoughts into the update. Just compare the full URL http://www.vicchi.org/2010/03/03/reclaim-and-own-your-short-urls at 65 characters against http://bit.ly/aG1RBx at 21 characters.

They can track and yield click and referrer information; the information that bit.ly provides is so useful, showing live clicks, geographic and referrer information amongst others.

But almost a year ago, Delicious founder and ex-Yahoo! Joshua Schachter made some pretty compelling arguments against short URLs:

The worst problem is that shortening services add another layer of indirection to an already creaky system. A regular hyperlink implicates a browser, its DNS resolver, the publisher’s DNS server, and the publisher’s website. With a shortening service, you’re adding something that acts like a third DNS resolver.

…

But the biggest burden falls on the clicker, the person who follows the links. The extra layer of indirection slows down browsing with additional DNS lookups and server hits. A new and potentially unreliable middleman now sits between the link and its destination. And the long-term archivability of the hyperlink now depends on the health of a third party.

Or to put it another way, you no longer own your links or the data clicks that those links yield. If the service dies, your links break, pure and simple, and that does happen, as the demise of the original tr.im and cli.gs services show.

But there is a way to take all the benefit that short URLs offer and keep ownership of your links and all the data that clicks on those links will give you and that’s to run your own URL shortening service, which is precisely what I’ve done with vtny.org which is running the YOURLS code behind the scenes. This gives me all the benefits and metrics that other URL shorteners provide but with the added and crucial benefit that I now own the links and the data they generate, in this case via the vtny.org/4 short URL.

Posted via email from Gary’s Posterous