Geographic

GeoRSS in the wild

I've been working today to try and get Drupal's GeoRSS module listening to more than just the deprecated Aggregator2 module to extract geographic locations from aggregated feed items.

The Feedparser module is the next on the list to support and as far as I'm aware is the only one of Drupal's aggregation modules to use an external parsing library, SimplePie. By using an external library it means that we don't need to deal with the sometimes complex task of parsing different types of feeds on the Drupal side, which is a bonus because efforts can be concentrated elsewhere whilst keeping the code nice and simple.

SimplePie is still in development stages but appears to have a good community around it as well as a couple of active developers. They're gearing up to their 1.0 release which includes functions to extract geodata from feeds using the W3C Geo and GeoRSS Simple encodings, the former being the most widespread of methods at present and the latter the one we should be moving towards using.

SimplePie's code is very much based around namespaces (e.g., xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#"), which a lot of other aggregator systems will often disregard in favour of the simpler method of parsing out just the individual element names from that vocabulary (e.g., geo:lat or geo:long) to identify the tags. Now that namespaces have suddenly become important (at least for SimplePie's code to work), it's interesting to see how easily overlooked they have been in the past.

Take, for example, the Geograph GeoRSS feed of their latest photos: they had a trailing slash after the GeoRSS namespace URI (http://www.georss.org/georss/ instead of http://www.georss.org/georss as it's defined in the spec). It was there because many namespaces do have the trailing slash, and simply left in by mistake, but because that's not what SimplePie was expecting, it didn't pick up the geodata in the feed. It's been fixed now (Thanks for the quick fix Barry!). There is also the case of the Flickr GeoRSS feeds that use the wrong namespace URI (using the one for W3C Geo instead of the GeoRSS one). Hopefully Rev Dan Catt or someone else at Yahoo will be able to fix that one up.

Even besides namespaces, some elements are often misused, and possibly the most widespread of those is geo:lon which should in fact be geo:long according to the spec. SimplePie doesn't understand the non-standard one and so can't pull the location information out of the feed. In this case, because it is so widespread, the parsing code should probably be extended to look for the non-standard element if it can't find the standard one.

Anyway, just some random observations of GeoRSS in the wild and how what seem like the smallest of differences can mean that the embedded location information will simply be missed by feed consumers.

If you've got a GeoRSS feed from your site, please do the right thing and make sure it's sending out the right information :)

Google doodles in Hyde Park

I was in the Apple section of a department store in town yesterday having a test drive of the latest MacBook and MacBook Pro laptops when I noticed that they had Google Earth running on them. Trying the MacBook Pro out first, I was very impressed with the responsiveness of the machine when exploring in Google Earth. The MacBook wasn't quite as impressive, but still very nice and an improvement upon the iBook I've got at the moment.

Google Maps version of Hyde ParkAs I was exploring, I zoomed into London and specifically into the area of Hyde Park and its north eastern corner. I had spotted a road pattern that didn't look quite right on top of the imagery that was being shown. Hyde Park is full of criss-crossing paths that are really quite distinctive from above, but what I was seeing didn't fit that pattern at all.

It rather looks like a glaring intentional error has been introduced, perhaps so they can tell when people have copied their maps verbatim (read the Maps that Lye page on the OpenStreetMap wiki for more information).

Yahoo Maps version of Hyde ParkWondering if it was perhaps a series of paths that had been introduced after the aerial imagery had been taken, I took a look at Yahoo Maps to see what they showed and they didn't have the paths included.

I suppose the logic in adding erroneous data here is that it doesn't matter if you follow it as you're in open space anyway, and so it won't matter to pedestrians if the paths don't actually exist.

Introduction to Neogeography

Have you ever been reading my blog and wondered what it is that I'm talking about, or why I'm so interested in everything geospatial (the vast majority of links I add to my del.icio.us bookmark collection use the term geo) and opensource software (especially in relation to the Drupal platform, which I got involved with through work)?

If you have, you may just find the Introduction to Neogeography by Andrew Turner a good primer. It's a great introduction (for those who already have a technological leaning) to the 'new geography', talking about concepts, common data formats, examples you can implement yourself and that sort of thing. It's a 54 page e-book and downloadable from the O'Reilly site.

The description reads

Neogeography combines the complex techniques of cartography and GIS and places them within reach of users and developers.

This Short Cut introduces you to the growing number of tools, frameworks,
and resources available that make it easy to create maps and share the
locations of your interests and history.

Learn what existing and emerging standards such as GeoRSS, KML, and Microformats mean; how to add dynamic maps and locations to your web site; how to pinpoint the locations of your online visitors; how to create genealogical maps and Google Earth
animations of your family's ancestry; or how to geotag and share your travel photographs.

I am glad that Andrew pointed out Drupal as a potential player in the GeoStack he talks about. As he puts it, "[t]he GeoStack encompasses the entire life cycle of geospatial data, from capture to consume using a variety of tools, formats, and applications." It's basically a suite of applications and services that can all speak geography to each other, sharing information with ease using standard formats. As an example, imagine going out with a GPS, uploading that information about your journey to somewhere, that site being able to share its information with sites that are designed to aggregate similar information and then have that available on demand, filtered as desired, to other services that can consume the information.

Drupal can actually play a part in each of these layers of the stack*, from allowing users to enter location information, serve it out, aggregate it from other sites and also be a consumer of that data.

* or will be able to again with a little work to get the newly updated location module and GeoRSS module talking properly with one another again

KML and GeoRSS now ready for Drupal 5.0

Over the past few days I've been readying the KML module (thanks to AjK for starting the work) and the GeoRSS module for new releases that will work on the latest, shiny, version of the Drupal content management platform: Drupal 5.0.

They are both now ready (with the exception of some minor bugs and some feature requests) and there are a number of bits I need to backport to the 4.7 version of KML module to ensure it starts working again with recent updates to the Location module. I also need to make sure that GeoRSS module is consuming feeds properly from the successor to Aggregator2, Leech, as well as Feedparser.

I've also been helping out a little with the port of Location module as it is an essential part of getting the two modules to produce their geodata. It's not quite ready to be tagged as being ready for Drupal 5.0 but most of it is already working in this release.

If you're interested in any of these modules, please try them out and report any bugs in their issue trackers. If you have any ideas for future features, please also add them in there. Ideas (and patches, if possible) are always welcomed!

Furthering the OpenStreetMap module

Having started implementing a Drupal module for OpenStreetMap back in October I have spent a few hours here and there on pushing it forwards. Here's a quick update.

The module is at a stage now where you can download data for a specific region from OpenStreetMap, parse it, filter it by certain tags (and their values, if desired) and then create basic location-enabled Drupal nodes based on the results. Because it ties into the existing location module any other modules which rely on the location API can begin to use these new OpenStreetMap nodes, for example by plotting them on a map, by making the information available through RSS feeds or by displaying them in Google Earth.

I also started working on an OpenLayers module, and at work have been putting effort into improving the MapBuilder module that Nedjo Rogers started a year or so ago. Both of these modules will allow us to reduce dependence in Drupal on commercial mapping providers, instead moving towards using data from other, standards compliant, sources.

Assuming there aren't too many distractions over the coming week or so, I hope to have at least an alpha-quality OpenStreetMap module available soon. Phase one of the module will simply be allowing site admins to keep their local information site up-to-date with geodata from OpenStreetMap. Future phases will almost certainly allow for editing of data and the publishing of that back to the OpenStreetMap project.

Free maps get a great new look

mapnik_isleofman.png

The free-to-use maps that volunteers in the OpenStreetMap project have been building over the past year or two have been given a new look. Thanks to the Mapnik project, the new maps look much more like Google Maps et al. and are actually available through the 'slippy map' interface on the OpenStreetMap website instead of having to download the data and generate them locally as had been true until recently (e.g. see the map of Stuttgart I generated recently).

There are a lot of tiles to generate, and it may take some time to generate tiles for the whole world. The focus therefore had initially been on getting England covered, but is now expanding to cover other areas. Nick Black stepped in to render the Isle of Man, where these examples are from.

Free map of Castletown, Isle of Man
mapnik_castletown.png

Free map of Port Erin and Port St Mary, Isle of Man
mapnik_portstmary.png

These maps are all available under the Creative Commons Attribution-ShareAlike 2.0 license.

Scanning a 1940s map of the Island

castletown-bay.jpg

I spent much of my evening today scanning in the 1940 Second War Revision map of the Isle of Man. Now that it's all scanned I took the opportunity to have a closer look around some of the places I'm familiar with back on the Island, as well as some things from the past which I'm not so familiar with.

Much has stayed the same on the Island since this map was made at the start of the Second World War, though there have also been some big changes. Towns have grown in size, bypasses have been built to take increasing traffic out of old centres, train lines that used to run from Douglas through St Johns to Peel and Ramsey have been dismantled, and the airport extended from its wartime status as an aerodrome into something a little bigger.

Before I started looking into the grid system used on this map this evening, I hadn't realised that the maps produced during the war weren't yet using the British National Grid for referencing, and instead were using a military grid that also consisted of 1km grid squares - just not the same ones as the National Grid.

Old maps of the Isle of Man

plan-of-douglas.png

A couple of weeks ago I started looking around on eBay for old maps of the Isle of Man, partly because they may be of some use to the OpenStreetMap project, and partly because it would be really interesting to see what had changed in the past fifty or hundred years around the Island.

The thing that triggered me to go out and find them was the launch of Richard Fairhurst's online browser of the 1930s and 40s Ordnance Survey New Public Edition map of England and Wales. He'd spent quite some time collecting the maps from secondhand shops knowing that 50 years after their publication the copyright on them expired and hence they get released into the Public Domain where anybody can do with the data what they wish. He worked with a couple of other people to scan all of the maps and build a site that lets people browse them and start to build up a copyright-free database of postcodes. It lowers the barriers for people willing to share the locations of their postcodes, and makes it much easier (though less accurate) than the Free The Postcode method of getting people with GPS units to submit precise coordinates for postcodes they know.

But back to the maps of the Island. The one map I was most interested in was the Second War Revision (Sheet 17) published by the Ordnance Survey in 1940. It covers the Isle of Man at a similar scale as the modern Sheet 95 of the Landranger series, but is now in the public domain, having been published 66 years ago under Crown Copyright. It will be a great reference point for extracting data for OpenStreetMap, showing the paths of rivers and roads between towns and villages on the Island. It doesn't really give enough detail to be useful within the towns themselves.

I also found some old street plans of Douglas, Onchan, Ramsey, Port Erin, Port St Mary and Castletown dating back to the late 40s (the Peel plan appears to state Nov 47). They can't be used for anything other than personal interest, as far as I'm aware, because they weren't published under Crown Copyright and the copyright instead falls either under the local authorities of those places, with the cartographers, or with the publisher. All of which mean that the maps won't fall out of copyright until at least 70 years after they were published, or 2017. Or at least, that's as I understand it.

The 1963 Ordnance Survey map (Sheet 87) I also found would have fallen out of copyright a little before that, in 2013, but in reality it probably doesn't give much more information than the 1940 version, and the Island will have long been mapped in OpenStreetMap by that point.

Talking of mapping the Isle of Man, I am planning to do some more mapping of Castletown and Douglas for OpenStreetMap between Christmas and New Year. If anybody is interested in helping out or finding out more, just get in touch. I have a spare GPS unit that can be used as well.

Mapping Stuttgart on OpenStreetMap

Map of Stuttgart, Germany on OpenStreetMap

Since the Munich mapping weekend a few weeks ago I've been putting some more work in to try and improve the mapping of the centre of Stuttgart in OpenStreetMap.

The image above shows the current state of the map for Stuttgart. It's still missing a lot of information including many of the major roads, mostly because I don't drive here. There were also very few streets in the centre of town until this weekend when I made a conscious effort to go in and collect data for some of the pedestrian areas of the city.

If you're in Stuttgart and interested in helping out with mapping, get in touch. There is GPS data appearing in the area from someone driving around, which acts as a good base for some of the major roads, so if that's you, get in touch and maybe we can work together to make sure they're tagged with street names and the like.

Mapping Munich in a weekend

Early on Saturday morning I got up to catch a train to Munich so I could meet up with a small group of other OpenStreetMap people with the aim of mapping the centre of Munich in a weekend.

I'm not a big fan of mornings but still I managed to get up early enough that I'd arrive into Munich not too long after the others had met. I arrived there, a little bleary eyed from snoozing on the train, and headed to Marien Platz to meet Ralf - the organiser - to find out which part of the city I should go off to map.

I went off to map inside the north western corner of Munich's Inner Ring road with a newcomer to the project. Mapping an inner city area, as I discovered when starting to map Stuttgart, is not an easy task. The GPS units stuggle because of the tall buildings and often narrow streets. It's difficult for them to keep a lock on the satellites, and when they do there is often still interference from all the buildings around that leads to tracks not being anywhere near as accurate as desired.

After a few hours of mapping it was time for lunch and time to meet the others. There were a group of eight of us, each paired up to take a part of the city centre. I struggled through lunch trying to understand as much of the German language conversation as I could, not really able to add much because my German's not good enough to butt into conversations with yet. After a pizza and a beer, it was time to go back to our areas and finish up the mapping.

By the time we all met again in late afternoon we were all starting to feel a little sore under foot, so happily sat down to see how we had all progressed on the first day. There was a slight worry at one point, after downloading the tracks from one type of GPS unit (the NaviGPS, which most of the group had) appeared to have no data from the morning. It turned out later on that evening that the data hadn't been lost, but for some reason there were some issues with the SD cards that the GPS units save the tracklogs onto, meaning that not all the files were showing.

By the morning, Joerg had printed out some updated maps to work from, as we'd all agreed that it was much easier to fill streets in on a skeleton map than to try and start from scratch. Some of the areas hadn't been mapped as a few of us didn't have the ability to go in and draw street segments on top of the GPS tracks, but Joerg and Ralf had both managed to do at least some amount of mapping the streets online in OpenStreetMap from the previous day.

There were four of us on Sunday, two on bike and two of us not. The two of us not on bikes headed down towards Museumsinsel - the location of the Deutsches Museum - to map that area before meeting for lunch. After lunch came another couple of hours of mapping an area around Ost Bahnhof.

After all of that walking, we still need to go in and create all this data in the OSM wiki based on our GPS tracks, notes, diagrams and memory. That's what I've been doing tonight, and still there are a lot of names I have to fill in from the map we drew. Already though, the central area of Munich is looking much more densely covered with map data than before the weekend.

Pages

Subscribe to RSS - Geographic