tommy's scratchpad

Good news.  With all the new features and performance improvements, I’m guessing I’m not the only one looking forward to this release.

Neat post by Matthew Perry. I couldn’t help but run the same command:

$ history|awk '{a[$2]++ } END{for(i in a){print a[i] " " i}}'|sort -rn|head

188 cd

151 svn

113 vi

88 ll

60 sudo

53 more

40 make

40 ./dd.py

30 ./run_test.py

30 ls

dd.* is my tempfile name prefix for quick and dirty scripts. run_test.py is from running the MapServer msautotest regression test suite when developing and testing code.

What does your shell history look like?

Between a dual boot and a MacBook, I have started to have bookmark synchronization issues.

I came across Foxmarks, a neat bookmark synchronizer for Firefox. Essentially, your bookmarks are placed on a central server so you can fetch them from wherever. The only change that I made was removing some internal / sensitive bookmarks which I wouldn’t necessarily want to be “out there”, so I stored those locally.

Pretty easy!

OGC service standards (such as WMS, WFS, WCS, SOS) support something called updateSequence. In short, this value represents an identifier which clients (catalogs, applications, etc.) can use to maintain their local copy up to date. This saves clients from fetching and parsing an entire (sometimes hefty) Capabilities XML document in cases where the server has not made any updates.

Thanks to a recent enhancement, MapServer now supports updateSequence for WMS Server, WFS Server, WCS Server, and SOS Server. Support is triggered by mapfile metadata MAP/WEB/METADATA/ows_updatesequence. The value can be a string, integer or ISO8601 timestamp.

We initially thought of using the mtime of the mapfile, but this may not have been the most indicative. What if other files (pointed to by the mapfile) are modified, and not the mapfile? What if the mapfile is touch’d and and no content changes are made? For this reason, we went with a user-specified value. Basically, if the user knows the purpose updateSequence, they can use it and manage it at their own discretion.

I suppose the OGC could have allowed using the HTTP Last-Modified header, however that may get slippery in terms of semantics.

Give it a shot! I’d be interested in hearing how this works with existing catalogues and applications. The enhancement seems to be passing the CITE tests initially (let’s keep our fingers crossed!). At any rate, it is fun to code an enhancement like this. It’s also nice to have dedicated tests to all the use cases surrounding updateSequence within msautotest/wxs/.

Some of the MapServer developers may have been getting more than their share of trac notifications this week. This is partly because I’ve taken the last few days to sift through many tickets which have no milestone attached to them. Between old issues, bugs that have since been fixed, etc., it was nice to clean house a bit. They contributed to ~20 tickets being closed just this week! It’s amazing to see how many old school tickets there are lying around in trac. I try to commit to a once a month sift of trac for the oldies, but sometimes goodies 🙂

Read the rest of this entry »

That’s how I take my coffee 🙂 You may have heard of Tim Hortons — a mecca for coffee shops in Canada. Many of us are lost without it during our morning commute. Even though they are seemingly everywhere, there are times when you just can’t find one.

A friend of mine pointed me to a forum working on collecting locations of all Tim Hortons locations in Canada and the US. Wouldn’t it be great to upload these to a GPS which supports GPX format? Or to visualize in Google Earth, for example?

This is easily done using open source geospatial tools. OGR supports the reading and writing of a number of formats, including GPX, KML, GeoJSON, Shapefile, and so on. Here’s how:

• Download http://www.gpspassion.com/upload/team/Dewi_TimHortons_xls_20071120.zip th.zip
• Save as CSV (there is a CSV online, but the XLS has more detailed description of each location)
• Use OGR’s Virtual Datasource support to read the CSV. Here’s the OVF definition:

<OGRVRTDataSource>
<OGRVRTLayer name="th">
<SrcDataSource relativeToVRT="1">./th.csv</SrcDataSource>
<GeometryType>wkbPoint</GeometryType>
<LayerSRS>WGS84</LayerSRS>
<GeometryField encoding="PointFromColumns" x="Longitude" y="Latitude"/>
</OGRVRTLayer>
</OGRVRTDataSource>

Transform to your heart’s content:

$ ogr2ogr -f KML ./th.kml ./th.ovf

$ ogr2ogr -dsco “GPX_USE_EXTENSIONS=YES” -f GPX ./th.gpx ./th.ovf

I’ve bundled up a zipfile of various formats for use as an example — have fun!

Since I did this last year, I thought I’d try this again for 2007. Here’s a lowdown for my 2007:

• REST is really here: and as popular as ever. For me, I finally realized that REST was a style, not a syntax, API or schema for that matter. Sean’s recent post sums it up quite nicely. It’s nice to see the OGC has acknowledged this
• Mass Market is really here: Things like Google, Yahoo Maps, and lowering the barrier to application development and, more ultimately, content, are paramount in the Web 2.0 sphere of things. Defacto standards / approaches matter
• Geospatial Catalogues: I blogged about this last year. Even with Cat 2.0 CSW / ebRIM formally approve, where is catalogue interoperability? Is a federated / distributed catalogue approach realistic in the near future?
• MapServer:
  • SOS Server: There has been much development here, including forthcoming support for 1.0.0, as well as implementation of POST support in mapogcsos.c. SWE DataBlock has also been implemented. We’ve also introduced automated testing for SOS in msautotest/wxs/sos.map. And a slew of bug fixes, memory leak fixes, etc. So I’m really happy about how SOS Server support has progressed this year. It goes without saying that major props to Assefa are in order here, as well Charlton and other MapServer SOS Server testers who have provided valuable feedback
  • OWS Common: Through increased usage in SOS Server, as well as the WCS 1.1 Server support (thanks Frank!), mapowscommon.c usage has been slowly increasing in the codebase to reuse existing functionality. This is a result of the OGC standards gradually adopting OWS Common for their “common” bits of XML and such. mapowscommon.c is also becoming more robust as a result of more integration and testing. I’m also happy about the increased libxml2 support in MapServer. We used libxml2’s xpath functionality in supporting SWE DataBlock, and have generic utilities now defined in maplibxml2.c
  • I’m happy to have become more involved in the MapServer codebase and project overall as part of the Project Steering Committee. There is definitely alot going on and alot to do, and the enthusiasm, commitment and helpful nature of the developers is great. MapServer is a special project to me, starting in early 2000 as part of my research / dissertation. The rest, as they say, is history. Oh, and “STYLES=” is optional again 🙂
• Publications: I was glad to see “The Geospatial Web” published this year. Nice to see new folks and goings on out there
• Adventures in Python: I decided to dive into python finally. With some sound advice from Sean, I embraced stuff like Genshi and etree. I managed to stand up a generic SensorML generator which I’m happy with, as far as my first Python project goes. My next goals are to experiement with doing things differently (like SQLAlchemy; I had to use good old SQL scripting as I ran out of time)

In other news:

• Construction time continues: I did some major work on my place, including new fencing, landscaping/gardening, central air conditioning, as well as a shiny new veranda. A bit more curb appeal for sure
• Condo: I finally picked up my condo this year. After much thought, I decided to sell. This was a beautiful property in a can’t lose location! Onwards
• Lifestyle changes: Perhaps my biggest victory is quitting smoking. I have been absolutely smoke free for all of 2007. I can’t say that there haven’t been times where I wanted one from time to time, but I feel much healthier and better overall

So as 2008 quickly approaches, here are a few things I’m looking forward to:

• MapServer: I look forward to continuing in helping more with OGC support, as well as become more familiar with the inner workings of the codebase.
• OpenLayers: I really want to get into OpenLayers this year. I have not done much on client side work since the initial Mapbuilder days
• New Springer publication: keep an eye out for a new, exciting book on GIS and open source this year
• OWSContext and KML: it will be useful to see a resolution or unification of some sort between these two standards
• OpenSearch: I think OpenSearch, with the geo extensions, will make a big hit this year. Nothing like a simple search facility which is already supported in browsers
• I’m renovating my basement this year. Gutting the entire basement and installing a new bathroom and kitchen. I hope to have this done by summer 2008 — wish me luck!

So that’s it from here. I wish you and your loved ones the very best in the holiday season and for 2008!

This past week has seen alot of activity in MapServer’s SOS Server Support. Remember when I posted about DataBlock? Well, Assefa did a great job in initially implementing this, and you can see the results in action right here. I’m going to be working on ensuring the XML conformance as per the SOS and OM standards, but it’s great to see this working!

This weekend, I added POST support. The SOS standard states that POST must be supported for all requests (except for GetCapabilities, which may be GET only). As a result, both GET and POST are now supported for all requests. I used libxml2‘s XPath support to parse the requests. XPath is an awesome way to parse XML! Support is not fully completed yet, as eventTime, result, and featureOfInterest are still yet to be implemented for POST (they are implemented for GET).

It will be interesting to see the mileage DataBlock gets. Especially seeing that now MapServer SOS presents a nice compact option to disseminate observation data over the web.

As part of MapServer’s support for standards-based interfaces, the SOS Server support is coming along quite well. With SOS 1.0.0 now adopted (although the schemas and standards document are not on the OGC public website yet), this provides a good opportunity to update the SOS support (which was previously at 0.0.31 and 0.1.2). This also was also a good time to move some of the GML code out of mapogcsos.c and into modular functions within mapgml.c.

Read the rest of this entry »

Fresh off the press, check it out.