tommy's scratchpad

So 2021 wasn’t much better than 2020. Another year of endless virtual meetings and the 24 hour office. Here are some updates from WFH life:

pygeoapi: both OGC API – Records and OGC API – Environmental Data Retrieval support were added to the codebase. The project also saw both CQL and i18n support, which is a positive indicator of contributions from various developers. Thanks Sander Schaminee and Francesco Bartoli!

pycsw: OGC API – Records and STAC API were both implemented. In addition, CQL support was added with the help of the impressive pygeofilter package — great work by Fabian Schindler!

QGIS MetaSearch: standards implementation needs both servers and clients, and so OGC API – Records support made it into MetaSearch. A nice by product of this enhancement is the implementation in OWSLib, which MetaSearch uses as its discovery library.

OGC API (Records, EDR): EDR is now an adopted standard! Records also made great strides in 2020, and helping clarify the relationship with STAC has proved valuable for all communities involved.

WMO: Lots of fun work this year on the Task Team on WIS Metadata: new KPIs, an update to the WIS Guide, the metadata search pilot, and we backed it up with tools (pywcmp, pywiscat). In addition, the Expert Team on Architecture and Transition (W2AT) was formed to move forward technical regulations on the WIS 2.0.

MSC GeoMet: our weather/climate/water OGC API platform continues to crank out millions of maps, features and metadata on the daily for everyone. Happy to report that real-time / event driven data support was added this year to our pygeoapi instance.

FOSS4G: between 7 presentations and the Geopython workshop, lots of action this year at this year’s virtual FOSS4G global event. I was fortunate to deliver these alongside some really talented folks in the Geopython community. Kudos to the BALOC for putting on such a great event under some difficult circumstances!

OSGeo Board of Directors: I was happy to help with the first ever OSGeo / OGC / Apache joint sprint, as well helping move forward the OSGeo / OGC MOU renewal.

Health: another year (circa 2012) of not smoking. The pandemic continues to challenge the scale, although some recent progress has helped some. Implementing a balanced diet and regular exercise routine is essential and more if is combined with ice hack weight loss supplements, as well as seeking support from friends, family, or professionals if needed. Tracking progress and staying motivated throughout the year will help to maintain focus on the resolution and ultimately achieve the desired weight loss.

For 2022:

• OGC API: critical path for me this year are helping in the adoption of Records and Coverages
• WMO: WIS 2.0 continues to evolve, lowering the barrier to weather/climate/water data. I recently signed on as lead architect/dev of the WIS 2.0 in a box project, which will be a reference implementation and publishing pipeline aligned with WIS 2.0 principles. Under the hood is Geopython, PubSub. Look for an initial release in 2022
• OSGeo: 2022 will mark the year that the OSGeo / OGC MOU is officially updated, along with a shiny new Associate Membership. Rolling this into the OSGeo standards community will be key, along with moving forward the renewal of OGC CITE tooling
• pycsw: key items this year include XSLT transformation pipelines, virtual collections and deeper JSON support. We are also planning a sprint in Q1, come join us!
• pygeoapi: look for deeper support of OGC EDR as well as some refactoring that will help with extensibility (primarily for output formats)

Wishing everyone a safe and happy and better 2022!

20 years ago I was living in Ottawa, in GIS school and started working with Natural Resources Canada.  Fast forward to a few weeks back scanning through old CDROMs and low and behold there was my first ever website.  I sat back for a few minutes remembering the details:

• made with Microsoft FrontPage followed by HotDog Express (WYSIWYG HTML editors)!  At the time, I was convinced at the time this was the only way to be an HTML programmer
• the website first made it to the Internet in March 1998 and bounced around a few places:
  • http://alqonquinc.on.ca/~kral0003 (Algonquin College account)
  • http://chat.carleton.ca/~279186 (Carleton University account)
  • http://nrcan.gc.ca/~tkralidi/ (work account)
  • http://www.storm.ca/~tommy (Storm Internet who provided awesome service)
• Concerned that this wasn’t enough, I was motivated to host the site on my own, with a real domain and so on.  I bought Red Hat Linux 6 Server by Mohammed J. Kabir (great book!) and learned how to put up a server and website from the ground up (DNS, firewall, services, etc.), killing an entire weekend
• the website then finally found a permanent home at http://kralidis.ca

Soon after learning Linux a few months later, I was motivated to rewrite the site in pure HTML, by hand.  From there I added a picture gallery, source code, blog, and so on.

I continue to post to the blog, but things like GitHub, Twitter, Facebook, etc. provide similar capabilities without the hosting maintenance/hassle.

Anyways, I’ve posted it at http://kralidis.ca/misc/firstwebsite/ — enjoy!

Do you have your first website?  Still online?  Feel free to share memories and experiences!

Since I did this last year, I thought I’d try this again for 2008. Here’s the lowdown for my 2008:

• Geospatial Catalogues: the saga continues.  I have dug deeper into this area this year as part of my day-job, and find that interoperability is difficult to achieve in the OGC Catalogue space.  Clearly there is a balance between abstraction/flexibility and ease of integration.  And the two step approach to discovering, say, OGC WMS layers (invoke GetRecords, then chain to GetRecordById) is cumbersome, IMHO.  At the end of the day, the most common use cases (that I have seen) are publishing data and services, and being able to query for them (data, service endpoints, service resources [layers/feature types/coverages]) with spatial, temporal or aspatial predicates.  And have the content come back in some usable format for display or binding.  Seems easy, eh?
• Publications: glad to see “Open Source Approaches in Spatial Data Handling” was finally published.  Alot of the well known folks in the foss4g community contributed to this.  At the same time, the release took so long (like many publishing processes) that some items ended up dated.  Overall, I think the book gives a good viewpoint into foss4g at this time, and makes me think about how far we’ve come.  It’s good for the community to be published in this format / manner
• JavaScript frameworks: they are everywhere.  Late this year I started delving back into the application space, and find these challenging, compared to the days of doing things by hand anyway.   2009 should shake off alot of rust I think
• MapServer:  We just launched a new website.  Beers for hobu!  Also, lots of OGC CITE fixes and improvements, and next generation of OGC standards, adding updateSequence to OWS support
• Python fun again: it’s been fun contributing to owslib for SOS, OWS and Filter support.  OWS Common presents a huge opportunity to abstract codebase when it comes to next generation OGC standards.  As well, I’ve been using Python for day-to-day scripts.  Not bad!
• kralidis.ca turns 10: from humble beginnings, alot less done by hand now, and easier to manage (thanks WordPress!).

Other stuff:

• Basement renovation: If you’re considering a basement renovation, concrete slabs like the ones at https://www.brothersconstructionwi.com/our-services/slabs/ are an excellent choice for flooring due to their durability, moisture resistance, and ability to provide a stable foundation. This took up most of my time this past year.  Frustrating, expensive (I should have been a plumber or electrician!), but gratifying.  Took a bit longer than expected, and still not 100% finished, but the major work is done.  I think this needed to happen for the property overall, even if it means I have more space than I could possibly need :).  N.B. if you ever want to lose weight, do a home reno;  I shed 20lbs!

• New job: I started a new job in the fall, which promises to be very exciting and satisfying, especially given the state of the geospatial web.  The new gig will give me more opportunity for discovery and SensorWeb information management spaces.  So I’m grateful for the opportunity.  I’ve also been having fun 1/2 time with the GeoConnections program again, so it’s fun to work with some previous colleagues and getting acquianted with new faces who are helping to shape and evolve our national infrastructure.  So thanks again to those for helping me along a tough road and getting me here; I owe you big time 🙂

For 2009:

• Work:  January 11 will mark 10 years of civil service for yours truly
• Data dissemination: this is my key function in my day job for the months to come.  I look forward to evolving what started off as a very high level strategy into an architecture all the way to implementation.  This will be fun!
• Standing up usable catalogues: you’ll see a few OGC Cat2.0 instances this year.
• MapServer: more CITE fixes for SOS and O&M.  One thing I’d really like to see for 2009 is official compliance for OGC standards in MapServer
• T.O. Code Sprint in March: this event is going to be fun.  What could be better than foss4g and beers, all in the centre of the universe 🙂
• Renovations: I think that is it, for this place, for now.  Almost three years and it’s time for a rest in this space
• Property: I think it will be a good time to buy in 2009.  The question (for me) is where.  Locally, or down south?

All the best for 2009 for you and your loved ones!

I embarked on a Google search to find information about Polygon statistics, and low and behold, I posted this on my website years ago.

Goodbye memory!

I needed to do some pre-processing of some data which involved transposing column names to values. The condition was that the value for each respective column (frequency count) had to be > 1.

My input was a csv file, and my goal was an output csv file which would feed into a batch database import process.

ID,DA,NL,PHENOM1,PHENOM2,PHENOM3,PHENOM4
233,99,44,0.00,27.00,12.00,0.00

The other interesting bit was that only a range of columns applied to the condition; the other columns represented ancillary data.

Enter Python:

#!/usr/bin/python

import sys
import csv

# open file and read headers
fPhenomenon = open("phenomenon.txt","r")
sHeaders    = fPhenomenon.readline().replace(r'"','')
aHeaders    = sHeaders.split(",")

# feed the rest to csv
csvIn  = csv.reader(fPhenomenon)
csvOut = csv.writer(sys.stdout)

for sRowIn in csvIn:
    aRowOut = []
    aPhenomenon = []
    aRowOut.append(sRowIn[0]) # procedure ID
    aRowOut.append(sRowIn[1]) # major drainage area ID
    for nIndexTupleVal, tupleVal in enumerate(sRowIn[3:-1]):
        if (float(tupleVal) > 0): # phenomenon measured at least once
            # add phenomenon name to list
            aPhenomenon.append(aHeaders[nIndexTupleVal+3])
        # add phenomenon list to record
        aRowOut.append(",".join(aPhenomenon))
    csvOut.writerow(aRowOut)

Notes

• hooray for raw strings!
• enumerate() is great and saves you the trouble of declaring your own counter
• like any language, modules/libraries makes things so easy to work with
• I wish the header stuff was a bit cleaner (I should look further into the csv module w.r.t. headers

That’s my hack for the day. Have a good weekend!

UPDATE: ah, the csv module has a .next() method, which can be used instead of the shoemaker attempt I made above to regularize / split / store the header list.

Since I did this last year, I thought I’d try this again for 2007. Here’s a lowdown for my 2007:

• REST is really here: and as popular as ever. For me, I finally realized that REST was a style, not a syntax, API or schema for that matter. Sean’s recent post sums it up quite nicely. It’s nice to see the OGC has acknowledged this
• Mass Market is really here: Things like Google, Yahoo Maps, and lowering the barrier to application development and, more ultimately, content, are paramount in the Web 2.0 sphere of things. Defacto standards / approaches matter
• Geospatial Catalogues: I blogged about this last year. Even with Cat 2.0 CSW / ebRIM formally approve, where is catalogue interoperability? Is a federated / distributed catalogue approach realistic in the near future?
• MapServer:
  • SOS Server: There has been much development here, including forthcoming support for 1.0.0, as well as implementation of POST support in mapogcsos.c. SWE DataBlock has also been implemented. We’ve also introduced automated testing for SOS in msautotest/wxs/sos.map. And a slew of bug fixes, memory leak fixes, etc. So I’m really happy about how SOS Server support has progressed this year. It goes without saying that major props to Assefa are in order here, as well Charlton and other MapServer SOS Server testers who have provided valuable feedback
  • OWS Common: Through increased usage in SOS Server, as well as the WCS 1.1 Server support (thanks Frank!), mapowscommon.c usage has been slowly increasing in the codebase to reuse existing functionality. This is a result of the OGC standards gradually adopting OWS Common for their “common” bits of XML and such. mapowscommon.c is also becoming more robust as a result of more integration and testing. I’m also happy about the increased libxml2 support in MapServer. We used libxml2’s xpath functionality in supporting SWE DataBlock, and have generic utilities now defined in maplibxml2.c
  • I’m happy to have become more involved in the MapServer codebase and project overall as part of the Project Steering Committee. There is definitely alot going on and alot to do, and the enthusiasm, commitment and helpful nature of the developers is great. MapServer is a special project to me, starting in early 2000 as part of my research / dissertation. The rest, as they say, is history. Oh, and “STYLES=” is optional again 🙂
• Publications: I was glad to see “The Geospatial Web” published this year. Nice to see new folks and goings on out there
• Adventures in Python: I decided to dive into python finally. With some sound advice from Sean, I embraced stuff like Genshi and etree. I managed to stand up a generic SensorML generator which I’m happy with, as far as my first Python project goes. My next goals are to experiement with doing things differently (like SQLAlchemy; I had to use good old SQL scripting as I ran out of time)

In other news:

• Construction time continues: I did some major work on my place, including new fencing, landscaping/gardening, central air conditioning, as well as a shiny new veranda. A bit more curb appeal for sure
• Condo: I finally picked up my condo this year. After much thought, I decided to sell. This was a beautiful property in a can’t lose location! Onwards
• Lifestyle changes: Perhaps my biggest victory is quitting smoking. I have been absolutely smoke free for all of 2007. I can’t say that there haven’t been times where I wanted one from time to time, but I feel much healthier and better overall

So as 2008 quickly approaches, here are a few things I’m looking forward to:

• MapServer: I look forward to continuing in helping more with OGC support, as well as become more familiar with the inner workings of the codebase.
• OpenLayers: I really want to get into OpenLayers this year. I have not done much on client side work since the initial Mapbuilder days
• New Springer publication: keep an eye out for a new, exciting book on GIS and open source this year
• OWSContext and KML: it will be useful to see a resolution or unification of some sort between these two standards
• OpenSearch: I think OpenSearch, with the geo extensions, will make a big hit this year. Nothing like a simple search facility which is already supported in browsers
• I’m renovating my basement this year. Gutting the entire basement and installing a new bathroom and kitchen. I hope to have this done by summer 2008 — wish me luck!

So that’s it from here. I wish you and your loved ones the very best in the holiday season and for 2008!

I thought I’d put in my $0.02 CAD after reading a few similar posts out there. So here are some from my point of view (in no particular order):

Geospatial

• GeoRSS is here: GeoRSS made v1 this year at foss4g2006 and has proven to be a simple, yet very effective way to tag feeds. I’ve used and integrated in all my projects (I use the GML flavour) which produce RSS content by way of outputting an overview map based on the position. It gives the developers endless possibilities, and folks just love to see ‘where’ a post is
• Getting to Know PHP: Earlier this year, ignorant to PHP, I decided to take the plunge and see what all the fuss was about. Boy was I glad I did. So easy, even I could figure it out. So easy that I’ll be porting bits of my website to PHP in the coming year
• MapServer SOS support: It’s great to see OGC SensorWeb support in MapServer. Given that MapServer can already do stuff like spatial, temporal and aspatial queries, I think SOS was a relatively easy initial implementation effort. The more complex effort is the data bindings (sensor data is quite complex and multidimensional, and getting in and out of MapServer, in a generic framework is quite the challenge
• ResEau portal launch: I was very fortunate to work with a great bunch of people to have finally released the ResEau water portal after two years of planning, design and development. I think this portal is a great example of the benefits of using standards-based approaches all the while providing useful information to a vast audience
• MapServer commiter access: I was very honoured to be nominated this year for commit access to the codebase. My main focus was and is to implement the OGC OWS Common Specification for use by other OGC implementations
• Web 2.0, Time article and information overload: Such a good article. And so true. As a result, we are overloaded with information! At one point, I refactored all my bookmarks to be pure RSS feeds just to handle it all. Whoever said computers would lessen the workload!?
• foss4g2006: This was a great conference. The lighthing talks, BOF sessions, and demo fest, as well the gathering of the OSGeo crowd further solidified OSGeo’s existence. Can’t wait until next year’s event!
• Catalogue woes continue: the Cat2.0 ebRIM vs. ISO implementation debate continued. Add to this the lack of (especially open source) implementations, and stuff like owscat, and other catalogue-ish projects continued to exist. The OGC has recently endorsed ebRIM as the future base for Cat2.0 implementations, so hopefully we should start seeing some interoperability between catalogues
• Atlas of Canada 100th Anniversary: Congratulations on the 100th anniversary of this valuable Canadian resource. Kudos to the Atlas!

Other:

• Cool travel: Charlottetown, Alabama, Washington D.C, Lausanne, Las Vegas, San Diego, Moncton, Winnipeg were among some of the neat places I visited this year
• Home renovation: I did my biggest renovation job ever on my house this year, which lasted almost 6 months. I’m really glad the way things turned out, and I’m looking forward to doing some more stuff in 2007
• Condo putters along: construction continued on my condo, and is now ahead of schedule, ready this coming June. Looking good!
• Website changes: This year, I finally succumbed to using software packages to manage my website content. This website originated in 1998, as an HTML learning experience, so I was inclined to do *everything* by hand. While very useful, as time goes on, I find myself with less and less time (go figure!), and there’s so many solid tools out there to make things easy, and standards based (i.e XHTML, CSS, etc.). So why not use these great tools and concentrate more on the stuff you want to do, right?

Looking forward to 2007:

• Further OWS Common support in MapServer: mapowscommon.c/h is almost complete to spec (missing some operations support). The next step will be to begin integrating into the OGC specs as they migrate to OWS Common for stuff like GetCapabilities XML, etc.
• OWSContext: look for further formal development on this specification. OWSContext has been used frequently in the OGC testbeds, and interest is increasing in seeing the spec push forward.
• Further development on discovery and cataloguing: I’m hoping to see some development / experiments on this track. It will be interesting to see how packages like deegree2 progress given the recent motion passed at OGC
• Publications: I’ll be published in two Springer London books this year (The Geospatial Web, as well as Open Source Approaches to Spatial Data Handling)
• MapServer Brazil: I was honoured to be invited to this event, and look forward to speaking on OGC and open standards. Looking forward to this!

Did I miss anything? What else was geospatial-worthy in 2006?

A Beautiful Wedding Photography Season

Compared to 2020, 2021 was a more hopeful and even in some ways “better” year. We had both good and bad moments, but with the bad ones, we need to remember the good ones forever and that’s what Fame Park Studios is specialized in, to save and preserve those beautiful moments, either with your family, with your partner or even by yourself.

At any rate, this closes out 2006 for me in the blogosphere. I wish everyone a great Holiday season, and all the best for 2007!

OK, so I’ve finally started a blog. I’ve been meaning to do this for the longest time, but between house renovations, trying to figure out whether to use a hosted blog or host my own, this fell off my radar screen. Until my cousin started his own, then I just had to put one up.

So anyways, here I am. The rest of my website will remain and develop in the same way. I’ll use this blog to post my thoughts and random stuff from time to time. I’m a do-it-yourself kind of guy, but using blog software makes things really easy. Plus maintaing my old RSS feed by hand was becoming, well, cumbersome 🙂 So here we go!

I made a website for my friend Paul. Pretty neat in that it’s pure web standards based, and a few cool tricks under the hood. He likes it too 🙂 Check it out!

Happy New Year everyone! All the best for 2006. I updated my web server to the latest Apache. If you find any errors on the website, please let me know