A Week in Ottawa: Partying With the Data Nerds

Me, to senior staffer from Stats Canada:

Given the red-hot Open Data movement, you must be a rockstar at parties now. What's that like? 

Senior staffer from Stats Canada:

*smiles softly* We've been popular for a long time. Son, you've just been going to the wrong parties.

In late May, I attended several conferences in Ottawa, regarding Open Data. This is a (long overdue) review. (Or perhaps highlights. It is now high summer on PEI, so I don't know how many will read this.)

I attended the Canadian Open Data Summit, CKAN Con, and the 3rd International Open Data Conference. Later on, I might write about CKAN Con for PEI Developers (as it is a software platform). This post is a mix of #CODS15 and especially the huge #IODC15.

Motivation

I'm a software developer. I've been interested in Open Data for some time, on the side, and guessed that attending #IODC15, and satellite conferences, in Ottawa would be a special opportunity, given its relative proximity. (I had no idea: as it turns out, the 4th IODC will be in Madrid.)

But that was a relatively small surprise. The larger surprises included: the size of the flourishing Open Data community, world-wide; the tremendous activity within Canada; and the duality that Open Data, as a movement, is both mature and adolescent, simultaneously.

Another surprise: my preconceived notions about Open Data and its practitioners were often wrong. See the opening quote. Though everyone was very friendly, I was definitely in foreign territory (see Big Tent below). It was intimidating, yet exhilarating.

And so, the highlights below are simply one nerd's observations, offered with a grain of salt.

Inside the Shaw Centre, at #IODC15

Open Data is a Big Tent

During the week, I was surrounded by an improbable mix of statisticians, policy wonks, map geeks, power brokers, social activists, vendors, librarians, entrepreneurs, academics, and technocrats. Software nerds were a minority.

Everyone knows about smart cities, but did you know that there is an open data movement in agriculture? In education? An Open Contracting Standard (re: government contracts)? 

Or that some estimate the economic impact of Open Data in trillions of dollars?

Open Data is a Small Subculture

From what I can tell, that trillion-dollar report above has become a drinking game in Open Data: every time the report is mentioned, take a shot.

Practitioners seem pleased with the gains of Open Data, yet frustrated that it hasn't grown more. The attendance was large for the conferences (1500-1800 for #IODC15), but this was mitigated by the sense that the entire Open Data Universe was often in the same ballroom. Some panelists wondered if Open Data is insular (versus mainstream).

Canada

Along with the UK and USA, Canada has a major footprint in Open Data. Federally, the subject falls under the Treasury Board. Resources include: the Open Data Portal (which includes a "no wrong door" plan to integrate with provinces and municipalities), the CODE hackathon program, and the recently announced Open Data Exchange program.

Provincially, many provinces are actively involved in Open Data. One of the leaders is British Columbia. BC had a strong turnout to Ottawa, and some fascinating ideas about engaging software developers to create apps (the critical next step after releasing data).

At #CODS15, the City of Edmonton won the Canadian Open Data Award.

Outside of government, there are many enterprises in the space. Check out the sponsors for #CODS15 (thanks in particular to Open North, which arranged the conference and co-ordinated accommodations).

Electrifying Moments

Though the mic didn't pick up the ambient vibe of the room, Sam Pitroda was a delight on the "Open Data Around the World" panel at #IODC15. This quip, "we're solving the problems for the rich, who don't really have problems" was a bolt of lightning. The panel went 15 minutes over time, and no one complained. It was fabulous.

Mark Headd, when in Philadelphia, was a pioneer as one of the first Chief Data Officers in the world. At one session at #IODC15, he mentioned that "there was no template on how to do this"... a welcome sanity-check. His team chronicled their experiences in the Open Data Handbook.

At a panel at #CODS15, Pamela Robinson asked (paraphrased) "what if HR performance reviews for government employees accounted for 'open data' activities? ". *goosebumps* Reward the rebels on the inside! Mind blown, I hammered out a tweet ASAP.

DJ Patil proclaimed that PDF is not open data. Again, the microphones fail here, as the ballroom heartily cheered this point. (I was shocked, thinking it was already settled law! See Open Data is a Small Subculture above.)

DJ also described the mission statement of Team Data. The top paragraph is pithy, indeed; there is a 10 minute talk, here, simply in deconstructing it (Responsibly! ROI!). Fantastic.

The value proposition: we already invest in data, so let's max ROI.

Photo by Mike Gifford, under CC BY-NC-2.0

A final highlight was a #CODS15 lightning talk by Catherine McGoveran on Ottawa's Open Data Book Club. The club is regular meet-up which focuses -- not on a book -- but a specific data set. Who needs War and Peace when you have Construction, Demolition, and Pool Enclosure Permit Data ?

I loved especially that the vibe is a counter reaction to "hack event fatigue" (!). Hackathons are fine, but there must be other options, and this one is highly intriguing (see Call to Action).

Alex Howard & The Honourable Tony Clement

If possible, set politics aside, and watch Alex Howard ask The Honourable Tony Clement about the long-form census, in front of 1000+ data nerds. Yikes. There was a palpable ripple through the crowd.

There'll be fireworks, n'est-ce-pas?! What happened? Well, Mr. Clement answered the question, and they moved on. You might agree, or not: I only mention it because the sharp-yet-civil discourse made Question Period look like a three-ring circus. Alex Howard is a terrific moderator.

Call To Action

At the end of #IODC15, and the culmination of a week's worth of conferences, a speaker made an effective pitch for a call to action. Here's mine, and I hope to write more about it in late summer.

Thank you

I've comfortable attending (and speaking) at software dev conferences. The week in Ottawa was a new ballgame. Many people were very kind, but special thanks to Greg Lawrance, Bianca Wylie, Chad Lubelsky, and especially Open Gov advocate Richard Pietro: thanks for the genuinely warm welcome.

With your support (and the kindness of a senior staffer at Stats Can), I felt I was at the right party.

M.

p.s. Here is the full webcast archive for #IODC15.

What is Open Government? (part 3)

This is the third post in a series on Open Data and Open Government. I'm assuming the audience is not technical, but has heard these terms used in the media. See the first post for more about me and my motivation.

Disclaimer: I've monitored the Open Data space for some time, and I'm a software professional, but I'm not an expert (yet).  I am passionate however, attending the 3rd IODC in Ottawa in May as vacation. I'm also non-partisan. Though I hope to participate in a dialogue at the municipal and provincial level, I'm not endorsing any party or platform.

Prelude

As I've blogged these posts and talked to people about them, I've realized that my intended audience include my friends and family: i.e. thoughtful Islanders who may not be familiar with these ideas. Because this series is intended to be conversational, rather than academic, it grants me some leeway in terms of definitions and historical record.

That's good, because Open Government is slippy.

Open Government 1.0

According to the dreary Wikipedia page, the basic principle of Open Government (Open Gov) is that citizens must have access to documents, proceedings, and other workings of government. The language conjures up images of Freedom of Information laws, Sunshine legislation, Open Meeting protocols, and so on.

These ideas are certainly important, but they've been around for a long time. e.g. Canada passed the Access to Information Act in 1983. Yet now, provinces such as Newfoundland are promoting Open Government Initiatives, and terms such as accountability, transparency, collaboration, and civic engagement are all the rage.

How did we get here? Why all the fuss?

Open Government: The Next Generation

There has been a second wave of Open Gov. This wave was initially called, in some circles, "Gov 2.0", but appears to have settled on the term "Open Government". That's confusing, because this is not our parents' Open Government! It's modern, energized, and inextricably linked to the Open Data movement and Open Source culture described in previous posts.

Here's a timeline of landmark events (at least in my journey... there's that leeway again) of the second wave:

• US President Obama's Open Government Initiative, issued in 2009. Its philosophy and language specifically mentions "transparency", and government being "participatory", and "collaborative".
• The book, Open Government: Collaboration, Transparency, and Participation in Practice. Published by tech darling O'Reilly in 2010, the book is a collection of essays on a new era of Open Gov. Highly influential in terms of motivating techies to enter "civic hacking": that is, using software skills for the public good.
• Online evangelists such as Tim O'Reilly and Beth Noveck.

In 2012, Canada joined the Open Government Partnership, which specifically mentions transparency and civic participation. Today, there's many Open Provinces and Municipalities in Canada. 

By 2014, these ideas have become a full-fledged movement, as demonstrated by Richard Pietro's Open Government Tour (and subsequent podcast). During Open Gov 1.0, did anyone ride across Canada on a motorcycle, to talk about civic policy?

Generation O

Here are some characteristics of the new Open Gov, that unites ideas from Open Data and Open Source:

Freedom-of-Information requests are great, but it's better to publish info proactively (within privacy constraints). Open Data is more efficient, and clearly heightens accountability and transparency. e.g. Imagine a world where federal Senators' expense reports are published online, in a standardized format.

Government should borrow tools from Open Source in the tech world. There is a virtuous circle where, first, government opens data, enabling software developers to create apps or visualizations. These apps attract the interest of citizens, and shine light on phenomena that impact their lives. (Alternatively, artifacts could be published to sites like GitHub and encourage an open, 2-way dialogue.)

Either way, the citizens are informed, provide feedback to government, and the circle is complete. This is truly civic engagement and collaboration. Critics might claim that no one really cares about the Paris budget, but as with most things, it is simply a matter of finding an issue that resonates (techies call this "scratching an itch"). You may not care about financials. Fine. Perhaps you'd like to avoid getting sick at a sketchy restaurant?

The Answer is in the Question

When someone in government -- be it municipal, provincial, or federal -- mentions "Open Government", our first question should be (replete with air quotes):

Do you mean "open government" (1.0) or "Open Government" (2nd wave)?

What is Open Source? (part 2)

This is the second post in a series on Open Data and Open Government. I'm assuming the audience is not technical, but has heard these terms used in the media. See the first post for more about me and my motivation.

Disclaimer: I've monitored the Open Data space for some time, and I'm a software professional, but I'm not an expert. (I am passionate however, attending the 3rd IODC in Ottawa in May.) I'm also non-partisan. Though I hope to participate in a dialogue at the municipal and provincial level, I'm not endorsing any party or platform.

What is Open Source?

The term Open Source is often used in conversations around Open Data and Open Government, but it is rarely explained. In my view, the concept and especially its culture form a critical bridge between Open Data and Open Government.

Rather than struggle through definitions, let's consider a non-geek example that will be familiar to residents of PEI.

Fruitcake Recipe

Fruitcake has a long tradition on The Island, at weddings and Christmas, and my parents' house is no exception. My mother's fruitcake has won several awards and many raves; for decades, I've seen many people beam with delight at receiving one of hers.

I've also heard, often, that baking one ain't easy. The recipe was handed-down by a dear, elderly neighbour, Granny MacPhail, decades ago. There's a long list of ingredients. Some are obvious, such as many types of raisins and nuts etc. Others are more involved: e.g. fresh-brewed coffee and homemade strawberry jam! The preparations start on the night before, and the fruitcake must bake for 3-4 hours. Though the odds improve with experience, there's simply no guarantee on how it will turn out.

What if ... we went 'open source' ?

As a thought experiment, what if my mother were to share her recipe with the world?

If she published the recipe online, it's likely the following would happen:

• Others would benefit by trying the recipe for themselves. They might try it outright, or use it as inspiration for new ideas.
• If they contributed ideas back to my mother, she might incorporate them into her own recipe.
• In any case, it's unlikely that anyone would match my mother's baking, as they simply don't have the experience. That is, opening the source doesn't really compromise her talent, or her potential to make the best fruitcake around.

Now, to be honest: my mother will never publish the recipe. She from a generation where such things are guarded secrets. (I'm working on her, but there's little cause for hope.) Either way, the example gives us enough material to define some terms.

Open Source

In the parlance of Open Source, the fruitcake recipe is, you guessed it, the source. The act of publishing it online makes it open. (The current state of Mom's recipe is called closed or proprietary source.) 

As with Open Data, the source is published with a license that dictates the terms of use. For example, my mother may insist that any recipes derived from hers are also published openly and so on.

Publishing & Collaborating

There are many ways to publish the recipe as open source, but these days, the king is a site called GitHub. GitHub is a darling with software nerds, and is used for a lot more than just software code (including recipes,  the US Budget for 2016, and -- spoiler alert! --  legislation proposals).

GitHub is built on top of a technology called Git. Together, they allow a collaborative workflow where:

• Alice publishes the source
• Bob copies the source and works on it independently
• Bob can offer changes back to Alice
• Alice can accept and incorporate the changes into the original work

This workflow is very common in Open Source and differentiates it from Open Data. With a dataset (such as weather info), we don't often contribute back: we consume and build upon; with source, we can collaborate in a 2-way dialogue.

Got it. But is this cyber-communism?

Nope. Many open source nerds want to maximize what is open, but it's not a requirement. e.g. GitHub, itself, is not open source (!).

And there's nothing preventing anyone from making money by adding values via services, advertising, and so on. I've already mentioned that my mother could go open source without fear: no one can match her decades of experience. In fact, if we were to start a fruitcake business, it might be a clever marketing campaign: "here's the recipe... Betcha can't beat our product!".

OK. When will you get to Open Government ?

Soon. The reason I wanted to introduce Open Source is because of the 2-way dialogue and collaboration. In the Open Gov space, this collaboration goes by another term: civic engagement.

If you're interested in chatting more about Open Data/Gov, please contact me at codetojoy @t gmail dot com

More on Open Data

A couple of notes on Open Data in Canada, as a follow-up to Part 1.

In this episode of the Open Government Podcast, near the 3:30 mark Tracey Lauriault mentions the following situation, back when the government charged $ for information (near late 1980s):

"... a decade of lost research on Canada. Because census data were so cost-prohibitive at that time, Canadian faculty and Canadian students became experts on the US because data there were free."

I have nothing to add. Just let that sink in.

Also, in this session (below) on Open Government in Winnipeg, Mary Agnes Welch describes the frustration of receiving PDF files (near 47 min 24 sec).

Although PDF files are "open" in the sense of disclosing information, they aren't machine readable. We can't easily write a program to use the data within, and so they're not in the spirit of the Open Data philosophy.

What Is Open Data? (part 1)

I'm fascinated by the movement known as Open Data and its cousin, Open Government. In the spirit of International Open Data Day, I thought I might contribute to the conversation by explaining some topics (as I understand them) for newcomers.

This post is an informal view from 10,000 feet, painted in broad strokes. A future post will address Open Data on Prince Edward Island, and my hope for a virtuous circle regarding software mentorship and civic engagement.

Disclaimer: I've monitored the Open Data space for some time, and I'm a software professional, but I'm not an expert on this. (I am passionate, however: an upcoming vacation is the 3rd IODC in Ottawa in May.) I'm also non-partisan. Though I hope to participate in a dialogue at the municipal and provincial level, I'm not endorsing any party or platform.

That said, let's go!

What is Open Data?

First, let's tackle data. It's so prevalent, it's tricky to define: data is just information (usually in files or a database). Data can be as mundane as noise-nuisance complaints in the UK, or as modern as disease reports by Ugandan volunteers (via text-messages) on banana crops.

The Open Data movement views public data as a resource that should be, well, open. That means it has these characteristics:

• freely available to all
• machine-readable format (e.g. simple text files)
• explicitly licensed for any use (e.g. Creative Commons)

Some counter examples:

• a bunch of PDF files isn't open, because nerds can't write programs to read the data
• similarly, map data published in a format that requires expensive software isn't open
• a set of simple text files is much better, but if it isn't licensed, then the data might be copyrighted (by default), which is a problem

The data for a given domain (e.g. crime reports for the last 7 days in Halifax) is called a data set. Data sets are published in a data catalogue as part of a larger website (often called a data portal). 

What's the big deal? What can anyone do with noise-nuisance reports?

A data set may seem boring to you and me, but in the right hands, it might prevent us from getting sick.

It's a big deal for software developers, because we're always looking for new ways to improve our technical skills. But trivial exercises (e.g. calculating prime numbers) are boring; and writing a game is a massive undertaking (good luck trying to come up with something "new"). By contrast, the possibilities for data-oriented apps seem endless. Even better, nerds have discovered that people find these apps useful. That's rewarding.

Open Data is a big deal for app users, because we can effect change (or offer fresh perspective) in our real-life community. Consider Ben Wellington's story, as he helped fix a confusing parking space in NYC.

This is a big deal for citizens, for many reasons, including accountability & transparency of public officials. We can have a debate about the role of government (less vs more and all that) in society, but no matter where you reside on the political spectrum, who argues against shining sunlight on data? (You can already see the close relationship between Open Data and Open Government).

All of these reasons culminate in a symbiotic relationship among the actors, and explain the push to "liberate the data".

Aren't there privacy concerns? What about my health records?

Good questions, but don't worry. The Open Data movement isn't about creating an anarchist society where there is no privacy. Not all data should be open, and thoughtful practitioners take measures to ensure that user-privacy isn't compromised.

So, that's good news on the movement at large. As for any given project, it's an important question. Those responsible for a recent data release by the Governor of Florida are guilty of "data malpractice" (my term).

Is this cyber-communism?

Nope. Though the data should be freely available, there's a wide array of players selling software services in this space. Granted, there is a certain esprit du corps for writing open-source software, but it doesn't appear to be a requirement per se. In my view, this seems fair: if someone adds value by writing an app, they can reasonably charge for it, so long as there are no impediments to the data itself.

(Edit: Sameer Vasta makes excellent points in this episode of the Open Government Podcast. He and Richard Pietro offer weather and GPS data as examples: when the government opens the data for use, it enables competition and innovation in the marketplace. Far from communist, that's pure capitalism, if not libertarianism.)

Open Government, Open Source, etc

For now, I've decided to cover these topics in a future post, if there is interest.

If you're interested in chatting more about Open Data, please contact me at codetojoy @t gmail dot com

Benefit Ceilidh for Sherry White

My cousin, Sherry, is a life-time resident of Hampshire, PEI. She's battling cancer, and requires multiple trips to Halifax for treatment. She also requires medication and medical supplies that are not covered by the province.

As our family will attest, she is the kindest person ever. Witty, super-positive, and has a heart of gold. (She also collects snowmen, re: graphic.) Naturally, we're having a benefit in support.

You're invited! Please join us at:

Central Queens United Church

Friday, April 4 at 7:30 pm

The ceilidh will feature local talent with a bake sale and/or auction at the intermission, with a light lunch provided.

If you can't make it, and would like to donate, there are donation jars at:

Clow's Red & White, Hampshire (aka Bobby Clow's)

Pedro's Island Eatery, Cornwall (aka Maggies)

Thanks so much. Our family is truly grateful.

M.

ps. Snowman art credit: here

Jewel's Audio Memory Game - An Android App

My old pal, Vic Douse, has a daughter, Jewel. She loves Disney, balloons, and some video games/apps. She has autism.

Vic works with databases during the day, but at night, he has been very active in the autism community on PEI, serving as president of the Autism Society for years. (He won a Queen's Diamond Jubilee medal for his contributions).

Last winter, he started tinkering around with an Android app, despite not coding for years. The result is a simple memory/match game. The UX is simple, and that's OK: the point is that Jewel likes to play it. It helps her with her vocabulary.

So, check out Jewel's Audio Memory Game on the Play Store. It's free with no ads. Vic is delighted with each download, so please give it a go!

It's actually fun in a nostalgic way. My personal best for the 4x4 grid is 36 seconds, no doubt based on lucky strikes.