PlanningAlerts API changes

Starting today, we’re beginning to roll out a change to the way the PlanningAlerts API works. We’re introducing API keys for all users of the API from low-volume non-commercial users to high-volume commercial users. We are making this change so that we can better measure and understand the way people are using the API as well as give us the ability to contact all API users in case of any issues (or changes we make) that might affect them.

We’re phasing in the introduction of API keys. Right now they are optional, though as a new user of the API you will be guided into using them.

On 1st of June 2015 the use of API keys will become mandatory

3 months from now on 1st of June 2015 the use of API keys will become mandatory.

This means if you are a user of the API today, you will need to move over to using API keys before 1st of June 2015. If you don’t do it by then the API will stop working for you on that date.

We’ve made it very easy for you

Thankfully, we’ve made it very easy for you to get an API key.

  1. Simply register for an account on PlanningAlerts
  2. Then visit the normal API documentation page to see your API key
  3. Update your API calls adding &key=[your key]

We recommend you don’t use the PlanningAlerts api via client-side javascript

If you use the API in client-side javascript you will obviously be sharing your api key with the world which is a very bad thing. It is your responsibility to keep the api key safe and secure. If we discover any abuse of a key (e.g. someone else is using your key) we will cancel the key and you will have to get another one.

As a result we recommend you don’t use the PlanningAlerts api via client-side javascript.

Either

  • Switch to getting the planningalerts data server-side where you won’t expose the api key
  • Or proxy the client-side requests through your own server where you add the api key

If you have any questions on how to do this, of course don’t hesitate to get in touch with us.

Posted in Announcement, PlanningAlerts.org.au | Tagged , , , , , , , | Leave a comment

Civic Tech Monthly, Feb 2015

Sailing image by Steve Bennett, used under Creative Commons. See flickr.com/photos/63439615@N00/8000733006

Photo by Steve Bennett, used under Creative Commons

Welcome to the first edition of Civic Tech Monthly breaks champagne bottle.

Today we set sail around the world of civic tech, its peoples and their projects. Civic tech at the moment can mean ‘tech that’s all about citizens exerting and obtaining power’, as well as ‘tech that’s all about improving government services’. We’re following Tom Steinberg’s lead and including them both. We can all explore what we mean here together.

This voyage is an experiment, we’re in uncharted waters, so please jump aboard. Let us know if the links are interesting and what you think about them (better yet let everyone know on twitter). Don’t hold back if you think we’re sailing off course, if you want more—or less links, etc. .

You can also contribute to the newsletter on GitHub, like all our projects.

If you think this is interesting you’re sure to have friends who would too. Please pass on the subscribe link and make noise via the social media tubes.

For those who are able to make it, we’ll see you at the OpenAustralia Foundation Sydney Pub Meet next Tuesday.

Without further ado:

News & Notes

3 ways to keep They Vote For You tickety boo

For the past 18 months Micaela Ash has done an amazing job building up the division summaries and policies on our latest project They Vote For You—it is a vast new resource of how our laws are made. Micaela has now moved on to new, exciting work, so she left us some parting instructions on how we can keep the resource in shape. Surely there are people on this list who could take a few minutes to summarise a division once every few weeks?

Percs: searchable pecuniary interest disclosures

Following January’s #nswvotes meetup, Chris Nilsson whipped up a way to search the text of the previous very inaccessible 2013-2014 NSW Legislative Assembly Pecuniary Interest Disclosures. A huge step forward in the accessibility of this information, and there is more in the works—see the OAF Meetup’s discussion board.

Unlocking Australia’s public data: A catch up with Rosie Williams

An interview with Rosie Williams about the latest work on InfoAus, a site of ‘political and social research tools’ that make Australian government data more accessible. Rosie is currently adding new ways to access data that shows the differences in wealth and poverty across Australia.

Cuttlefish is now an official Poplus component

Cuttlefish is an easy-to-set-up transactional email server developed by OpenAustralia Foundation Founder Matthew Landauer. Last year the OpenAustralia Foundation was awarded a Poplus mini grant to create a hosted version of Cuttlefish which is freely available to the Poplus and international civic tech communities. Read Matthew’s post, Email is your secret weapon, on the Poplus blog to find out more. Now Cuttlefish has been accepted as an official component, more people will find out about and use it.

Poplus is a global federation for civic tech, developing components that can support projects everywhere. Join the Poplus maillist for updates and discussion.

Federal Government announces the Digital Transformation Office

The Government is establishing a “small team of developers, designers, researchers and content specialists working across government … to make services simpler, clearer and faster for Australian families and businesses.” It is an important project for the civic tech community to get involved with and help make it as effective as possible. You can read more about the DTO at the Communication Minister’s FAQ. The UK’s Government Digital Services have also written a blog post about the project.

Growstuff

Growstuff is ‘a community of food gardeners who are building an open source platform to help you learn about growing food, track what you plant and harvest, and swap seeds and produce with other gardeners near you.’

We love this one because it’s a practical open source project helping people grow food and community. It helps keep information held in plants and seeds open by making it easy to share. Growstuff releases the data it collects under open licenses, giving power to it’s developers as well as growing a community of gardeners. It promotes pair programming as the norm.

This is a very active project. Volunteer to help test, write for the Growstuff blog, contribute code to the application or just dig in, grow some food and share your seeds and data.

Nick Evershed injects some context into the news

After Tony Abbott apologised for saying Labor created a “holocaust” in defence industry jobs, Nick Evershed used theOpenAustralia API to provide more context about the use of the term in Parliament.

Hackfarming Blood Buddies

Jeremy Keith takes us through the process of planning, building and deploying a hack to help more people donate blood at Clearleft’s Hackfarm getaway this year.

Their 2012 Hackfarm resulted in a prototype for a personal political opinion tracker—a few people at the OpenAustralia Foundation meetup have suggested similar concepts for projects they want to work on.

Local events for International Open Data Day

This Saturday, 21st Feb, is International Open Data Day. The Open Knowledge Foundation Australia have a list of events for you. Code for Australia’s day of workshops in Sydney tomorrow caught our eye. It is a free event and registrations are still open.

Learning to contribute with Git

Contributing through the Git version control system is an important skill for people working on civic tech and open source generally—but it ain’t that easy to learn and it’s not always clear where to start. We’ve observed that it can be a barrier to people getting involved.

Once you’re rolling, it really is easier than you’d think. You’ll quickly see why it’s so popular. If you learn well with audio, try the Web Ahead Podcast’s Git episode. If video, interactive tutorials or reading works better, try one of the many useful tutorials on Github’s resources list. Once you’ve learned it you’ll have superpowers, so don’t be afraid to ask if you need help.

12 steps toward a better neighbourhood

Henare and Luke visited the Friends of Erskineville community group in Sydney last week. We introduced the group toPlanningAlerts and had a great time finding out about how they’re working to improve their area. In return we picked up this interesting list of 12 steps that they were discussing. These are practical ideas for getting engaged in your local community. Maybe these steps will spark some ideas for your next civic tech project?

Civic Hall launched in NYC

Civic Hall is an enormous space now allocated for civic tech hacking and to house civic tech organisations. If anyone is in NYC, drop in and let us know what it’s like!

Posted in Civic Tech Monthly | Tagged , , , , , , , , , , , , , , , | Leave a comment

Unlocking Australia’s public data: A catch up with Rosie Williams

Photo of Rosie Williams

Rosie Williams has independently made a huge contributions to budget transparency and the accessibility of public data in Australia through her project InfoAus.

It’s been a while since we were able to catch up at the last pub meet so I sent through a few questions.

 

What are you working on at the moment? and what was the path to this point?

I am re-developing my open data project KnowYourPlace to include data visualisations of ABS SEIFA data. The idea is to provide this information on social and economic inequality to the public in ways we can interact with and make use of. KnowYourPlace is the part of my website (InfoAus) that displays political information based on geographical and political boundaries. 

I implemented the original version of this data over the Australia Day long weekend in 2013. This redevelopment is aimed at giving meaning to the data- providing visual context to the data so that users can look at the pages and see what the data means for them.

Another aspect of my work is to push the boundaries on data provided for re-use by the government which forms the basis of open data projects like those on InfoAus. For example, after speaking with the ABS last week about the information needed to filter data by federal electorate, the ABS agreed to publish electorate concordances along with locality information. Previously, federal electorate was only published with postcode information rather than suburb-level information. This is a great step forward.

With the projects I work on it is often the first time data has been presented in ways that allow grouping by political boundaries so it is only with such use-cases that the government can get the feedback that it needs to provide data in ways that provide meaningful analysis for web applications.

Why should people be interested in the ABS’s Socio-Economic Indicators For Areas data? Has that data shown you anything surprising?

The most interesting and sad thing about the SEIFA data is that the lowest percentile of socio economic localities all appear to be Indigenous communities. I got a bit of a shock when I first mapped the top and bottom percentile of advantage and saw Australia’s racial inequality so starkly illustrated.

How have you found working independently on projects?

Working independently is difficult in that I can often go for long time periods without any feedback on what I’m doing. However in the past couple of weeks I’ve really enjoyed creating original solutions to data visualisations that are entirely my own code. Being responsible for both the front and back end of my projects means I have been able to provide custom solutions for the entire project, designing visualisations that provide data in ways that have meaning and that people can interact with. Being a sociologist with coding skills has enabled me to provide this custom solution.

For people wanting to contribute to your projects, or find out more, what is the best way for them to get involved?

InfoAus and it’s projects are the result of what is now years of hands-on work and the university education that preceded them. My passion regarding inequality in Australia began with my own experience of disadvantage and through a degree in Sociology, programming and the years of work and activism that has gone into building the projects I have made real progress in improving political and budget transparency in Australia.

One of the problems facing open data projects is the difficulty in finding a way to support the people who work on them.

In order to continue working on budget/political transparency through my projects, I will have to begin earning money to support myself. I will look be looking at crowd funding and other options for the survival of my projects.

I’ve introduced Google ads to the site and I’ll be looking at doing iphone apps and marketing the databases to libraries. This will probably mean the databases will eventually be put behind a paywall.

What has impressed or interested you from the worlds of civic tech or open data recently?

The world of civic tech had a really important win last year that I became aware of in December. Grants.gov.au is a site being created by the federal government that- when available- will host information from all federal agencies on the community grants they provide. Currently, agencies publish this information in different ways, in different places with no consistency in terms of the information or format. A new specification has been developed for this grants data that requires agencies to provide the data in ways that allow it to be cross-referenced with the Portfolio Budget Statements released on budget night. This is consistent with the requests that I have made to the Department of Social Services regarding their grants funding data so I was very pleased to see it be implemented across the board.

The upshot of this is that grants funding (which will be published with geographical information) can then be matched with budget data and made searchable by geographical or political boundaries. The first implementation of this new specification made searchable by political and geographic input is part of my budget transparency project BudgetAus at http://infoaus.net/budget/funding_by_electorate.php

Contacts that I work with within the government are also making in-roads in providing avenues for representatives of projects like mine to give feedback to for example, the Joint Committee of Public Accounts and Audit which sets out the specifications for agencies to provide information in their Annual Reports. Annual Reports data have implications for budget transparency so this is an important matter to pursue. Most of this current progress in budget transparency is being ignored by the media so it is good to be have this publication in which I can report it.

Posted in Uncategorized | Tagged , , , | Leave a comment

3 ways to keep They Vote For You tickety boo

Keeping They Vote For You up to date can seem a little challenging. So, I’ve prioritised the three most important jobs to make it easier for people to get involved.

If these three tasks are done regularly by enough people, the site will remain tickety boo (which is urban slang for ‘useful and current’).

1. Manage the policies

Firstly, mark any problematic policies you see on the site as draft policies. Problematic policies are, for example:

  • Biased (i.e. too ideological or a case of political point scoring)
  • Impractical (i.e. include too many aspects that should be broken into separate policies)
  • Redundant (i.e. have no relevant divisions)
  • Nasty (i.e. a swipe at a particular party or politician)
  • Silly (i.e. clearly a joke)

Note: if existing policies are turned into problematic policies by cheeky editing, reverse the change.

Secondly, combine any duplicate policies.

2. Edit at least one division

Try to do this weekly, particularly when Parliament is sitting (see the sitting calendar).

This will keep the site nice and relevant.

For a reminder on how to edit divisions, see our blog post.

3. Advertise the policies

We need to keep reminding each other what policies we have – there are a lot of them!

For example: when an article appears in the media that is relevant to a policy like For live animal export, share it on Facebook or tweet it on Twitter along with a link to that policy.

Happy editing!

 

Posted in Projects, They Vote For You | 1 Response

How to find what you’re looking for on Right to Know

Following on from some recent queries on Twitter, I thought it was a good idea to sit down and provide a guide on some of the ways you can keep in touch with a request on Right to Know.

Right to Know is a simple, free service provided by the OpenAustralia Foundation that takes the confusion out of asking the Federal Government for information. It enhances accountability by showing you all requests made using the site, and the responses from the Government.

There are a few ways you can find requests, some include:

  • Searching by authority
  • Searching by status
  • Searching by user

You can also “follow” almost anything on Right to Know, which allows you to receive a daily digest when something matches what you’ve followed. Most of the time people follow single (or multiple) requests, but what you may not know is that you can follow an authority, a user, a search term or even the entire site!

With Right to Know growing (we have over 570 authorities, and over 830 users who have made 880 requests so far!) I thought it would be good to describe some of the ways you can keep up to date with requests and topics on Right to Know.

Before you begin

Anyone can search for requests, but you’ll need to register to be able to make a request, comment on a request, or follow a request.

Registering is quick and easy, all we need is your name (you can use a pseudonym if you want) and email address. We believe in transparency, so we describe how we will use your information on our Privacy Page.

Searching and following requests by authority

If you’re interested in a particular authority (such as the Department of Immigration, or ASADA) you can see all requests made to that authority, and also follow it to be updated whenever a request is made or updated.

To do this, click View Authorities on the top. Here you see a list of all authorities, and ways to group them (on the left hand side). You can also search for authorities using the search bar just above the list:

0001-allauthorities-resized

 

When you find the authority you’re interested in, just click it and you’ll be taken to the page for that Authority (I’ve chosen the Immigration Department as an example):

0002 - dibpauthpage

From this page, you can:

  • Find information on what the authority does
  • Follow (for email) or subscribe to an RSS feed to receive updates on requests
  • Find out how many requests have been made in total
  • Browse a list of all requests, or look at requests which are successful, unsuccessful, or unresolved.
  • Search for requests made to that authority

Searching and following requests by keywords

Some people might be interested in a specific topic which could apply to multiple authorities (such as “Technology”, “Open Government Partnership” or even “Religion”).

Right to Know makes it easy to find exactly what you’re looking for, using the search bar at the top of the page, or the Advanced Search feature.

If you want to follow anything about, for example, Religion, you simply search for it:

0003-searchreligion

Then on the search page, simply click the Follow button on the right hand side:

0004-religion

That’s it! Right to Know will automatically email you once a day with any new requests or comments related to that search.

Searching and following people on Right to Know

Right to Know has some amazing contributors, and sometimes it can be great to see what some of these contributors say.

To follow a contributor, simply search for the user or click their name, and you’ll be taken to their page (I’ve chosen a new user to Right to Know, Vera Lystich):

vera

Simply click Follow, and that’s it! You receive an email with that person’s comments and requests every 24 hours.

 What if I want to stop following something?

While we only send you one email per day (we combine everything you follow into a helpful daily digest) we don’t want to send you emails about things you’re no longer interested in. That’s why we include a link to change your preferences. Simply click the link to be taken to a list of the things you follow, where you can unsubscribe with the click of a button.

More Information

You can find more information on how to search and follow requests on the MySociety blog – https://www.mysociety.org/2014/09/19/how-to-keep-up-with-hot-topics-in-your-local-area/ and https://www.mysociety.org/2014/09/29/subscribe-to-foi-requests-on-any-topic/

You can also get in touch with the team at Right to Know, either via email (contact@righttoknow.org.au) or on Twitter (https://twitter.com/RighttoKnowAu).

If you want to support Right to Know, please consider Volunteering or making a small donation to the project.

 

 

Posted in Presentation, RightToKnow.org.au | Tagged , , | 1 Response

Getting to know you

A mess of stickies, collecting and sorting observations we made while speaking with our supporters.

A mess of post-its: collecting and sorting observations made while speaking with our supporters.

Over the last couple of weeks we’ve been reviewing and trying to improve the way the OpenAustralia Foundation handles donations. While there are tens of thousands of people using our projects, over the past few years we’ve only received a very small amount of funding from small donations by individuals.

We know that many people are enthusiastic about our projects and so contribute their time and skills. Last year we launched a monthly meetup for people like this and it’s been a great success. There are also other people who aren’t able to contribute code or are busy. They can still support us with financial donations. We’d like to make this process a lot simpler and rewarding for those people, and of course increase this kind of donation.

Research aims and process

Reaching out and getting to know the individuals that have donated to us has been an one key goal of this mini-project. We’ve been follow the process for design research set out in Erika Hall’s excellent book, Just Enough Research:

  • Define the problem
  • Select the approach
  • Plan and prepare
  • Collect the data
  • Analyse the data
  • Report the results

Our goal, or problem statement is:

Describe the kinds of people who donate to the OpenAustralia Foundation and why they do it.

Because the number of number of people who had donated to the Foundation is quite small, we decided to contact them with a small survey and then conduct short semi-structured interviews with as many as possible. This would give us a chance to thank these awesome people, establish a connection with them and find out how they see the world. To quote Hall:

The goal of interviewing users is to learn about everything that might influence how the users might use what you’re creating.

Finally we would collect and analyse our data and then present the findings in a blog post (here it is).

Survey

We sent a very short Google Forms survey to the 160 people who have donated to the OpenAustralia Foundation.

The questions were:

  • Roughly, how old are you? (Under 20, 20s, 30s, 40s, 50s, 60s, 70s, Above 80)
  • What is your occupation?
  • Which OpenAustralia Foundation projects are you familiar with?
  • Do you regularly donate money to organisations or projects?
  • Why did you donate to the OpenAustralia Foundation?
  • Would you be happy to have a brief chat to help us learn more?

The idea was to try and keep the survey as short as possible so that people would fill it out and be happy to chat, whilst prompting some useful insights.

So far we’ve received 38 responses. Here’s a basic summary of the responses:

  • Over 90% of respondents were over 30 years old, with 75% being evenly distributed between 30s, 40s and 50s.
  • After categorising the occupation responses, IT/Tech was the most group, followed by retired and professional services. There were two smaller groups working in government and in civil society organisations.
  • 75% of respondents said they were familiar with PlanningAlerts, 65% OpenAustralia.org and 50% Election Leaflets. 45% were familiar with They Vote For You, 40% Right To Know and 10% morph.io.
  • 65% of people said they regularly donated money to organisations or projects.

While we are dealing with a small amount of data, this information is helping turn some of our assumptions into knowledge.

Twenty-one people were happy to be interviewed and so far we’ve spoken with eleven. It’s been a pleasure to learn about the people who use our projects. A few very generous people even made donations after speaking with us.

While the survey was a few basic questions, the interviews were a deep dive. We wanted to get to know these people and find out about their relationships with NGOs, their needs and concerns.

So what did we learn?

Passionate and knowledgeable in their area of focus

The people that we spoke to were passionately interested and knowledgeable in their chosen area of focus. For a number of them, that was local planning and they were either working professionally in the area or were members of resident action groups.

These people are using one of our projects as part of a wider project or campaign they were active in.

Here are some of our observations about who these donors are:

  • These donors are generally quite self directed. Many of them were self-employed, involved in community action groups, or had established independent projects of some kind.
  • A number of people were involved with local community organisations.
  • The people we spoke to who were using PlanningAlerts (most people), had acted upon information they had received in an alert by making a submission to council.
  • People who donated made little use, if any, of more than one OpenAustralia Foundation project. They were often aware of a number of them, but were only an active user of one project. In the survey, a quarter of people selected just one project that they were familiar with.
  • Most people we spoke to had discovered their first OpenAustralia Foundation project by being directly introduced to it, often by someone from the foundation, rather than finding it through Google or social media.
  • Everyone we spoke to had found the OpenAustralia Foundation through one of the projects, not the other way round.
  • Everyone we spoke to was computer and web savvy, a number of them having experience with programming or data management.

People donated because they use the projects and support OAF’s broad work towards better access to civic information

In the survey there were two very clear themes in the responses to the question “Why did you donate to the OpenAustralia Foundation?”

  • Because they support the values of the OpenAustralia Foundation
  • Because they get value from an OpenAustralia Foundation project and want it to continue

These two themes came through strongly in the interviews—in particular it was clear that these people were successfully using one of the projects.

“I believe in freedom of information, open and transparent government, open data”

In the survey the majority of people mentioned their support for the general mission of the OpenAustralia Foundation to create better access to civic information. Some examples:

  • “Because I believe in OpenAustralia and the ideals therein”
  • “I believe public, open access to parliamentary records is important, and that politicians need to be held to greater account. They Vote For You is a step in the right direction to holding politicians accountable.”
  • “Believe in freedom of information, open and transparent government, open data – that governments should be more transparent, should publish the information they produce – briefings, data, revenue and expenditure detail – as a matter of course, without the need for FOI requests from the public or journalists.”
  • “I believe in your open data/technology agenda whole heartedly. Democracy is a closed broken old boys club and needs to be re-designed and re-built.”

“Because it provides a service I use!”

Many people also said they donated because they were getting value from one of the projects and wanted to support it. Examples:

  • “Because of the free service offered and that donations are what keep [organisations] like this going.”
  • “As a small thank you for the emails (about my MP’s speeches in Parliament) that you make it possible for me to receive without having to read/search all of Hansard!”
  • “I use the PlanningAlerts service and I value it. I also highly regard OpenAustralia Foundation and the sense of transparency and freedom it brings into the society”
  • “Because it does provide a useful service, making me aware of up coming proposed planning applications in my immediate neighbourhood that I might otherwise have not been aware of. Due to this early notification I have had some success with council and even VCAT in influencing the resulting plans to maintain respectful scale and privacy within my immediate neighbourhood.”
  • “i use it”

People don’t need to know much about OAF to donate to it, but appreciate a clear explanation when they ask

The majority of people that we spoke to knew very little about the OpenAustralia Foundation. Those who did know about the Foundation had an accurate picture of our size and structure. No-one thought the OpenAustralia Foundation was a large, well funded institution.

A number of people who had donated also donated to other non-profit websites. Wikipedia, Getup, Mozilla and Black Box Voting were mentioned.

There were quite a few questions about how the OpenAustralia Foundation is funded. A majority of people mentioned that they were concerned by the amount of money NGOs put into marketing rather than their core mission. They appreciated it when they received a clear answer from the NGO they were questioning.

People like to receive updates but appreciate brevity

Donors were generally positive about receiving regular email updates from the organisations or campaigns they supported. There were strong bad reactions to being signed up to campaigns they did not support.

People who receive organisation update emails from liked concise messages that were quick to read and scan.

A few people mentioned that the email newsletters they liked were ones that gave them some actionable information: TV shows to note down to watch or facts about the issues they were interested in to discuss with friends.

What now

In designing the way we handle donations, we don’t want to just follow the standard path. Everything we do should be focused on making our projects more useful and effective to more citizens.

The insights gained through this initial research will provide a basis for more experiments to see how we can help new groups of people get use from and support our work. We’ve already got some on the boil.

Posted in OpenAustralia Foundation | Tagged , , | Leave a comment

Email is your secret weapon

(Cross posted from the Poplus.org blog)

Earlier this year I was lucky enough to attend PoplusCon in Santiago, Chile. It was a remarkable experience spending two very intense days working through problems, sharing experience and knowledge, and making connections with incredibly talented civic tech coders and activists from around the world.

If you were lucky enough to be there as well, I hope we met and I also hope you remember the lightning talk I gave on a tool I had been working on for some time. Hopefully it’s something that you’ll find useful.

Lightning talk at Poplus Con

The tool is called Cuttlefish.

Cuttlefish

I have a cool new announcement to make about Cuttlefish.

That comes a little later. First I want to tell you a little bit about the motivations behind Cuttlefish and of course what it actually does.

What is transactional email and why should I care?

Transactional email is email sent from applications in “transactions”. This could be for example, the email you get when you sign up for a new service; it could be the email alert that notifies you when a politician has spoken in parliament. In simple terms it’s anything that’s generated automatically by you doing something in an application.

Email is your secret weapon. Email is what reminds people your website exists and keeps them engaged. It’s what allows useful information to come direct to people.

As an example at the OpenAustralia Foundation we run a project called PlanningAlerts which is a really simple service that collects up applications from around Australia for people and companies wanting to build or knock down buildings. This is information that’s published on local government websites.

Now, who in their right mind goes and looks at their local government website once a week to see if there are any new planning applications that might be of interest? Maybe you would do it if it was your job but ordinary people are hardly going to do that.

What instead we do with PlanningAlerts is allow people to sign up for email alerts when there is a new planning application in their area. They don’t have to find or navigate their local government website and most importantly they don’t have to remember to regularly check.

When there’s something new it arrives in their inbox. At the moment around 50,000 people per week receive an email alert from PlanningAlerts in their inbox.

If you’re anything like us you probably send a lot of email too.

Here’s the problem – Sending email and doing it properly is actually surprisingly really hard and it gets even harder the more email you send.

Commercial route

One option is to use one of many commercially available transactional email services, such as Sendgrid, Mandrill or Amazon SES. They are great and incredibly reliable. They make sending email so much easier.

However, it does come at a financial cost. It can be inexpensive or even free if you’re not sending a lot of email. However, as a civic technology organisation you likely send a lot of email and you don’t have much money.

Then, there’s the tricky issue of working in civic tech fundamentally depending on a piece of closed-source technology. What if the service disappears or changes? What if I can’t afford to keep paying the bills?

It’s a practical limitation as well as an ethical issue.

Can open-source help?

The open-source route

Alternatively you could build the email smarts into your open-source application.

That is what has tended to happen so far in the open-source world. Every application that sends a lot of email has had to figure out for itself how to do bounce detection, link tracking and all those tricky things.

For example, Alavateli from mySociety has lots of smarts for figuring whether an email has been delivered (so that government agencies have a hard time claiming that they never received an freedom of information request when it can be shown that it was delivered to their email server).

CiviCRM can send emails to large lists of people. It has all this built-in smarts for link tracking, figuring out whether people have opened emails and detecting bounced emails. It’s not exactly easy to set up though and frankly is crazy buggy.

FixMyStreet and WriteToThem also by mySociety similarly have their own code for handling bounce detection.

The goal behind Cuttlefish is to create a platform and language agnostic open-source service for sending email so that every application doesn’t have to figure it out for itself.

Let’s solve the hard (and boring) problem once and share the winnings.

Sounds a bit like a Poplus component doesn’t it?

So here are some of things that Cuttlefish can do for you, your organisation and your civic application:

  • Send email from your application using smtp in the usual way and get all sorts of added benefits for no effort
  • A lovely web UI to browse what’s happening
  • Monitor in real time which emails arrive at their destination and which bounce
  • Works with any web framework and language
  • Automatically not send emails to destinations that have hard bounced in the past
  • Track which emails are opened and which links are clicked
  • Statistics on emails sent, soft/hard bounced and held back
  • View the full email content for recently sent emails
  • Multiple applications can each have their own SMTP authentication
  • Built in, super easy to set up, automatic DKIM signing
  • Postfix, which you know and trust, handles email delivery
  • Open source, so no vendor lock in.

Cuttlefish and Poplus

Fast forward to October, two months ago, the OpenAustralia Foundation was awarded one of the Poplus mini grants to create a hosted version of Cuttlefish which will be freely available to the Poplus community for all your email sending needs.

This involves several related parts

  • Allow you and your team to use a hosted Cuttlefish server as if it’s your own. You will be able to use a shared server without seeing any of the emails being sent by other teams because we’re adding multi-tenanting to the Cuttlefish software.
  • Making Cuttlefish server provisioning completely automatic using Ansible.
  • Providing free access to the Cuttlefish server to the Poplus community for a minimum of 1 year (until at least the end of 2015).

Also we will be applying shortly for Cuttlefish to become a Poplus component.

The big announcement

The work is done and you now have access to a hosted version of Cuttlefish

  • Completely free to members of the Poplus community
  • Send emails from your own domains
  • No restrictions on volume of emails
  • Hosted and maintained by the OpenAustralia Foundation
  • Available at https://cuttlefish.oaf.org.au now.

How can I get started?

The plan is to add a simple form to https://cuttlefish.oaf.org.au from which you can apply for access. We haven’t quite done that yet.

In the meantime, if you want access please email us at contact@oaf.org.au and include the following information:

  • Your name
  • Your email address — where you will receive the Cuttlefish invitation
  • Organisation name & website — If you’re not part of an official organisation, that’s fine too. Simply include something about the people you’re working with.
  • Describe the emails you would like to send with Cuttlefish and the applications they come from (e.g. email alerts for our parliamentary monitoring site …)

The obvious disclaimer here is don’t use Cuttlefish to send anything that could be mistaken as spam. No unsolicited emails. No mass marketing emails. Only things related to civic technology. If you have any doubts or questions just ask.

If you’d like to look at the code or would like to try installing it on your own server go tohttps://github.com/mlandauer/cuttlefish.

IMAGE CREDIT

Posted in Announcement | Leave a comment

Luke and Henare talk They Vote For You on FBi’s Backchat

On Saturday Luke and I were really glad to be invited on FBi radio’s excellent Backchat programme.

Luke and Henare with Backchat hosts Luke Marshall and James Colley

Luke and Henare with Backchat hosts Luke Marshall and James Colley

You can listen to the podcast here:

Thanks for having us, FBi!

Posted in Media, Projects, They Vote For You | Tagged , , , , , , | Leave a comment

Designing They Vote For You

For the last three and a half months I’ve been working with the OpenAustralia Foundation to design their latest project, They Vote For You. For me, as always, this was a broad interpretation of the role “designer”: digging into functionality, language, information architecture, appearance, feeling, right down to the purpose and direction of the site. At the core we are trying to respectfully support citizens in using the tool to achieve something they want. I’m very proud of where we’ve taken the project and to now be joining the team permanently.

I’ve gained so much from others who’ve opened up and I want to pay it back by documenting some of our process. Sharing experience is also a key part of the Open Source Design Manifesto, “I will share my design experiences; both the good and the bad”.

There were a lot of unknowns and first-times throughout the project. For me it was a new team and their strong commitment to a project methodology with which I had little experience, Agile development. I was also tasked with a lot of Ruby on Rails development—I had a little experience from Detention Logs, but had never written ruby for a client project.

Similarly, the OpenAustralia Foundation hadn’t worked with a designer in this capacity before: someone working on broad iterations of design and implementing it in code during major application development. Previously they’ve handed a fairly established interface over to a design phase. That is still a common process, but in the wake of responsive design practices more and more designers are being embedded into teams, helping to develop an application or website from day one.

Photograph of Luke Bacon presenting the website to a crowd of volunteers.

Luke Bacon presenting They Vote For You at the OpenAustralia Hackfest September 2014, a month out from launch, Photo by Lisa Cross

Approach

The first recorded commit in our working repository for They Vote For You is 5th of August 2003, 11 years ago. The original version, Public Whip, has been running in the UK for a decade. As with most OpenAustralia Foundation projects They Vote For You is an implementation of a project proven overseas.

Many would have wiped away the decade-old PHP implementation and started from a clean slate. We briefly discussed doing this with the front-end page templates, but decided to keep them in place. There was 10 years of experience in the current version that we wanted to get as much value from as possible. We progressed by making changes in small iterations as our understanding of the project developed. Some of the site remains largely unchanged. The decision the build atop those foundations has defined the project.

We kicked things off with a whole day workshop to discuss our approach and get a feel for each-others’ perspectives. I ran a 20 second gut test exercise which helped me understand the others’ feelings towards basic colour, imagery, typographic concepts.

Most importantly we established design principles to guide our decision making:

  1. Focus on enabling actions that citizens want to take and access to the knowledge that they are looking for. Reduce the cost of them taking action. The action is the important bit. Citizens don’t need to understand the bureaucracy in order to use and access government.
  2. Assume people are smart & busy—present the important information up front and put detail within reach.
  3. Strive for universal accessibility.
  4. Design and implement with Progressive Enhancement.
  5. Create change by doing. Implement ideas quickly and assess—don’t be frozen by documentation and proposal.
  6. Maintainability is crucial, this project should last and evolve long into the future.

The next day, with great support from Matthew, Henare, I dove into the code. We targeted simple, obvious improvements; removing duplicate content and streamlining pages. As I went I took advantage of my fresh perspective to document the questions I was asking of the key interfaces. These user questions were helpful early on, but to be honest, I haven’t returned to them for a month or so.

The project’s persona

I picked up Aarron Walter’s much referenced Designing for Emotion and was inspired to create a design persona for the project. I haven’t seen other open source projects publish something like this but it’s been extremely useful. They Vote For You’s persona is a statement of how our site fits into the world of our citizens. It avoids design jargon and talks about a character, something anyone should be able to engage. Like the design principles, the persona helps the team make consistent, thoughtful design decisions. It is something to aspire to. I don’t think we’ve achieved every aspect of it, but bit by bit we’ll get there.

I think it can also help people outside the core team contribute. It includes some general references for colour and typography so you can get a rough understanding quickly: ‘more towards Wikipedia Mobile, not like Heroku’ for example. Someone making a contribution to the project should be able to look to those basic guides and avoid doing something completely out of character.

Using familiar tools

I think we made a good decision to aim for something simple and straight-forward. Whenever I tried to get fancy it never fit anyway. I’d alway come back to the logical, familiar choices that made communication clearer.

The typeface is one example. I researched a bunch of open source typefaces as potential candidates for the site. I played with Open Sans, Source Sans Pro and Clear Sans in particular. While Source Sans Pro and Clear Sans are great for reading and comparing data and interface elements, key tasks in They Vote For You, we also had a lot of extended text and needed something more comfortable for reading paragraphs. Those typefaces also have a more neutral, almost futuristic feeling, and our design persona calls for something humanist with a bit of character.

Many typographers recommend that you get to know how a small set of typefaces perform best by using them across many projects. I’m quite familiar with FF Meta by Erik Spiekermann which we often used at Collagraph. Fira Sans is a humanist, sans serif typeface based on FF Meta, designed by Eden Spiekermann for Mozilla and released under an open source license. It has a personality that stands out against the sea of Helvetica. It’s also designed specifically for digital displays and to perform well across a huge range of devices. I think it works very well in They Vote For You and it felt like a logical choice to use something I knew was reliable.

 

Screenshot of They Vote For You showing the use of the Fira Sans Typeface

Fira Sans in use on They Vote For You.

 

The team

I think the biggest contributor to the success of the process so far is the support and openness everyone at the OpenAustralia Foundation gives each other. We all know that this is an unusual and challenging project. We give each other the space and support to voice our arguments on a given decision. At other times we can step back and allow someone to pursue a bold and uncertain solution.

Photograph of civic hackers discussing a project at the OpenAustralia Hackfest 2014

OpenAustralia staff going through parliamentary process with Hannah from the Parliamentary Library at the OpenAustralia Hackfest September 2014, Photo by Lisa Cross

Henare has said he didn’t like the yellow section headings when I first implemented them, but over time has come to appreciate its affect. I think we did a good job of knowing when to keep discussing and when to give something a chance. At other times someone would step in to veto an addition. Those moments were crucial in protecting our aim for simplicity, it’s very easy to start adding unhelpful detail.

Screenshot of the /people page on They Vote For You

The lemon section header on the People page of They Vote For You

When

When is the right time to work on a specific element or layer of design? When is a design iteration done? When will we readdress it? For me, the hardest question through out this project has been ‘When?’

Aarron Walter suggests a Maslow’s Pyramid of UX: we should strive to design systems that fulfil people’s needs by being functional, reliable, usable, and finally pleasurable.

Working through my usual process, building up something from scratch, I knew when it was time to start thinking about colours or the finer, graphic layout challenges. Suddenly I wasn’t so sure. There were large sections of the site untouched, wording that was still confusing and unhelpful, but the team wanted me to start adding this top emotional layer. It didn’t feel right when we hadn’t achieved functional or reliable in many cases.

That week of the project was extremely difficult for me. I thought I was spending time on the important things and the team wanted me to do something else. I became unsure that I was the right person for the project.

I was being rigid with the wrong aspects of the process—I think I had to go through a mini-crisis at that time to break it. This was not a project, team, or workflow like any I’d worked with before, and I was going to have to adapt to make it work. The order and answers I intuitively wanted just weren’t coming through, and I wasn’t developing the things that the others needed so they could be developed through interation in time for launch.

I started making small, focused decisions and then just taking the next step—slowly cutting a path. Suddenly, sometime later, I looked around and realised I was back on track. In late September we set ourselves a deadline to launch in mid October. Nothing works like a deadline to help you adapt, get over it and move on.

Back to basics

When I was feeling most unsure I returned to design basics: empathise with the user and create simple stories through the site. Surprise is a classic way to stop someone and make them consider something. An unusual bright orange makes people consider the short, direct question “How does your MP vote on the issues that matter to youon the homepage. More than 60% of people follow that narrative and search for an MP.

They_Vote_For_You_—_How_does_your_MP_vote_on_the_issues_that_matter_to_you_

The colour scheme more broadly comes simply from the design persona:
light and refreshing in stark contrast to the formal, exclusive character communicated through the school-tie maroons and navy blues of traditional parliamentary design. Lemon yellow, lots of white space and a bright, cool blue were a simple, colour theory grounded choice—I’m no expert there but was brushing up with The Elements of Colour at the time. So far, we’ve had very positive reactions.

There’s a lot of room to keep evolving the personality of the site, particularly through the feedback and interface text. Injecting appropriate emotion could reward and support people in contributing to the resource.

Talking about design

I hope that explaining some of the choices and the ups and downs we made is useful to you. There are not enough discussions about visual and emotional design decisions in open source/civic hacking projects. It’s as if we think these aspects are just poured on by magical, inspired designers. I’ve found it extremely helpful to document and discuss as we’ve gone along, it’s been the foundation that has allowed me to be flexible.

They Vote For You will continue to evolve to better serve citizens. You can play a part. If you have ideas or changes you think would improve the project, please let us know by creating an issue on the repository. Check out the design persona and see all the #design and #wording issues listed. We’ve already had people submit very useful issues that are making the project better for everyone.

It’s been a fantastic experience to work on this with Henare, Matthew, Kat, Micaela and all the other contributors. I can’t wait to see the project grow in its new form, a decade after it began on the other side of the world.

Posted in Development, Projects, They Vote For You | Tagged , , , | Leave a comment
  • Civic Tech Monthly

    An email newsletter for people interested in the development of technology to enable civic change.

  • Categories

  • Archives

    • [+]2015
    • [+]2014
    • [+]2013
    • [+]2012
    • [+]2011
    • [+]2010
    • [+]2009
    • [+]2008
    • [+]2007