New! Three easy steps to a better PlanningAlerts

By Matthew Landauer | Published: December 20, 2022
Photo by David Clode

Do you use PlanningAlerts and have an email alert set up? Do you realise you’ve never set a password for an account? Have you ever found unsubscribing from an alert not entirely obvious or needed help to confirm a comment or update your email address for an alert?

Good news! From now on you’ll be able to make an account! Once you’ve confirmed your email address, simply login any time to manage your alerts, email settings and any comments you make all in one place. Phew! 

Don’t worry, you can still search for development applications and see all the usual details on the website without logging in. You’ll only need to login to set up your email alert or create a comment. All the usual details are freely and openly available at planningalerts.org.au as before. 

When you next get an email alert you’ll be asked to “activate your account” which simply involves these three simple steps

  • Step 1: Confirm your email address
  • Step 2: Click on the link in the email “Activate your account”
  • Step 3: Fill in your name and password

And hurrah! You’re in! 

What? And why now? 

Back when we first launched PlanningAlerts, we wanted the sign up for email alerts to be as quick as possible with the fewest clicks standing between you and what you want to know. We also thought it was rude to ask people to sign up for an account just to see what your service did before you had a chance to look around. So we decided early on we would avoid accounts if we could. We could, so we did. In fact by avoiding user accounts altogether we could take a couple of steps out of the process entirely.

Times have changed! People now expect to login to services and be able to update details with their online services on their own. Using an email and password are now completely standard and password managers even built into browsers to help us stay sane while needing unique passwords for many day to day services. In fact it now seems plain weird PlanningAlerts didn’t allow this. 

This has helped keep PlanningAlerts super easy to use. However this approach has come with limitations.

We also hear from some of you that it can be tricky to update or change an email alert or unsubscribe and there’s been no way to manage your alerts in one place. You had to wait for an email alert and follow a link in the email. With a user account you can simply log in, review your details any time, and make changes yourself.

Also there’s been no connection between different alerts and comments that the same person has. Over time we see that this can lead to confusion and also extra effort.

As the number of people using the service has grown so has the amount of support we have to give people for common problems such as these. In the spirit of giving you access to tools and information you need to stay informed on issues that matter to you in your area, it’s also important that we don’t do something for you that you could easily do for yourself.

Easier for you

You’ll find it easier to 

  • Sign up for extra alerts – no need to confirm each one
  • Change your email address – just confirm the new email address once and any alerts you have are then sent to your new address
  • See any and all the alerts you have currently setup in one place
  • See any comments you’ve made in one place
  • See when each alert was last emailed to you

Tiny changes, big impact, or, How it will make things easier for us to support your community

At PlanningAlerts it’s important to us that you can contact us directly with your questions. Frankly, we’d like to see fewer support requests on some of those more mundane requests in our inbox. This will help us have greater capacity to focus on better supporting people through the planning process. 

As a tiny organisation we think we’re most effective when we figure out small changes that can have the biggest possible impact. For PlanningAlerts that means giving you control over your alerts and comments. So we’ve taken the time to focus on improving your experience of this local information service over the last few months. By providing you information and tools to take care of your ongoing information needs, we figure it might be that much easier to connect with the big picture; we even suspect you may start to feel a growing sense of confidence you have a part to play in shaping your neighborhood.

Some of what’s happening under the hood

So now you know why we’re doing this, I’ll outline how we’ve been rolling out this change. It includes some technical details.

There are two cases when PlanningAlerts asks people to confirm their intention by following a link sent to their email address, setting up a new alert and making a comment. Previously each new alert or comment had its own discrete record of the email address attached to it. Each alert and comment needed to be individually confirmed.

The first step in this migration was to attach each alert and comment to a user record. Where there is no user record associated with an email address, we create one in a special un-activated state where someone can’t actually yet log in.

Then, any new alerts or comments are similarly automatically hooked up to these otherwise invisible user accounts.

Then, we started rolling out changes to production that are only visible to people who are logged in. They don’t need to put in their email address to sign up for alerts and comments. They also don’t need to confirm their email address (because it’s already been confirmed in the user account setup). Phew!

The advantage of rolling the changes out in this way is that we can do it step by step. We don’t have to make one giant change to the service which touches many many aspects of how it works.

Then, the next stage is that to create an alert or make a comment people now have to login. This we hide behind a “feature flag” in the application so again we can roll it out to production without changing anything for current users, so that we can have confidence we’re not breaking anything. 

That’s better!

We are enabling this final step for everyone today. We’re excited about this change and we hope you will be too. 

Matthew Landauer and Katherine Szuminska

Posted in Announcement, Development, PlanningAlerts.org.au | 11 Responses

What are people contacting PlanningAlerts about?

By Joanna Hill | Published: December 7, 2022

Guest post by Service Designer and Researcher, Joanna Hill

Since first launching, PlanningAlerts has grown a lot. The number of people writing to PlanningAlerts has crept up and up, as of February 2011 sending out just over 900,000 alerts (thanks Wayback Machine). By the time of writing (Dec 1 2022) PlanningAlerts has sent out a whopping 192 million alerts across Australia. Only a small percentage of people who receive all those alerts ever get in touch. Yet even that tiny fraction of responses now adds up to thousands of emails a year and currently, PlanningAlerts responds individually to each one.

As PlanningAlerts continues to reach more people, they had questions like, do we need to plan to provide the current level of support people need, or could some tweaks to our existing services reduce how often people need to reach out? They want to be available to connect when there’s clear value in responding personally, and not be swamped with emails.  So they needed an independent view to help them better understand what people are getting in touch about and why. Insights from such research could help inform decisions about future directions the OpenAustralia Foundation could take to improve service,connect with people directly where it most matters to them, so they can have a say on issues that matter to them. 

What might that look like?

Approach

I was invited to join the team for a short stint. I’d be using my research and service design skills to look for opportunities to improve the experience for both users and staff. 

How exactly would I do that? Well, we would allow the best approach to reveal itself in time. The team at PlanningAlerts gave me a generous open brief to trust my instincts and my experience. 
An obvious starting point was to have a look at the PlanningAlerts inbox. I’d be able to see exactly what questions were motivating people to write in.

Some of the things I expected we might uncover were:

  • Where automation/canned responses might be helpful (and just as importantly where it wouldn’t)
  • Inspiration for new features
  • Opportunities for improvements on existing features
  • An understanding of why people might be leaving the service

Researchers and service designers are rarely given access to such a rich and raw source of user data. Organisations can be pretty protective of their correspondence with customers. Someone in my role might, at best, be given a pre-approved snapshot or summary. This can limit our ability or just make it a slower process to understand the user’s underlying needs.

Getting started. First dig into the inbox

I wanted to start by getting a first pass quick impression of the correspondence, the ‘shape of the data’ we might say. That would tell me what analytical approach to take. So I dived in and started reading emails over a sample of one month. It was quickly apparent on first viewing that there were indeed a huge range of issues. It wasn’t like “oh! everyone was talking about one or two things”. It was everything from small technical hiccups to “I want this service to do more for me!”. On an emotional scale the tone of the email had everything from joy to rage. 

Based on this, I knew I’d need an understanding of how the service worked from beginning to end, top to bottom. Then I could plot some of the correspondence to points in the service and start making some sense out of this spaghetti bowl of an inbox. So I asked the team for a ‘Service Blueprint’ or service map of PlanningAlerts. Nope, they didn’t have one. Right, I’d build one. 

Building a thing to learn a thing

I never build a Service Blueprint for fun. 

It’s rigorous and meticulous work. It also comes with the very real risk of getting lost in the weeds or worse, falling in love with my beautifully colour coded artefacts and losing sight of bigger things. 
I wanna be up there in the crown of the tallest trees looking out across the entire forest thinking about the big, strategic issues. But the truth is I can’t do that unless I have a connection to the earth too. It’s a balancing act – existing between the tiniest details and the biggest of issues. I’ve made enough mistakes in my time to know where there be dragons so I approached this exercise with a determined ‘means to an end’ philosophy. Google sheets would do just nicely. 


All I needed for now was:


Some columns – to mark out the stages of the service in chronological order

Some rows – to mark out the activities of the main four actors (1. the submission, 2. the public, 3. PlanningAlerts and the 4. council/planning authority)

To fill out the frontend experience (the parts visible to the user) I pretended I was a user (a legit technique thank you) and used the live service, documenting as I went. I then sat down with Matthew to document things that were happening in the backend processes that users couldn’t see.

Here’s a snapshot of the Service Blueprint showing it’s sections:

Categorising is a skill

Done. Phew. Came out alive and brain not melted. Now I could start categorising and plotting emails against the map. One bonus benefit of taking this approach would mean I’d be able to capture a bit of insight based on volume – noticing just how many of the same requests were occuring. At this point I still didn’t know exactly what I would learn but I knew it would come out in the wash if I trusted the process. 

Category creation itself requires great skill. Archivists etc know this well. If you do this bit without deep consideration, you can set yourself up to miss important learnings down the track. The job here was for me to unpack a user’s underlying need in each email, which isn’t always immediately obvious or stated upfront. I also need to articulate it in a fair and neutral language. This deserves a bit of time. 

Here’s some examples of categories I created:

  • I want to see historical applications for an area /property
  • I want to view/get alerts by council area
  • I’m having technical trouble signing up 
  • I want to know why this application is not on your website?
  • I want to amend or add to my comment but don’t know how
  • What does “delivered to the planning authority” mean? 
  • Why isn’t my comment on the Council website?
  • Why isn’t my comment on the Council website?
  • I want to report a rude/ racist/ discriminatory comment
  • I am no longer receiving alerts, what’s going on?

Here’s a snapshot of the Service Blueprint showing the categories plotted along it:

Oh look, incidental value

At this point I did a bit of a shareback with the PlanningAlerts crew to discover that quite apart from any specific learnings, the organisation gets a kick out of having their service mapped for the first time. We recognise this as a separate and highly valuable asset. Even without the emotional user content of a Customer Experience Map, this artefact can enable easier conversations about the service in lots of ways. 


As a result, I took some time to do a slightly more polished and simplified version of the map.  Upgrades include icons, numbering and arrows.

The simplified version looks like this:

Download the pdf

Ok but what did we learn?

After categorising two months of inbox enquiries, it was hard not to ignore the story showing up in the volume spikes. While a characteristic such as volume is usually the domain of quantitative analysis, it did at least point me at where some of the biggest tension points might sit in the journey. Here’s 3 big volume points where a lot of email was generated to PA.

Learning #1: Receiving an alert

Getting an alert from the service, puts it front of mind and user will often undergo an assessment of worth
When a user (service subscriber) receives a email alert from PlanningAlertsregarding a new DA in their area, we know that generates a whole bunch of traffic to the website, but through the inbox analysis it also appeared to act as a trigger for a user to adjust their preferences with the service. The two biggest examples being – asking to be unsubscribed (exiting the service) and asking how to change their search location (making the service more useful).

Learning #2: Publishing a comment

Only after a user publishes a comment can they asses if they’re happy with it

Another noticeable volume of enquiries occurred right after a user makes a comment on a development application. This was mostly about needing technical support with wanting to modify their comment. An early theory on what’s happening here is that once a user sees their comment published in the public domain in cold hard internet print, they immediately have feelings about wanting to modify it – be it to be more anonymous or alter the tone or language.

Big email point #3: Abuse reports
Abuse report messages are very different from everything else, and require much sensitively in their handling

Following along chronologically, the third point in the journey where planningalerts.org,au receives a lot of email is where users are sending an abuse report to the service, regarding a public comment. Of the three groups, this is by far the most fiery in nature, users are feeling an injustice has occurred and they’re wanting that to be addressed. The careful response to these type of emails by planningalerts requires a different skillset from the more technical support stuff.

What next?

From here, the next steps of taking this work further could look like:

  • Undertake more analysis increasing the time span from 2 months to perhaps 6 months. To level out the data and look for any missed email categories
  • Cluster and grouping email categories around user need
  • Take each cluster and reframing them as an opportunities
  • Tag and prioritise the opportunities against the organisations needs and agendas.e.g quick wins, automation opportunities, deeper inquiry needed, partnership opportunity  or encourage greater debate.

Posted in Uncategorized | Tagged , | Leave a comment

Senator for NSW Andrew Bragg threatens OpenAustralia Foundation with legal action

By Matthew Landauer | Published: March 23, 2022

Liberal Party Senator Andrew Bragg has stepped up his campaign against the OpenAustralia Foundation. The senator has hired high profile lawyer Rebekah Giles to threaten legal action over how the OpenAustralia Foundation website They Vote For You presents his parliamentary voting record. 

They Vote For You takes the official voting record from Hansard and presents these publicly available records in a more accessible form so that voters can see how their elected representatives actually vote in parliament. As They Vote For You explains: “Forget what politicians say. What truly matters is what they do. And what they do is vote, to write our laws which affect us all.”

Senator Bragg, who serves as a member of the Liberal-National Party government, has objected to being recorded as an MP who has voted with his government colleagues on issues such as closing the gap, public school and university funding, increasing the Newstart allowance, the Paris Climate Agreement and increased funding for renewable intention etc.
Senator Bragg, and some of his Liberal colleagues such as Wentworth MP Dave Sharma, have joined in a campaign against They Vote For You.

First, Senator for NSW Andrew Bragg wrote to the Australian Charities and Not-for-profits Commission (ACNC) attempting to deregister us as a charity. This happened late last year. We don’t know what specifically was said in that letter.

Then, Dave Sharma, MP for Wentworth, wrote, in a coordinated action, to the Australian Electoral Commission (AEC) to try to stop us by claiming that we were breaking the law by not displaying “authorised by” on They Vote For You. That complaint was dismissed by the AEC because “…They Vote For You’s communications did not appear designed to influence elections and therefore were outside the commission’s domain.”

The first we knew of either of these was when they were reported in the Sydney Morning Herald “MPs call for ‘partisan’ political transparency site to lose charity status“.

Now, Senator Andrew Bragg has hired lawyers to threaten us with legal action for “misleading and deceptive conduct under Australian Consumer Law”. Furthermore they don’t want us to be able to openly discuss the issues because they have claimed that their legal threat is “PRIVATE & CONFIDENTIAL – NOT FOR PUBLICATION”.

The unfortunate thing is that this legal threat letter contains the first concrete and specific allegations which we can meaningfully respond to. Up until now it’s been months of mud-slinging in public from their position of power, claiming all sorts of ludicrous things while multiple attempts from us to invite a productive discussion with them have been met with silence.

It is our belief that the legal threat letter does not contain anything that is “private & confidential”. In reality it’s quite the opposite. It’s largely a discussion of the voting record of Andrew Bragg. Furthermore it is essential that discussions of the facts of politicians’ voting record happen in public. For that reason we have decided to publish a complete copy of the letter below.

07.03.22-Letter-to-M-Landauer-OpenAustralia-A-Bragg_redactedDownload


Our legal advice is that Senator Bragg has no cause of action and we’ll revisit that legal question below, but first, let’s look at the body of their complaints.

Help They Vote For You remain trusted and independent

Donate

Responding to Senator Andrew Bragg’s complaints

(a) Voted generally against closing the gap between indigenous and non-indigenous Australians

While it is true that the votes mentioned “have no substantive effect on the legal rights and duties of Australians, nor to raise or lower funding or taxes”, they are still votes that took place in our parliament on a particular subject matter of interest to Australians.

We are only able to include votes that go to division on They Vote For You, as they are the only votes that are recorded in any detail in the official parliamentary record (which is where we get our data). That is, only division records tell us exactly who was in the room at the time and how each of those individuals voted.

We cannot find any divisions on the bills listed at [7] that would be relevant to this policy. The votes on whether to pass those bills were only recorded as votes ‘on the voices’ in the Journal of the Senate: Aboriginal Land Rights (Northern Territory Amendment (Jabiru) Bill, Territories Stolen Generations Redress Scheme (Consequential Amendments) Bill 2021 and Territories Stolen Generations Redress Scheme (Facilitation) Bill 2021. Note that the Corporations (Aboriginal and Torres Strait Islander) Amendment Bill has not yet been voted on in the Senate. We also cannot find any divisions specific to the funding listed at [8]. Therefore, these votes cannot be included in our record.

A solution we have recommended previously is for all votes to be adequately recorded in the official parliamentary record, which could be achieved most efficiently by introducing electronic voting to parliament.

(b) Voted consistently against increasing funding for public schools

When attaching a division to a policy, we mark them as either “strong” or ordinary (e.g. “Yes (strong)” or “Yes”). Divisions that actually change the law are “strong” while divisions that are merely symbolic are ranked as ordinary. The two divisions you mentioned are ordinary and ranked accordingly. A division on an appropriation bill directly relevant to this policy would be ranked as “strong” and therefore have more weight on the senator’s voting record. However, we cannot find any such division to add to this policy.

(c) Voted consistently against increasing funding for university education

While appropriation bills may be the only bills that impact direct funding arrangements, many bills impact funding arrangements indirectly, including those mentioned at [11], which is why they are included in this policy.

(d) Voted consistently against increasing investment in renewable energy

Low emission technologies are not renewable, which is why these two disallowance motions are included in this policy.

While appropriation bills may be the only bills that directly impact government spending, many divisions either indirectly impact funding arrangements or express an opinion towards funding arrangements, and so are included in our voting records.

We have not found any divisions specific to the funding listed at [16]. Therefore, they cannot be included in our record.

(e) Voted consistently against increasing the Newstart Allowance rate

While appropriation bills may be the only bills that directly impact government spending, many divisions either indirectly impact funding arrangements or express an opinion towards funding arrangements, and so are included in our voting records.

We have not found any divisions specific to the funding listed at [19]. Therefore, they cannot be included in our record.

(f) Voted consistently against the Paris Climate Agreement

Our parliamentarians may not control whether we enter the Agreement, but as our elected representatives they should be expected to have an opinion on the matter. These divisions express that opinion.

(g) Voted consistently against a carbon price

These motions expressed an opinion on the carbon price. Elected representatives are expected to have opinions on important matters of concern to Australians.

(h) Voted consistently against increasing legal protections for LGBTI people

These motions expressed an opinion on legal protections for the LGBTQI+ community. Elected representatives are expected to have opinions on such matters.

As far as we can see, no relevant divisions on the bills mentioned in [26] were recorded in the official parliamentary record, so cannot form part of our voting records. The only record of voting on the Religious Discrimination Bill and the Human Rights Legislation Amendment (we presume this is the bill you referred to as the Sex Discrimination Amendment) is in the Senate Journal, which noted that a first reading vote took place ‘on the voices’ before debate was adjourned. 

Regarding the subjects listed in [27]:

The new category called “We can’t say anything concrete about how they voted on” was created to take account of the fact that not all divisions are equal. 

Only divisions that make real legal changes are classified as “strong”, while the rest (including symbolic motions) are classified as ordinary. After exchanging emails with several staff members of elected representatives, we agreed that no policy position should be determined solely by a single ordinary vote. Instead, where a representative has only voted once in an ordinary division for a policy, their voting record now says “We can’t say anything concrete about how they voted on…”.

We acknowledge the information you have provided regarding Senator Bragg’s position on these subjects, but our site runs solely on data drawn from the official parliamentary record. Without relevant divisions, we cannot make any changes to the record.

(a) We can’t say anything concrete about how they voted on increasing workplace protections for women

We agree that it is bizarre that divisions are not always taken on important voting matters, such as on whether to pass a bill. Unfortunately, this is common voting practice in our parliament.

Our record does not include a division on whether to pass the Sex Discrimination and Fair Work (Respect at Work) Amendment because there was no such division. That vote was taken ‘on the voices’ and so cannot be included on our site. The reason we cannot include votes ‘on the voices’ is that such votes are not recorded with any detail in the official parliamentary record – we do not know exactly who was in the room at the time of the vote, nor how each individual voted. In this case, the Senate Journal simply states, “On the motion of the Attorney-General (Senator Cash) the report from the committee was adopted and the bill read a third time.”

As mentioned above, a solution we have recommended previously is for all votes to be adequately recorded in the official parliamentary record, which could be achieved most efficiently by introducing electronic voting to parliament.

(b) We can’t say anything concrete about how they voted on increasing funding for road infrastructure

While appropriation bills may be the only bills that directly impact government spending, many divisions either indirectly impact funding arrangements or express an opinion towards funding arrangements, and so are included in our voting records.

We have not found any divisions specific to the funding listed at [33]. Therefore, they cannot be included in our record.

(c) We can’t say anything concrete about how they voted on increasing the foreign aid budget to 0.7% of Gross National Income

While appropriation bills may be the only bills that directly impact government spending, many divisions either indirectly impact funding arrangements or express an opinion towards funding arrangements, and so are included in our voting records.

(d) We can’t say anything concrete about how they voted on a constitutionally enshrined First Nations Voice in parliament

While a referendum would be required to actually amend the constitution, this division expressed an opinion on the matter. Elected representatives are expected to have opinions on such matters.

The legal threat

Now back to where they are threatening to sue us under Australian Consumer Law. For this we defer to the letter that Michael Bradley from Marque Lawyers wrote. He is generously helping us pro-bono. Here is the letter he wrote in response.

Letter to Rebekah Giles re OpenAustralia Foundation – Senator Andrew Bragg_2611028_1_redactedDownload

Where to from here?

I hope after all this it is more than obvious that there is no attempt by us with They Vote For You to deceive or in any way misrepresent the voting records of any politicians.

However, there are serious issues with the completeness of the parliamentary voting record that have been until now largely unknown, ignored and invisible to the general public. We need a complete record of all votes, one that lists how every single person votes on every single vote, whether “on the voices” or by division. This is the only way that a complete picture of every politician’s voting record will be available so that citizens can truly and fairly hold their elected representatives to account.

How can we make this more widely known? How can we work together so that parliament improves how they record all the votes?

Matthew, Kat and Mackay

Help They Vote For You remain trusted and independent

Donate
Posted in They Vote For You | 3 Responses

A new category has been added to your representative’s voting records!

By Mackay Ash | Published: January 31, 2022

You may have noticed that your MP and senator voting records are looking a bit different.

 Voting records are now divided into nine different categories:

  • Voted consistently for
  • Voted almost always for
  • Voted generally for
  • Voted a mixture of for and against
  • Voted generally against
  • Voted almost always against
  • Voted consistently against
  • We can’t say anything concrete about how they voted on

The newest of these categories is the last one, “We can’t say anything concrete about how they voted on”. It will be used for those cases where an MP or senator has only voted once on a policy and that vote didn’t have any particular significance.

This new category is part of our response to recent feedback we’ve received from concerned citizens and staff members of elected representatives about how the voting record is presented on our site. 

What’s wrong with the current voting record?

While They Vote For You has always tried to ensure its content is as accurate as possible, current voting practice in our parliament makes this difficult.

Today in parliament, most votes occur ‘on the voices,’ which is when our representatives yell ‘Aye!’ or ‘No!’ and the loudest side wins. These types of votes are not recorded in any detail, so we have no way of knowing who was in the room at the time or how they voted.

The only officially recorded votes in our parliament are divisions. When a division takes place, our representatives walk to one side of the room or the other (each side represents either ‘Yes’ or ‘No’) and their names are taken down, leaving us with a record of who was present, who was absent, and how each present representative voted.

Divisions are only called under particular circumstances and are much rarer in the House of Representatives than the Senate because most MPs belong to one of the two major parties, which rarely call for them.

Because They Vote For You gets its data from the official record, the only votes that appear on our site are divisions.

Why is this a huge problem?

As we’ve discussed before, having only data from divisions on They Vote For You means that we are only seeing part of the story of how our representatives vote – and a smaller part at that.

Our parliament could do something about this situation but has so far chosen not to. For example, they could introduce electronic voting so that all votes are quickly and efficiently recorded, but a committee that investigated this possibility in 2016 concluded that it was unnecessary.

If the incomplete official voting record concerns you, we encourage you to contact your representative and let them know.

What does this mean for us?

Having an incomplete voting record makes our task on They Vote For You much harder, particularly as not all divisions are created equal. Many divisions are purely symbolic – they express a view, but don’t actually make any legal changes.

After the recent blaze of public interest in our site, we received several emails from concerned citizens and staff members of elected representatives who believe our current approach to weighing divisions can be improved. And we agreed with them.

While our site already allowed for votes to be weighted differently, with ‘strong’ divisions (Yes (strong)/No (strong)) and ‘normal’ ones (Yes/No), these labels were not being applied consistently across all policies. Further, all policies were showing up on representatives’ voting records, even when that representative had only participated in only one ‘normal’ division. This skewed their record and made those representatives appear more or less in favour of a policy on the basis of a single ‘normal’ division.

We agreed that this was a problem and set about trying to find a solution.

What changes have we made?

First, we went through all our policies and ensured that the only divisions marked as ‘strong’ were those that propose real action – such as a change to our law – and are directly relevant to the policy.

Then we created a new category on our voting records: “We can’t say anything concrete about how they voted on”. Policies that appear in this category are ones where the representative has only voted in a single ‘normal’ division, and never in a ‘strong’ division.

We hope that these changes will make They Vote For You a more meaningful resource as we continue to hold our representatives accountable.

Posted in Announcement, They Vote For You | 2 Responses

Productive discussions and pointed attacks

By Matthew Landauer | Published: December 3, 2021

It’s been a busy few weeks for They Vote For You. We’ve benefited from a surge in interest despite, or because of, some very pointed attacks by two MPs on the integrity and motivation of the site and the work of the OpenAustralia Foundation in general.

With a renewed surge of interest from the general public comes renewed interest from other MPs.

On 16 November we received the a letter from the crossbencher MPs Helen Haines, Rebekah Sharkie, Zali Steggall and Andrew Wilkie about how their voting record appears on theyvoteforyou.org.au.

We include the text of the letter here for easy reference:

They Vote For You is an important platform and we commend Open Australia Foundation’s initiative enabling voters to easily access parliamentarians’ voting record on matters significant to them. However, we have concerns that the They Vote For You platform is misrepresenting how we vote, in some instances quite significantly. We acknowledge that this is not intentional and can attribute several reasons to why this may occur.

  1. If fewer than five Members place a dissenting vote on a matter then a formal count is not held and the question is resolved in the affirmative under Standing Order 127. The dissenting votes are recorded in Hansard and the Votes and Proceedings, but are not picked up by They Vote for You. Dissenting votes are often crossbenchers expressing disagreement with the two major parties. For example on 25 March 2021 the House divided on the Northern Australia Infrastructure Facility Amendment (Extension and Other Measures) Bill 2021 and only Mr Bandt, Dr Haines, Ms Steggall and Mr Wilkie voted ‘No’ and the question was resolved in the affirmative.
  2. The Opposition routinely move second reading amendments to Government bills that are not directly relevant to the substantive matter of the Bill before Parliament, and are often used for political purposes or as a procedural tactic. This is of course their right, but it greatly expands the substantive matter of the debate. The effect of votes on second reading amendments are also materially different to votes on Bills themselves, which reflect a Members position on specific proposed legislation as opposed to a general proposition. It seems that They Vote For You records a vote for or against these second reading amendments in the same manner as votes are recorded for Bills. This distorts our voting record on some topics quite significantly, and does not accurately reflect the question we are voting on. For example, the Opposition’s second reading amendment on the Health Insurance Amendment (Prescribed Fees) Bill 2021 was “whilst not declining to give the bill a second reading, the House urges the government to implement policies to better protect Australians’ Health”. Further, second reading amendments moved to the National Skills Commissioner Bill 2020 concentrated on funding for vocational education, rather than the Bill itself which established a new statutory office of the National Skills Commissioner. This may mislead people on a Members’ views on health policy or vocational education funding. 
  3. Due to COVID-19, remote participation in Parliament has been necessary but regrettably, the Government and Opposition have done a deal that prevents crossbenchers from pairing in the House of Representatives. This means that crossbenchers participating remotely cannot have their votes cast in the moment during a division. However, we do have an arrangement with the Speaker that our voting intention on each issue is tabled at the end of each sitting. This is not recorded by They Vote For You and instead we are recorded as voting with the majority or as absent. 
  4. Finally, where there are multiple votes on the same Bill or issue, some consideration should be given to the effect of the legislation or division. For example with the Cashless Debit Card, there were four divisions on the trial card but only two divisions on the decision to make the card a permanent feature of the social security system. Some crossbenchers supported the trial in order to gather evidence for whether it worked or not and when it was decided that it was ineffective voted against making it a permanent feature. Despite this their overall record shows that they voted ‘for’ the card because the number of divisions on the trial outweigh the number on the permanent card. The summary text written by They Vote For You therefore does not represent the true substance of the vote. 

We strongly believe it is important for the public to have full and transparent access to Members’ voting records in plain English. We also believe that the public is best served when this information is as accurate as possible, and reflects the true workings of the Parliament and the votes we cast on behalf of our constituents. This would be best achieved if They Vote For You accommodate the above concerns. Our offices would be pleased to work with you to achieve this. 

We ask that you review this website to ensure that our voting record is accurately represented and that voters are correctly informed. 

Yours sincerely 

Andrew Wilkie MP
Member for Clark

Rebekah Sharkie MP
Member for Mayo 

Dr Helen Haines MP
Member for Indi 

Zali Steggall OAM MP
Member for Warringah 

We replied two days later.

Dear Mr Wilkie, Ms Sharkie, Dr Haines and Ms Steggall,

Thank you for your letter. We appreciate that you ‘strongly believe it is important for the public to have full and transparent access to Members’ voting records in plain English’. 

We thank you for taking the time to list your concerns. We are always pleased to receive information that may improve They Vote For You (theyvoteforyou.org.au).

We have gone through your four points and will respond to each in turn.

  1. Currently, the only form of voting that is properly documented is division voting. While, as you mentioned, dissenting votes are recorded in Votes and Proceedings, it is only those few names that are recorded. We do not know who else was in the chamber at the time, and we cannot know without doubt how other present members would have voted in any case (we can assume, but we don’t have written evidence). Even if we could input these dissenting votes into our system, we would have to mark all other members as either absent or voting with the dominant position, and we have no actual record of either of these things. So we could be in a position of recording many more other votes incorrectly in that case.

    We’re between a rock and a hard place with this one.

    This is a very unfortunate aspect of the parliamentary voting system as it is now, but one that can only be solved by parliament.

    If only the votes were all recorded.

    In 2016 in our submission to the Standing Committee on Procedure’s inquiry into electronic voting we suggested that electronic voting may be a solution. By using electronic voting for all votes the full voting behaviour of all members of parliament could be recorded quickly and fully.

    We would hope that Mr Wilkie, Ms Sharkie, Dr Haines and Ms Steggall would support a move by parliament in that direction.
  1. All divisions are included on They Vote For You, but not all of them are attached to policies. And even where they are attached, we have the ability to weight the votes more or less strongly to account for their different levels of importance (e.g. votes are connected to policies as either “yes” or “yes (strong)”). Generally, votes on second reading amendments like those you describe would be classed as weaker policy connections or not included in a policy area at all.

    For example, you mentioned the bill Health Insurance Amendment (Prescribed Fees) Bill 2021. There is currently only one division on that bill, and it is to keep the usual second reading motion unchanged. It is not connected to any policy area, which means it is not affecting anyone’s voting records.

    You also mention the National Skills Commissioner Bill 2020. There are two divisions listed for this bill in the House of Representatives. The first is on Dr Haines’ amendment in relation to regional Australia and is currently attached to a draft policy we are developing called “For increasing support for rural and regional Australia.” This policy is still in development, so it doesn’t appear on your voting records.

    The second division is a vote to keep the usual second reading motion unchanged after Ms Plibersek proposed an amendment of the type you mentioned (“whilst not declining…” etc). While Ms Sharkie, Dr Haines and Ms Steggall voted “Aye” in this division, Mr Wilkie voted “No”. The vote is currently attached as a weaker vote to the policy “for increasing funding for vocational education” and it appears appropriate to leave it there. It doesn’t seem to be overly political (though, of course, all motions are a little political) and its subject matter is clear. That is, by voting for or against it, you are not voting on the bill itself but on the subject matter of the motion (i.e. vocational education funding). The vote on the bill itself came afterwards and it was done ‘on the voices’ and so doesn’t appear on our site.

    We would be interested to hear your thoughts on this matter.
  2. Thank you for sharing this information. This isn’t something we’ve been made aware of before. Could you please provide a specific example on this point? If your votes are being erroneously included with the majority, this is a serious issue. On face value this looks like a fundamental issue with the way your votes are now being recorded in parliament. However we would like to better understand this problem before we comment further on this.
  3. We do try to take account of the effect of the division, which is why divisions that are connected to a policy are classed as either strong or weaker. That said, the issue in this point seems to be that the cashless debit card has become highly politicised and it would therefore be useful to make policies as specific as possible – and we agree. The original policy was created by a user on our site, and back then (five years ago) there didn’t seem to be a problem with its phrasing. Today, however, we do agree that it is worth amending. We will change the existing policy so that it is specific to temporary trials and create a new policy on making the card ongoing.

    Thank you for bringing this issue to our attention.

We want They Vote For You to be as accurate as possible, and so the more eyes on it the better. If there are any specific instances where you believe the voting record is inaccurate, please do not hesitate to reach out again. The more specific you are, the better, so please try to provide us with the actual policy name at issue and – even better – the division links. We do appreciate your assistance in working through the details.

We thank you once again for your letter and the explanation of the concerns you have. We feel that this is an important conversation that deserves public attention and so we would like to share it publicly, including our response in the context of a discussion on how voting happens in Australia.

Yours sincerely,

Katherine, Matthew and Mackay

Since this exchange we’ve had a follow up meeting with the offices of Andrew Wilkie and Zali Steggall to work through more details. We also have a meeting planned with Helen Haines’ office.

There are huge issues around the completeness of the official parliamentary voting record as we outlined in our response. It’s something that we highlight whenever we can because it so fundamentally makes it harder to get a complete picture of how your representative in parliament votes on your behalf. Only around 20% of all the votes in parliament happen through divisions. They’re the only ones where people’s votes are accurately recorded and so that’s all we have to go on right now. This is terrible.

We’ve had a few ideas, still in their early stages on how we might improve the situation. To do that it’s important we get various perspectives on what “votes on the voices” really means and how to interpret them. This is why we called out on twitter last week for any staffers belonging to the major parties that would be willing to talk to us. So far we’ve not heard from anyone unfortunately. Can you help?

With this renewed surge of interest comes a welcome scrutiny to They Vote For You. In fact we designed the site to be scrutinised. Every change is logged. The source-code for the site is open-source so anyone can inspect and understand how it works and what it does in every minute detail. There is no hidden black box. We believe this makes it as straightforward as possible to dig into how we’ve made sense of and characterised the limited data we’re working with.

Posted in Uncategorized | Tagged | 1 Response

Putting Councillors to Bed in PlanningAlerts

By Matthew Landauer | Published: September 13, 2021

This week we’re removing councillors, and the ability to write to councillors from PlanningAlerts. We disabled the feature in 2019. Now we’re removing the functionality from the code that runs the site and as a side-effect historical comments to councillors made between 2016 and 2019 will now no longer be visible on PlanningAlerts.

Why on earth would we do this?

I thought long and hard before doing this. It was in the front of my mind that hard work went into building the feature, it did what was intended and many people used it. So, why remove it?

The main reason is maintaining a complete and up-to-date list of all councillors for the hundreds of local councils in Australia was not a task that we were able to do with the way that things were set up. Regular council elections cause everything to change as well as big events like the forced merger of 40 odd local councils in NSW. In 2019 when we disabled the feature we were so behind on updates and the experience for people is that it says you can write to councillors who are no longer councillors which is terrible. Rather than show stuff that is plainly misleading we decided to remove councillors completely.

There were some other big technical issues as well. Write-it, a service (and open-source software) developed by a Chilean not-for-profit that we were using to send and receive the emails to councillors, was not being supported anymore. We were increasingly running into issues where we had to do things like redact comments and were having trouble contacting the administrators. In the intervening time the service has gone completely offline and the software is not being developed any further. So, we would have to either take over support and development of the Write-It software and host a service ourselves or develop an alternative approach, perhaps using Cuttlefish, the open-source transactional email service that we use for sending out email alerts from PlanningAlerts.

I wish we could have gotten back to this and fixed all this but the truth is we haven’t. So, rather than leaving this unfinished, not working thing around for us to feel guilty about, we’ve decided to grasp the nettle and excise it completely from the codebase. This has the added benefit of significantly simplifying the codebase which makes it easier for anyone new to pick up as well as simpler for us to support and add new features to. 

If we rebuild or recreate this feature in the future we will likely have to focus earlier on finding a sustainable way for the councillor information to be kept up-to-date. One passing idea is to only add councillors to a particular local council if a small group of local people nominate themselves as maintainers of the data for their council. They are the people most motivated to keep things up to date. Of course we don’t really know if that would work in practise but it might be one place to start in the future. If this is something that you would be interested in doing were it to come back, please do let us know.

What about the historical record?

It doesn’t feel great to be making changes to old applications visible on PlanningAlerts by removing comments to councillors (comments to planning authorities are unaffected). To try to be good citizens about all of this we’ve archived all the publicly visible information in a machine readable format (YAML) at https://github.com/openaustralia/planningalerts-councillor-comments-archive 

We’ve also made sure that snapshots of all pages with comments to councillors have been archived on the Wayback Machine from the Internet Archive. Hurrah for the Internet Archive. We wrote some code to do this all automatically too:

You can access the archived snapshots on the Internet Archive by taking the URL of the application on PlanningAlerts (e.g. https://www.planningalerts.org.au/applications/687643) and putting it into the Wayback Machine URL https://web.archive.org/web/20210909/<insert URL here> so that you get a full URL like https://web.archive.org/web/20210909/https://www.planningalerts.org.au/applications/687643

Thank you

Thank you to Luke Bacon, Henare Degan and Hisayo Horie for developing the features for writing to councillors. 

Posted in PlanningAlerts.org.au | 2 Responses

Right to Know Pro For Journalists, Researchers, Academics, Campaigners, and FOI professionals.

By Kat Szuminska | Published: March 30, 2021

A new way to make FOI requests, for professionals

Since we launched Right to Know we’ve regularly heard from journalists, who would love to use Right to Know, but need to keep Freedom of Information (FOI) requests under wraps until a story comes together.

We also know it’s not only journalists who need to work carefully through the issues of complex FOI requests, until a body of work is concluded.  We know how vital it is that you have the option to keep requests and responses private while you get the facts straight. And we know that takes time. 

So now, we’re launching Right to Know Pro, a fully featured toolkit, for journalists and researchers.

Use all the powerful tools of Right to Know to help save time, make requests more easily, and keep them safe until you’re ready to share your work with the world.

When you’re ready, you can then share requests and responses with your audience. 

Make FOI requests using Right to Know Pro, your requests and responses remain connected to your story, helping the public understand the substance, and verifying the source of truth with citations. 

You make and track multiple complex FOI requests, so Right to Know Pro provides a clean and powerful dashboard to simplify your FOI workflow across multiple and complex requests.

In the absence of easily traceable information, people can and do make up their own stories about the truth of your work, especially when it challenges their existing world view.

Reshare and refer people who want to see your FOI sources down the track. The presence of your hard work will also continue to be available, even if the Government agency later removes their disclosure log. No dead links when machinery of government or policy changes turn up dead links on a Government pr media website.

  • Keep requests and responses private while you work on your story
  • Send requests to multiple authorities with batch requests
  • A super-smart to-do list: follow the progress of your requests
  • Save as draft to get back to it later
  • Action alerts: know when it’s time to take the next step
  • Daily summary emails to keep your inbox clean
  • A powerful actionable private dashboard: track and manage all your FOI projects

Get Started with Right to Know Pro now

Posted in Announcement, RightToKnow.org.au | Leave a comment

Thank You for All Your Comments

By Kat Szuminska | Published: January 13, 2021

Today we removed the ability for people to comment on speeches published at openaustralia.org.au

We’re taking a pause on letting anyone comment on OpenAustralia because, in our work we want to make it a little easier for people who want to make a difference, to have their voices heard, for their voices to matter in the collective decision making we call democracy. We don’t think we’ve been successful in offering that here.

The commenting facility takes work to maintain, so we need to know that it’s adding value for the people who use OpenAustralia. The truth is, only about 350 people have commented on a speech since the site launched in June 2008. Maybe that says we’re not providing a facility people want, or that we’ve not provided quite the right way to do it. We just don’t know.

To everyone who has taken the time and effort to read, consider, and respond to speeches made in Parliament on OpenAustralia, a heartfelt THANK YOU. Listed by order of recent comments:

Tibor Majlath, Charlie Schroeder, Bill Thomson, Mark Duffett, Steven Azar, Warren Mccahill, Andrew Jackson, Romeo Alvaro, tony mckinnon, Alis Jacob, Isabel Storey, Michael Peters, Errol Nilson, Zhan Pintu, Dirk Bossard, Brett Hilder, Meredith Doig, Bernie Ramsay, Gill Caunce, Michael Dixon, KENNETH JURY, Johannes Kock, Robert Ashman, Carl HANICH, Adrian Menzel, John McWhirter-Whitlock, Leigh Wiggins, WARREN RODWELL, jules makk, Philip Hodges, Dwight Walker, Carlo Larocca, marcia flower, Stephen Caddaye, Madeleine Chapman, Patti Luskan, Bianco Salsano, Alcy Infinity, Kellie Nelson, Vicki Stebbins, Michael Rynn, Alison Keen, Tony Zegenhagen, David Simpson, Stephen Tittensor, Ray Bazouni, Jacob Mulquin, Grant Miller, Pete Gello, Harry Makris, Margaret Neal, Christopher Grey, Claire Bettington, William Boeder, Luke Walker, Mark C-F, Deirdre Ryan, Planecrashzone Planecrashzone, Andrew Simmons, Benton, Kay Spurr, Jane Russ, Anthony McIntyre, Brett Sanderson, Peter Rogers, Tom Peace, Arthur Ventham, Patti Farnell, Nathanael Coyne, Kathleen Minassian, Pat Patrick, Jennifer Jary, Shelley Travers, Kaye Schwartz, Ryan Brown, Glenn Kerswell, Steve Butel, Natalie Davis, David Hill, Chas Van Hulsentop, Rodney Ross, Donna Connors, Shay Holmes, Mary Alderson, Jim Gray, Carmel Connors, David Collins, dan diver, Thomas Nel, George Peterson, Ahmad Mostafa, Jenny Watson, Bill Macfarlane, Brian JAMES, K N, William McCann, robyn furci, David Redwood, Nerryl Brown, Henare Degan, Roger Helbig, tim leung, Mark Nicol, Mark Addinall, Munir Chagpar, 22 football, Jill trotter, Jason McClurg, John Goss, Pauline Zerbo, George Parker, James Walker, Keran Carsburg, John Griffiths, Roger Colclough, David Barratt, Bernie McGurgan, Chris Weir, Margaret Major, Antony (Tony) Amos, Elaine, carlos aparicio, al collick, Tim Handley, Steven Knowles, Eric Palmer, Maksim Stojanovic, Rona Goold, Brett Silich, Patricia Norman, Penelope Bell, Nathan Campbell, John Clapton, Margaret-Rose STRINGER, Chris Edwards, Hank D, Mark Addinall, Debra Lovi, Ernie Gimm, Michael Wells, kristopher kubique, Hari Guduru, Belinda Schneider, Renato Bright, Patrick Shanahan, Barbara Olsen, Jan de Wit, Robin Sharee Eastick, Pui Ho Lam, Free Mind, Melissa Stewart, Craig Bellamy, R Barron, G Gough, john jeayes, robert, David Gardiner, Ross Ulman, Delphine Stagg, Emily, michael spalding, abc 123, marlene huff, karan krishna, Daniel Kinsman, Dot Newbold, Margaret Makewell, Minnette Sheiles, Rebecca Dunstan, steve callaghan, al, Peter Demmery, Bill Realph, Sarah, treena day, Steve Wickenden, Nicolette Norris, James Kruithof, Julian Fidge, Dean Cottier, Ben Christy, jaime schultz, Barbara Johnstone, Verden Bell, Graham Martin, jim coleman, Jackie Bowden, Michael Cranny, Travis Broes, Mark Thorp, Ern Chang, Ray Borradale, cath blue, Andrew Black, Joy Butler, brian cooper, paul, Josh Cooper, Susan LakesNeedWater, Melissa Raven, Ken Jury, Stella Dyball, Dallas Beaufort, Maarten Vanderhaar, Colin Gourley, Philip Copeland, Lisa Hodgson, Richard Eden, Corey Irlam, Secret Ballot, christian borleis, Mele Fotu, Simon Dodds, Warren Rankin, Ralf Kluin, Deirdre Ryan, Iain Murchland, Margaret Muddle, Mike Male, Tony Huynh, Yodie Batzke, John Winterbotham, Donald Tate,Kevin Russell, Nish, jewel mahmud, Brian Armour, shirley williams, James, Greg Boyles, Peter Adnams, Brandon Chant, chris barrett, Philip Clark, J Jackman, Philip Lillingston, rachael dersch, Roger Coates, Rachel Teesdale-Smith, Rochelle Roberts, David Edgar, Shayne Cummin, Collin Van Uden, Adam P, Dale McClelland, Ian Rist, Lindsay Holmwood, Josef Tadich, John Rodda, Thomas Castiglione, Rod Chaffey, clive Osmon, Bruce Reyburn, David McNeil, Mick Loeckenhoff, Laszlo Kovassy, Dominic Gladheart, John Trigge, Anthony Moore, kerrie watherston, Andrew Smith, Harjeet Sing, MOrgs MOrgs, Bernie Glynn, Sherif Mansour, Peter Thornton, Debbie Bayliss, SYED HASSAN RAZA NAQVI, Shaun Lambert, Ernest Chamberlain, harry spicer, Geoff Mason, Kris Gesling, Caroline Cox, Sally Rose, Bill Ray, Dominique Quirke, Melissa McFarlane, Karen Fehring, Elly Bingham, Lloyd Stephen, Bronwyn Moir, Tonia Paroz, Nicole Carver, Kiera Pedle, Frank Baarda, connaught linton, Judie Gade, Hadley Baker, John Brown, David Leigh, James Fehon, Jonathon Singleton, Juiced Pixels, Ben Gran, Grendel, Trent Murray, Steve H, Ashton McAllan, Ben, Leonard Matthews, Joshua Moran, Joel Dignam, Graham Inskip, James Hay, Raymond J Warren, David Tangye, Mal Boyce, Paul Burns, Gary Jarrad, Matthew McDaniel, Delory, Paul Esson, Andrew Rossiter, Ben Rogers, Simon Rumble, Reng Ten, Brett Carnes, Daniel Snow, Amanda Kelly, David Elliott, Jason Geddes, Shelby de Piazza, Alexander Bakharev, Rasto Petrovic, Richard Jary, Mental Health Nurse, Robert Nelson, Philip Holder, Jaye, Kat Szuminska, Dan Oost, Patrick Terrett, Kieran Bennett, Michael O’Meara, Iain Murray, Ross Frove, Morgan Gobson, Noel Kelly, Peter Wood, Alan Howard, Sean Carmody, Cameron Reilly, Scott Bulfin, Darren Entwistle, Peter Daams

I want to personally thank you for speaking up for something you feel strongly enough about in this way.

For the time being, any and all comments you made before will remain visible, at least the next few months.

What role might commenting play a role in the future direction of OpenAustralia?

To help us decide what to do, if you’ve made a comment before, we would love to talk to you. Would you be willing to say a little about your experience of making a comment on OpenAustralia or have a short conversation by phone? Email us at hello@oaf.org.au

Posted in Announcement, OpenAustralia.org | Leave a comment

How we deal with moderation on Right to Know

By Ben Fairless | Published: November 4, 2020

There have recently been a few articles published in The Guardian about a request on Right to Know that we were asked to moderate. As one of the volunteers who spends the most time looking after Right to Know, I wanted to explain what happened in this case and explain how we moderate requests on Right to Know generally.

How this specific request was handled

In October 2019, the Australian Public Service Commission (APSC) reached out to us requesting the removal of a number of requests. You can find our full email chain with the APSC here.

The first thing we asked the APSC was to tell us what they were specifically concerned about. We then worked through each concern with the APSC.

Instead of just removing the requests (as they asked us to do), we agreed to hide the names of 3 public servants and keep the requests on the site. We felt this was the right balance between someone’s name being associated with an allegation of illegal conduct, and transparency and requests remaining public on our site.

Once we had made the changes, we then annotated the requests and contacted the requestors so they knew what had happened and why.

Investigation by the Commonwealth Ombudsman

The Guardian then reported a complaint had been made to the Commonwealth Ombudsman that the APSC had engaged in “legal misconduct”. The report went on to say that the APSC had been cleared of misconduct.

We provided the below statement to The Guardian as soon as we were aware of the story:

Right to Know was not contacted at any point during the investigation by the Commonwealth Ombudsman into allegations the APSC intimidated us.

When the APSC made us aware of potentially defamatory content, we worked with the APSC and took steps to redact as little of the request as possible to keep the request public.

We feel that it would have been difficult, if not impossible, for the Ombudsman to get the full story without contacting us first. We would have appreciated being advised of the complaint and the outcome before learning about it through the media.

You can read the follow up story from The Guardian here.

Transparency and Moderation – Getting the balance right

We are obsessed with transparency, and with giving you tools that you can use to effect change in our democracy. Moderation is not something we take lightly, however there are some things you can do to help us.

The Simple Stuff

We ask people who make requests to keep them focused. This means:

  1. No arguments about your case
  2. No statements that could defame or insult others

If you follow these rules, it’s likely you’ll never come into contact with moderation on Right to Know.

There are a few things that we will remove as soon as we are made aware of them:

The software we use to run Right to Know also attempts to automatically hide telephone numbers and email addresses, however it’s not 100% perfect so we may hide these manually.

More Complex Moderation

We sometimes receive requests from government authorities (like the APSC and the Australian Taxation Office) who are concerned about a request. They may ask us to remove a request saying that it’s defamatory or there are other safety or “public interest” concerns.

We have a published Takedown Policy, however each request is different and we review each request on it’s merits.

The first thing we ask any authority who wants us to take moderation action is to put in writing exactly what they want removed and why they want it removed. We may follow up with the authority if we have further questions or to get a better picture of what’s going on.

As the email contact happens via our contact email address, it’s not automatically put on Right to Know. We can provide you with a copy if the authority agrees, otherwise you can request the email trail via Freedom of Information (using Right to Know for full transparency if you want!).

Once we the information from the authority, we review the request, any other relevant information, and our policies. Our goal is to keep as much of a request as possible online. Once we’ve worked out what we can do, we then let the authority know. In some cases, we will refuse to take action on a request (if it doesn’t meet our guidelines).

If we make any changes to a request, the last step (and most important) is putting a note on the request to indicate what we changed and why. We will also reach out to the requestor privately to let them know so they can ask us any questions.

More Information or questions?

We have quite a bit of useful information in our help pages, and you can always reach out to us via email if you have any questions. Our team are here to help with anything FOI related.

Posted in RightToKnow.org.au, Uncategorized | Tagged , , , , | Comments closed

The story of a tiny, almost invisible change

By Matthew Landauer | Published: September 29, 2020

I don’t really like technical blog posts.

Instead this is a story of a tiny, almost invisible change we made to PlanningAlerts, what led us to make this change and the surprising amount of under-the-hood work it took to get there.

We made a “tiny” change to how comments are shown on PlanningAlerts. “delivered to the planning authority” is now a link (it’s underlined).

screenshot of comment

When you click on the link it gives you some more information

screenshot of comment after clicking

This should give you extra confidence that the comment was definitely delivered to the local authority.

How it all started

A kind and considerate soul emailed us to let us know of a strange comment that had been posted on PlanningAlerts.

Does this post have any truth in it?

Is this post damaging the Planning Alerts site?

Was the CAPS really necessary?

Here is the comment in full

”DONT” NOT LEAVE COMMENT HERE IF IN FAVOUR OR AGAINST APPLICATION. MANY IF NOT ALL DO NOT MAKE IT TO COUNCIL MOST COMMENTS HERE NEVER MAKE IT TO THE RELEVANT DEPARTMENTAL AUTHORITY!!

Ive been let down by this page thinking mine and many others are being heard here. It makes a difference. It does not. After contacting council in past, they had no clue about my rejection of development or from others on this page. It seems Acting authorities never know about what written here. This site should be taken down as everyone presumes it has direct or indirect authority to distribute to relevant departments.

*** PLEASE CONTACT COUNCIL OR RELEVANT AUTHORITY DIRECTLY ***

There were similar comments made by the same person on three other applications.

My first emotional response was “how dare this person say this, how rude!”.

That person didn’t even contact us directly to ever let us know there was a problem. We don’t read all the comments. There’s just too many for a tiny team doing too much. And then to throw around those blanket accusations.

Ouch! It really hurt to hear that. We have put so much time and effort over the years into ensuring that emails from PlanningAlerts get successfully delivered. We went as far as developing a new open-source transactional email server, Cuttlefish, to give us visibility and improve deliverability across the board. We made PlanningAlerts to give people access to the democratic process and getting their comment delivered is pretty core to that objective.

Then to my horror I realised that what the rude comment was saying was partly true. For the last couple of months 15 or so comments were not delivered to that one particular council. This was terrible!

We are human and our software is written by humans.

In 10 years PlanningAlerts had an amazing track record. We actually only know of one instance when emails were not delivered to a council In that case, a council changed their email address and didn’t forward emails to their old address. As soon as we were notified, we manually readdressed the emails and worked with the council to get them delivered in time.

So, what was going on? Down the rabbit hole of email sending

(Here’s where we get technical)

What was super weird at first was that only some emails to that one council were bouncing. The bounce message mentioned “DMARC” which was not something I knew about at that stage. So why were only some messages getting bounced and not others when all the comment emails are generated the same way? Nothing in the content of the comments looked like spam, the formatting of each email is identical. I found it really confusing.

Then, I noticed a pattern. All the comment emails that were bouncing were made by people with “yahoo.com” or “yahoo.com.au” email addresses. A quick search of “yahoo DMARC” led me to an explanation.

Yahoo had an unusually strict DMARC policy which meant that any problems with emails that look to originate from yahoo domains would get hard bounced. So, we had a DMARC problem with our comment emails but it had been unknown to us up to that point because almost nobody was enforcing it so strictly.

At this point I’ll just quickly backtrack a little. For a long time we’ve been ensuring that our emails pass SPF and DKIM tests. Cuttlefish helps enforce this and does it all pretty much automatically. However DMARC, does two main things. It allows an owner of a domain (e.g. yahoo.com) to say how email servers should handle mail from that domain when SPF and DKIM tests fail. So, in our case SPF and DKIM was fine, or so we thought, so why were the yahoo.com emails failing to get delivered?

Well it turned out it was to do with the second part of what DMARC does. It also enforces domain alignment; the domain in the “from” address of the email needs to be the same domain as used in SPF and DKIM. This is the problem that got us. It turned out that our comment emails had headers of the following form:

From: <email address of the person making the comment>
Sender: contact@planningalerts.org.au
To: <email address of council>

The Sender header is used in circumstances where emails are being sent on behalf of someone else. It’s defined in the email RFCs but generally has not been particularly well supported.

The purpose of all that was to ensure that when someone makes a comment via email that they receive replies from the council rather than us.

It turns out that DMARC essentially stops the Sender from ever being used. It appears that this was a conscious choice made by the DMARC designers because the Sender field is not shown to users in email clients.

So, we switched the email headers to:

From: contact@planningalerts.org.au
Reply-to: <email address of the person making the comment>
To: <email address of council>

And this magically solved our problem.

We resent all the recent comments that had previously bounced because of this, and our job was done.

Actually no. It would have been easy to stop there but actually we wanted to make sure that if something similar to this happens in the future we don’t find out about it because someone emails us asking about a comment on the site. We should be the first ones to know there is a problem.

After we fixed the way that comment emails are sent we started working on a whole host of other changes to improve the visibility and the impact of any potential future hard bounces for comment emails.

Improving visibility and reducing the impact of problems

In explaining earlier that only emails from the “yahoo.com” didn’t get delivered I was over-simplifying. What actually happened was that a yahoo.com email would hard bounce and then the email address of the council would get added to a “deny list” in Cuttlefish. Once an address gets added to the “deny list” any further emails sent to that address will not be sent. Addresses then get cleared out from the “deny list” after one week. This is an automatic feature of cuttlefish designed to reduce the chances that we send out bad emails which improves the deliverability of our emails across the board.

So the case of sending emails to local authorities is different from sending transactional emails (email alerts, etc..) to “random” people on the internet. We should be really confident that the email address of the council is correct and that emails to them generally should not bounce.

So, next we added a feature to cuttlefish.io to allow PlanningAlerts when it’s sending the comment emails to say “please ignore the deny list when sending this email”. This means that if an email hard bounces to a council, that cuttlefish will still attempt delivery for any subsequent emails to the same address. So now if hard bounces do happen the impact will be reduced.

Then we started working on the “visibility” part. If a hard bounce happens for a comment email how can we get notified of this urgent problem? One option would be that we get an email. In recent years we’ve been trying hard to limit the number of emails generated automatically for notifications and instead send those notifications to Slack. So, why not just try sending a notification directly to Slack instead?

How to do this? We don’t want Cuttlefish to have the knowledge to notify us of certain problems with certain emails but not others. It makes much more sense for PlanningAlerts itself to do that because it’s the one that knows the difference between the different kinds of emails. This is where we need to get some of the delivery information from Cuttlefish back into PlanningAlerts.

This finally gave me the reason and motivation to add a feature to Cuttlefish that I’ve been wanting to do since I first built it – which is to add webhooks where every time there is a successful or failed email delivery a custom URL (on another site of your choosing) is hit with information about the email. In our case we’ll be making a new URL endpoint on PlanningAlerts to accept those webhook callbacks.

So how do we identify if the email corresponds to a comment email or an alert email and which comment email? I think the most elegant solution to that is to allow the sender of an email to attach custom metadata.

Here’s an example of a comment email as seen in Cuttlefish showing the attached metadata (in this case comment-id):

screenshot of cuttlefish metadata

So then it was a relatively simple matter of hooking up a bit of logic in PlanningAlerts where on every “delivery event” it receives from cuttlefish, it checks whether it’s from a comment and whether it was a hard bounce and if so it sends a notification to our slack channel.

screenshot of slack notification

At the same time as we put this in we noticed another council was bouncing emails! This time for some reason they had just decided to specifically block our email server. Thanks for that. Thankfully they quickly fixed it when we asked them to and we again resent the emails.

Making it visible to people using PlanningAlerts

We’re confident now that all the changes we made to improve the delivery of emails to councils and to notify us of problems makes it unlikely that similar problems will reoccur in future.

However all this stuff is under the hood and invisible to users. Perhaps that’s how it should be?

However, it’s not uncommon that we get emails from people who just want to make sure that the council did in fact receive the comment they made through PlanningAlerts. The council tells them they never received it, so we check in Cuttlefish to find that the council absolutely received the email.

What if we could make this more clear for people?

Why don’t we do something similar to what is in RightToKnow.org.au where the message says whether it was successfully delivered?

screenshot of RightToKnow request

So that’s what we did. The final piece in a long chain.

Posted in Planning, PlanningAlerts.org.au | Tagged | 1 Response

  • Occasional News

    Stay in the loop with occasional news and notes from the OpenAustralia Foundation in your inbox.

  • Categories

  • Archives

    • [+]2022
    • [+]2021
    • [+]2020
    • [+]2019
    • [+]2018
    • [+]2017
    • [+]2016
    • [+]2015
    • [+]2014
    • [+]2013
    • [+]2012
    • [+]2011
    • [+]2010
    • [+]2009
    • [+]2008
    • [+]2007