This post originally appeared at https://sunlightfoundation.com/2013/03/11/openstates-report-card/ but has moved here for posterity.
Today we’re making available our Transparency Report Card, a byproduct of the work we did in producing Open States.
In the course of writing scrapers for all 50 state legislatures, our Open States team and volunteers spent a lot of time looking at state legislative websites and struggling with the often inadequate information made available. Impossibly difficult to navigate sites, information going missing and gnarly PDFs of tabular data have become daily occurrences for those of us working on Open States. People are always curious to know how their state stacked up compared to others — in fact one of the most frequent questions we have been asked has been “so which state was the worst?” That question got us thinking: How could we derive a measure of how “open” a state’s legislative data was?
After some consideration, we came up with six criteria on which each state could be evaluated, based on six of the Ten Principles for Opening Up Government Information: completeness, timeliness, ease of access, machine readability, use of commonly owned standards and permanence. We omitted four of the original ten criteria (primacy, non-discrimination, licensing and usage costs) that tended not to present serious differences between states.
Evaluating each state on each criteria was a large task, and with community support we ensured that each state was evaluated by multiple people. After the evaluation was complete, we converted the qualitative data on how a state performed to numeric scores (specific scoring details are available on the report card itself. After summing these scores, states were also assigned a letter grade according to where they fell among their peers. A state with a net score below negative one was given an F, a negative one or zero became a D. With the average total score among states being a 1.5, we gave states with a net score of one or two a C, three became a B, and four and aboe became an A.
The final breakdown was 8 As, 11 Bs, 20 Cs, 6 Ds, and 6 Fs. If you’re interested in how your state did compared to others you can check out all the details on the Open Legislative Data Report Card.
We know first-hand from our ongoing dialogue with state legislatures and open government technologists that identifying these commonplace problems can go a long way toward addressing them. In that spirit, and in the spirit of Sunshine Week, we offer this report card and recommendations today.