I just got off a call with SpotCrime, a company that aggregates crime data, standardizes it, and then publishes it via API and on a map. (They make money from the advertising.)
They called up Code for America because they’re looking for local allies to advocate to maintain access to open data – specifically a bunch of jurisdictions that are about to see the existing open data cut off.
Read more in this GovTech article:
From the article:
News of the split between Socrata and Motorola Solutions was disconcerting for some, such as market developer Brittany Suszan of SpotCrime, who feared the loss of a major open data resource.
Brittany says that this largely affects many jurisdictions in Maryland, California, and Utah, but any police department that is a client of Motorola Solutions is affected.
If anyone is interested in doing some open data advocacy around this work, reach out to me and I will connect you to Brittany. Any improvement to open data that would be helpful to your project would also be helpful to them, so perhaps they could also be helpful to you in advocating for something you’d like to see locally.
I followed up on Tom’s offer, and just got off the phone with the SpotCrime folks. (I’ve been working on OakCrime.org, Oakland’s effort to pump Oakland Police Department data to our citizens, and have been tracking SpotCrime for awhile.) While I still have serious concerns about some aspects of their for-profit products, I also think there is sufficient overlap between their agenda and CfA’s that we should think about advocating with them for preserved access to public data.
In the past, commercial crime data vendors like TylerTech (the company that acquired Socrata) have provided much less data than cities/counties have published directly via portals, like Socrata’s! (Does the consolidation of the data publishing industry remind you of Game of Thrones?!) So getting Motorola and TylerTech to agree that, while they work out their other corporate issues, these existing data sources will remain alive (via CKAN or ESRI’s ArcGISPortal or <your suggestion here>) seems like a good request to make of them.
Go to moto.data.socrata.com and you’ll see a list of ~500 city and county agencies that currently provide their public-facing data sets there. The immediate concern raised by the GovTech article above is: what happens to these feeds after June 2019?
Next steps: could we mobilize local brigades to contact any agencies near them to lobby with their local Socrata/now TylerTech contractor for an agreement like this?
YES! We’ve been politely dancing with LexisNexis Community Crime Mapping now for two years. Their contract with our local government clearly violates both state and federal statues when it comes to open records, freedom of information, and to accessibility of public data.
We pay this vendor $170K annually just to host our crime data on their servers and prevent it from reuse or scraping. There’s a lot more here on their deceptive business model:
… Government agencies, in many instances, have given contractors exclusive rights to the data. The government then removes it from public view online or never posts the data, laws and documents that are considered public information.
Public datasets that state and local governments are handing off to private contractors include court records and judicial opinions; detailed versions of state and local laws and, in some cases, the laws themselves; building codes and standards; and public university graduation records.
Much of the information collected and stored by private data companies such as LexisNexis, Westlaw or CrimeMapping.com is not available to the public without a price. The information that is available often is not searchable, cannot be compared with data from other jurisdictions and cannot be copied unless members of the public pay hundreds or thousands of dollars in subscription fees.
From the original source:
“This is public data. I should be able to print it on my underwear if I wanted to,” Drane said.
Note, @ninakin, it looks like this vendor was found to be complicit in underreporting crime rates in coordination with city officials a few years back.
Hey Carl - Brittany with SpotCrime here. We have resulted to FOIAing the data from Savannah. Not optimal way to get data, especially if other groups want the info too. Feel free to ping my email and I can get you info on the file we request, maybe you can hop on the same request.
We historically have had to put in a Ga. Open Records Requests to get it from the local police directly (which Georgia has a very progressive Open Record Law). We’re kind of loathe to do that because every open records request we submit we know is a massive burden on the Office of Public Communications, with whom we don’t want to establish a poor relationship with by asking for data consistently. Previously, we’d worked with them iteratively on opening other datasets, but the police data was always pulling teeth.
From what I understand, LexisNexis actually has a piece of sotware installed on the city’s in-house servers with full unfettered access to the city’s RMS (record management system, right?) data, including even legally protected data. The agreement boils down to the City trusting LN in good fath to redact any sensitive data responsibly. Which seems like an insane liability on the part of the city. Especially given LN’s track record of not reporting all incidents in other jurisdictions.
Thanks for all SpotCrime does! It would be nice to get more than a week of reports, and if LN gets the reports in real time then anyone should be able to. I digress…
If the vendor is ‘in charge’ of the police departments data, the vendor needs to create an open data feed as well -
GA law is pretty specific when it comes to this:
(2) ‘Public record’ means all documents, papers, letters, maps, books, tapes, photographs,
computer based or generated information, data, data fields, or similar material prepared and
maintained or received by an agency or by a private person or entity in the
performance of a service or function for or on behalf of an agency or when such
documents have been transferred to a private person or entity by an agency for storage or
future governmental use.
and
(h) Additionally, if an agency
contracts with a private vendor to collect or maintain public records, the agency shall ensure that
the arrangement does not limit public access to those records and that the vendor does not
impede public record access and method of delivery as established by the agency or as otherwise
provided for in this Code section.
Thank for reminding me that. The reticence has been mostly not wanting to pick a fight with police dept. but your point about the onus lying with the vendor is well-taken. In LN’s ToS, it explicitly bans any reuse or scraping of the data provided by the site (again, of course, they can’t legally stand on that, but that’s what their TOS says), claiming only the department and LexisNexis has the authority, at its discretion, to provide the data to anyone else. Then in the very same breath the TOS also claims it bears no liability for the accuracy of the information… so in their view they have provenance over the data, but bear no liability as to its accuracy. I can’t make this sorts of illogical legalese nonsense up!
You’d want them to push anyway (specifically to an open data portal). We want the department to be in charge of the feed, not someone else. We’ve created programming to help agencies push from RMS/CAD systems (free - they just have to promise to share the file they created with the programming with the public) to help with ‘pushing’ rather than ‘pulling’. Pulling gives control to a third party. Pushing gives control, accountability, and responsibility to the department.
The file we get is weekly. Feel free to jump on my request ‘Same file SpotCrime gets’ or something like that.
I doubt they will honor a standing request, so you’d have to send something every week (although they deliver data to the vendor with no FOIA request daily). Maybe this will get them to see the demand for the data and possibly help push them to open data.
This is working in Houston FYI. They were delivering a data feed to TWO private vendors that updated multiple times a day. Then they had a separate feed delivered to the city that was only updated daily, if that. The city feed would constantly break, but the vendors access was never interrupted. Incredibly frustrating. I’ve been told that they’re working on moving everyone to the same feed and it should launch this summer.
