The raw data behind the apps listed at https://data.gov.uk/apps
This is exported in a relatively raw format. It is refreshed nightly.
The fields should be relatively self-explanatory when comparing the data with the website. But these are the conversions provided:
- Dates were originally in unix format (e.g. 'created_unix') and so the iso-version is provided too (e.g. 'created') .
- 'uri' is an internal equivalent for a URL, and the external url is stored in 'uri_expanded'
- tags have been converted from internal ids ('field_tags') to names ('tags')
- related dataset ids have been converted from internal drupal IDs ('field_uses_dataset') to ckan IDs ('field_uses_dataset_ckan_ids'), which can in turn be converted to ckan names & URLs using the metadata dump.
The file is JSON lines format http://jsonlines.org/ , so you don't need to load the whole thing into memory to parse the json - just do a line at a time. It is also zipped with gzip.