A MapIt instance for approximate postcode boundaries based on the Voronoi diagram of ONSPD

mhl, updated 🕥 2022-12-08 04:35:02

Voronoi Postcodes MapIt

This repository is for a version of MapIt which contains postcode boundaries calculated from the Voronoi diagram of UPRNs with postcodes in NSUL.

For more information see the blog post here:

https://longair.net/blog/2021/08/23/open-data-gb-postcode-unit-boundaries

If you're looking to create a MapIt in your country, work on any of the underlying code, or re-use it as an app in another Django project, you probably want https://github.com/mysociety/mapit instead.

How to generate the data

If you just want the data yourself, there's no need to follow these instructions - you can download it. instead. If for some reason you really want to regenerate i yourself, read on.

Create a virtualenv for this project, change into the cloned repository directory and run pip install -r requirements.txt to install the Python package dependencies.

Download NSUL

You can get NSUL from the ONS's Open Geography Portal. e.g. here is the July 2021 version

Unpack the archive somewhere convenient.

Create the database and schema

Create a PostgreSQL database and install the PostGIS extensions with CREATE EXTENSION postgis and CREATE EXTENSION postgis_topology.

Create a ~/.mapit file with your database credentials, something like:

{
    "DJANGO_SECRET_KEY": "some long random string",
    "MAPIT_DB_NAME": "mapit-postcodes",
    "MAPIT_DB_USER": "",
    "MAPIT_DB_PASS": "",
    "MAPIT_DB_HOST": null,
    "MAPIT_DB_PORT": "5432",
    "COUNTRY": "GB"
}

Then run ./manage.py migrate to create the database tables.

Load the NSUL data into the database

You can do this with:

./manage.py mapit_postcodes_populate_nsul_table -r data/regions/gb_regions_multipolygons.shp ../NSUL/Data/*.csv

... adjusting the wildcarded parameter for where you've actually extracted NSUL.

Calculate the Voronoi diagram

This command will calculate the Voronoi diagram of all those points, and store the results in the database:

./manage.py mapit_postcodes_populate_voronoi_table

Generate postcode GeoJSON files

This command will create unions of the Voronoi regions to create polygons representing all postcode areas, districts, sectors and units and output them to GeoJSON. (Warning: even with a fast machine with many cores, this can take several days to run, so you definitely want to run it under tmux or similar.)

./manage.py mapit_postcodes_union_postcode_regions --skip-vertical-streets \
    -r data/regions/gb_regions_multipolygons.shp \
    -o data/gb-postcodes

That will generate those GeoJSON files under data/gb-postcodes

Package up the data

To build a bzip2-compressed tar archive of these GeoJSON files, you can run:

bin/make-data-archive.sh data/gb-postcodes

... which will create the archive as data/gb-postocdes.tar.bz2.

Import the data into MapIt

If you want to import this data into a MapIt instance, then you can do that with:

./manage.py mapit_postcodes_import_postcode_areas data/gb-postcodes 1

... where 1 is the ID of the MapIt Generation you want to import the areas into. (It won't try to reimport any areas that already exist in that generation.)

If you want postcode lookups to work from the front page of the MapIt instance you will also need to import postcodes from ONSPD or Code-Point: see http://code.mapit.mysociety.org/import/uk/ for more details.

Contact

Contact: [email protected]

Issues

LSOA21

opened on 2023-03-10 13:54:26 by ctdh

Hi Mark

Many of the LSOA on https://postcodefinder.net are not valid ... (also the contact us form broke on me)

Strangely many are only a few digits out...

Where you have implemented an LSOA21 update from LSOA11, the new code is often not correct, here are some examples:

E01034395 Lewisham 012E was E01003296 Lewisham 012E now should be E01034392 Lewisham 040A E01034192 Greenwich 040E was E01001668 Greenwich 036C now should be E01034191 Greenwich 040D E01034173 Greenwich 005F was E01001706 Greenwich 005B now should be E01034173 Greenwich 005F E01034175 Greenwich 034G was E01033726 Greenwich 034E now should be E01034174 Greenwich 034F E01034180 Greenwich 035G was E01001640 Greenwich 035A now should be E01034178 Greenwich 035E E01034185 Greenwich 037G was E01001662 Greenwich 037A now should be E01034183 Greenwich 037E E01034190 Greenwich 040C was E01001667 Greenwich 036B now should be E01034188 Greenwich 040A E01034194 Greenwich 041B was E01033733 Greenwich 036E now should be E01034189 Greenwich 040B E01034195 Greenwich 041C was E01033733 Greenwich 036E now should be E01034189 Greenwich 040B E01034196 Greenwich 041D was E01033733 Greenwich 036E now should be E01034189 Greenwich 040B E01034207 Newham 033G was E01003482 Newham 033B now should be E01034205 Newham 033E E01034228 Newham 042E was E01033577 Newham 037E now should be E01034227 Newham 042D E01034267 Wakefield 024H was E01011775 Wakefield 024A now should be E01034266 Wakefield 024G

Bump certifi from 2019.11.28 to 2022.12.7

opened on 2022-12-08 04:35:01 by dependabot[bot]

Bumps certifi from 2019.11.28 to 2022.12.7.

Commits

9e9e840 2022.12.07
b81bdb2 2022.09.24
939a28f 2022.09.14
aca828a 2022.06.15.2
de0eae1 Only use importlib.resources's new files() / Traversable API on Python ≥3.11 ...
b8eb5e9 2022.06.15.1
47fb7ab Fix deprecation warning on Python 3.11 (#199)
b0b48e0 fixes #198 -- update link in license
9d514b4 2022.06.15
4151e88 Add py.typed to MANIFEST.in to package in sdist (#196)
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/mhl/postcodes-mapit/network/alerts).

Bump lxml from 4.6.1 to 4.9.1

opened on 2022-07-06 19:38:50 by dependabot[bot]

Bumps lxml from 4.6.1 to 4.9.1.

Changelog

Sourced from lxml's changelog.

4.9.1 (2022-07-01)

Bugs fixed

A crash was resolved when using iterwalk() (or canonicalize()) after parsing certain incorrect input. Note that iterwalk() can crash on valid input parsed with the same parser after failing to parse the incorrect input.

4.9.0 (2022-06-01)

Bugs fixed

GH#341: The mixin inheritance order in lxml.html was corrected. Patch by xmo-odoo.

Other changes

Built with Cython 0.29.30 to adapt to changes in Python 3.11 and 3.12.

Wheels include zlib 1.2.12, libxml2 2.9.14 and libxslt 1.1.35 (libxml2 2.9.12+ and libxslt 1.1.34 on Windows).

GH#343: Windows-AArch64 build support in Visual Studio. Patch by Steve Dower.

4.8.0 (2022-02-17)

Features added

GH#337: Path-like objects are now supported throughout the API instead of just strings. Patch by Henning Janssen.

The ElementMaker now supports QName values as tags, which always override the default namespace of the factory.

Bugs fixed

GH#338: In lxml.objectify, the XSI float annotation "nan" and "inf" were spelled in lower case, whereas XML Schema datatypes define them as "NaN" and "INF" respectively.

... (truncated)

Commits

d01872c Prevent parse failure in new test from leaking into later test runs.
d65e632 Prepare release of lxml 4.9.1.
86368e9 Fix a crash when incorrect parser input occurs together with usages of iterwa...
50c2764 Delete unused Travis CI config and reference in docs (GH-345)
8f0bf2d Try to speed up the musllinux AArch64 build by splitting the different CPytho...
b9f7074 Remove debug print from test.
b224e0f Try to install 'xz' in wheel builds, if available, since it's now needed to e...
897ebfa Update macOS deployment target version from 10.14 to 10.15 since 10.14 starts...
853c9e9 Prepare release of 4.9.0.
d3f77e6 Add a test for https://bugs.launchpad.net/lxml/+bug/1965070 leaving out the a...
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

Bump numpy from 1.19.4 to 1.22.0

opened on 2022-06-21 21:18:25 by dependabot[bot]

Bumps numpy from 1.19.4 to 1.22.0.

Release notes

Sourced from numpy's releases.

v1.22.0

NumPy 1.22.0 Release Notes

NumPy 1.22.0 is a big release featuring the work of 153 contributors spread over 609 pull requests. There have been many improvements, highlights are:

Annotations of the main namespace are essentially complete. Upstream is a moving target, so there will likely be further improvements, but the major work is done. This is probably the most user visible enhancement in this release.

A preliminary version of the proposed Array-API is provided. This is a step in creating a standard collection of functions that can be used across application such as CuPy and JAX.

NumPy now has a DLPack backend. DLPack provides a common interchange format for array (tensor) data.

New methods for quantile, percentile, and related functions. The new methods provide a complete set of the methods commonly found in the literature.

A new configurable allocator for use by downstream projects.

These are in addition to the ongoing work to provide SIMD support for commonly used functions, improvements to F2PY, and better documentation.

The Python versions supported in this release are 3.8-3.10, Python 3.7 has been dropped. Note that 32 bit wheels are only provided for Python 3.8 and 3.9 on Windows, all other wheels are 64 bits on account of Ubuntu, Fedora, and other Linux distributions dropping 32 bit support. All 64 bit wheels are also linked with 64 bit integer OpenBLAS, which should fix the occasional problems encountered by folks using truly huge arrays.

Expired deprecations

Deprecated numeric style dtype strings have been removed

Using the strings "Bytes0", "Datetime64", "Str0", "Uint32", and "Uint64" as a dtype will now raise a TypeError.

(gh-19539)

Expired deprecations for loads, ndfromtxt, and mafromtxt in npyio

numpy.loads was deprecated in v1.15, with the recommendation that users use pickle.loads instead. ndfromtxt and mafromtxt were both deprecated in v1.17 - users should use numpy.genfromtxt instead with the appropriate value for the usemask parameter.

(gh-19615)

... (truncated)

Commits

4adc87d Merge pull request #20685 from charris/prepare-for-1.22.0-release
fd66547 REL: Prepare for the NumPy 1.22.0 release.
125304b wip
c283859 Merge pull request #20682 from charris/backport-20416
5399c03 Merge pull request #20681 from charris/backport-20954
f9c45f8 Merge pull request #20680 from charris/backport-20663
794b36f Update armccompiler.py
d93b14e Update test_public_api.py
7662c07 Update init.py
311ab52 Update armccompiler.py
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

Bump ipython from 7.19.0 to 7.31.1

opened on 2022-01-21 18:56:49 by dependabot[bot]

Bumps ipython from 7.19.0 to 7.31.1.

Commits

e321e76 release 7.31.1
67ca2b3 Merge pull request from GHSA-pq7m-3gw7-gq5x
2794330 back to dev
be343e7 release 7.31.0
0fcf2c4 Merge pull request #13428 from meeseeksmachine/auto-backport-of-pr-13427-on-7.x
b8db9b1 Backport PR #13427: wn 731
7f253dc Merge pull request #13412 from bnavigator/backport-inspect
4f26796 fix xxlimited_35 import name
77ca4a6 don't run nose-based iptest on py310, only pytest
533e509 back to decorator skip
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

Bump django from 3.1.3 to 3.1.13

opened on 2021-09-22 17:39:53 by dependabot[bot]

Bumps django from 3.1.3 to 3.1.13.

Commits

43873b9 [3.1.x] Bumped version for 3.1.13 release.
0bd57a8 [3.1.x] Fixed CVE-2021-35042 -- Prevented SQL injection in QuerySet.order_by().
8dc1cc0 [3.1.x] Added stub release notes for 3.1.13.
1471ec4 [3.1.x] Fixed docs header underlines in security archive.
6022181 [3.1.x] Added CVE-2021-33203 and CVE-2021-33571 to security archive.
064c0c5 [3.1.x] Post-release version bump.
625d3c1 [3.1.x] Bumped version for 3.1.12 release.
203d4ab [3.1.x] Fixed CVE-2021-33571 -- Prevented leading zeros in IPv4 addresses.
20c67a0 [3.1.x] Fixed CVE-2021-33203 -- Fixed potential path-traversal via admindocs'...
aa8781c [3.1.x] Confirmed release date for Django 3.1.12, and 2.2.24.
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

Mark Longair

GitHub Repository

A MapIt instance for approximate postcode boundaries based on the Voronoi diagram of ONSPD

Voronoi Postcodes MapIt

How to generate the data

Download NSUL

Create the database and schema

Load the NSUL data into the database

Calculate the Voronoi diagram

Generate postcode GeoJSON files

Package up the data

Import the data into MapIt

Contact

Issues

LSOA21

Bump certifi from 2019.11.28 to 2022.12.7

Bump lxml from 4.6.1 to 4.9.1

4.9.1 (2022-07-01)

Bugs fixed

4.9.0 (2022-06-01)

Bugs fixed

Other changes

4.8.0 (2022-02-17)

Features added

Bugs fixed

Bump numpy from 1.19.4 to 1.22.0

v1.22.0

NumPy 1.22.0 Release Notes

Expired deprecations

Deprecated numeric style dtype strings have been removed

Expired deprecations for loads, ndfromtxt, and mafromtxt in npyio

Bump ipython from 7.19.0 to 7.31.1

Bump django from 3.1.3 to 3.1.13

Mark Longair

Expired deprecations for `loads`, `ndfromtxt`, and `mafromtxt` in npyio