tap-onspd is a Singer tap for the ONS Postcode Directory (ONSPD), the Office for National Statistics' quarterly file of every UK postcode. Each postcode comes with its latitude/longitude and the geographies it falls in: country, region, county, local authority, ward, constituency, LSOA, IMD rank and more. The tap also loads the code-to-name lookups that ship with each release.
Built with the Meltano Singer SDK.
Each ONSPD release is published as an ArcGIS Online item (owner ONSGeography_data, tag PRD_ONSPD). The item's data is a zip of about 250 MB that unzips to about 1.5 GB of CSV.
- Finding the release. The tap asks the ArcGIS search API for the newest ONSPD item.
- Discovery needs the CSV headers to build schemas. It reads them straight from the remote zip with HTTP range requests, transferring about 1 MB in total. Nothing is downloaded to disk.
- Sync.
- The zip is downloaded to
tap-onspdin the system temp directory. It is streamed in 8 MB chunks to<name>.<modified>.zip.part, where<modified>is the item's last-modified time, and an interrupted download resumes with aRangerequest. Once the file is complete and passes a zip check, it is renamed to<name>.<modified>.zip. - A zip that is already there is reused, so a later run on the same machine does not download it again. A release that ONS republishes gets a new modified time, so it is downloaded again.
- Rows are then streamed out of the zip one at a time (
zipfile+csv). Nothing is extracted to disk, so memory and disk use stay flat.
- The zip is downloaded to
Release bookmark. Every record carries _onspd_modified_at, the release's last-modified time, which is used as the replication key. When the bookmark already holds the current release, streams emit nothing and the zip is never downloaded. A scheduled run between releases therefore finishes in seconds. To load the release again, run the tap without --state, or with meltano run --full-refresh.
| Stream | Source file in the zip | Primary key |
|---|---|---|
postcodes |
Data/ONSPD_<MON>_<YYYY>_UK.csv |
pcds |
countries |
Documents/CTRY …csv |
ctry_cd |
regions |
Documents/RGN …csv |
rgn_cd |
counties |
Documents/CTY …csv |
cty_cd |
local_authorities |
Documents/LAD …csv |
lad_cd |
wards |
Documents/WD …csv |
wd_cd |
constituencies |
Documents/PCON …csv |
pcon_cd |
Every record also has _onspd_release (e.g. AUG_2026), _onspd_item_id and _onspd_modified_at.
Schemas come from each file's header, so columns that ONS adds in a later release show up without a code change. Use Meltano select to choose the columns you need.
latandlongare numbers, and theimdNNindranks are integers. Everything else is a string, which keeps leading zeros (e.g. innorth1m).- Empty values become
null. - Postcodes with no grid reference have
lat = 99.999999andlong = 0.
Column names.
- ONSPD puts the geography vintage in its column names (
lad26cd= local authority code on 2026 boundaries), and the vintage moves on every year. Withstrip_vintage_suffixes(default on), the year is dropped (lad26cd→lad_cd,RGN25NM→rgn_nm), so names stay stable and lookup codes line up withpostcodescolumns. - Columns that exist in several vintages keep their original names, because stripping the year would make them collide (
oa11cd/oa21cd,imd20ind/imd25ind). All names are lower-cased.
Lookup file choice. When several files match a lookup pattern, the one with the latest as at MM_YY date is used.
County. ONSPD's county code is only set for two-tier county council areas. Unitary authorities get the pseudo code E99999999. Fall back to the local authority if you need a county for every postcode.
Channel Islands and Isle of Man. ONSPD also lists postcodes for these Crown Dependencies (L93000001, M83000003). They are not part of the UK, so the tap leaves them out by default as well.
| Setting | Default | Description |
|---|---|---|
exclude_countries |
["northern_ireland", "channel_islands", "isle_of_man"] |
Countries to leave out, from england, scotland, wales, northern_ireland, channel_islands and isle_of_man (default: Northern Ireland, see Licence, plus the Channel Islands and Isle of Man) |
strip_vintage_suffixes |
true |
Drop the vintage year from column names |
Run tap-onspd --about for the full list, including the SDK's built-in settings.
tap-onspd --about
tap-onspd --config config.json --discover > catalog.json
tap-onspd --config config.json --catalog catalog.jsonWith Meltano, this repo's meltano.yml is a dev project:
meltano install
meltano run tap-onspd target-jsonlRun time. Discovery takes about 20 s. A full load takes about 30 s to download, then the time to stream about 2.7M postcode rows; with a database loader, most of the time goes on inserting them.
The tap's code is licensed under Apache-2.0 (see LICENSE). This repository contains no ONSPD data: the tap downloads it from ONS each time it runs. The test fixtures contain only a few sample rows.
ONSPD is covered by the ONS's own terms, not by the tap's licence. Anyone who loads it with this tap must follow them. The terms below come from section 3 of the ONSPD User Guide (Licensing Requirements – Copyright and Reproduction, shipped in the release zip under User Guide/) and the ONS Licences page.
-
Licence: ONS postcode products are derived from Code-Point® Open and published under the Open Government Licence v3.0. The data can be re-used free of charge in any format or medium, commercially or not, under the data owners' terms. ONS and data-owner logos are not covered.
-
Attribution (required). The following statements must be acknowledged or displayed wherever the data is used, including reports, dashboards and apps built on the loaded tables. Use the year of the release you loaded;
_onspd_releaseholds it, e.g.AUG_2026→ 2026.Contains OS data © Crown copyright and database right 2026
Contains Royal Mail data © Royal Mail copyright and database right 2026
Source: Office for National Statistics licensed under the Open Government Licence v.3.0
-
Northern Ireland (
BTpostcodes):- NI data is excluded from the free re-use above. It comes with Land & Property Services' Northern Ireland End User Licence, which allows internal business use only, and using it counts as accepting those terms.
- Commercial use requires a separate licence direct from LPS (see LPS copyright, licensing and publishing).
- The tap excludes NI by default (
northern_irelandinexclude_countries). Only remove it if your use falls within the End User Licence, or you hold an LPS licence.
uv sync
uv run pytestThe tests use a small fixture zip shaped like a real release and a mocked ArcGIS API, so they need no network access.