Measuring Clean Cities

1Why city cleaning needs quality assurance

A quality assurance system tells a city how clean its public spaces actually are, why, and what the contractor should be paid for it. Most cleaning programmes have inspection; few have quality assurance.

This paper draws on a quality assurance system DAIS designed for a national city cleaning programme covering several regional city authorities and their municipalities. The system was designed for national roll-out and piloted in one major city. The pilot tested the tools and the authority's ability to run them. For this paper we have updated the design for cities in any country: the index grades outcomes only, uses five grades and pays on a sliding scale.

A review of contracts, organisation, operations, finance and supervision found four gaps:

• Contracts bought quantities, not quality: they specified numbers of workers and machines and how often each service ran. They said little about the result residents should see.
• Monitoring grew without quality growing: contracts relied on large numbers of unskilled workers (e.g. for litter picking and manual sweeping), so authorities needed large teams of inspectors to watch them. Costs rose; cleanliness did not rise with them.
• Information was thin: with weak IT tools, audit records were lost, past lessons could not be reused, and residents had few ways to report problems or rate the service.
• Roles overlapped: consultants hired to supervise contractors also advised the authority, and each authority used its own standards. Results could not be trusted or compared.

The design answered with a "best value" approach in five moves: design a target model from international good practice, engage residents to measure their satisfaction, build the tools (checklists, visual standards, key performance indicators or KPIs), automate the audit, and pilot before roll-out.

2The system: a closed loop in six steps

The system is a closed loop: each month's results change what is checked, where and how often next. Inspectors audit every week, the contractor is paid every month on a three-month average, and results are published every quarter. Six steps run in sequence, fed by residents and by the contractor.

Results feed back into the next cycle: weights are reviewed yearly, samples are resized to keep the margin of error small, and visits increase where performance slips.

1. Design: checklists cover eight elements residents can see, from containers to graffiti. Each element is graded on specific criteria, with a visual standard for every grade.
2. Sample: sites are drawn at random within zones, and complaint hot spots are sampled as a group of their own. An app allocates each week's audit rounds to inspectors at random times, including evenings and weekends, taking each inspector's capacity into account.
3. Audit: inspectors grade each criterion from A to E on an electronic checklist and attach photos as evidence. Every C, D or E grade becomes a work order with a deadline. Residents' complaints and periodic surveys feed the same database.
4. Score: the database rolls the grades up into the City Cleaning Index, weighting zones by population, and calculates KPIs on cost, timeliness and satisfaction.
5. Pay: the monthly payment slides with the three-month average index, with a bonus for sustained excellence and penalty points for missed deadlines.
6. Compare: results are benchmarked across zones, districts and cities, and the lessons turn into new policies and public campaigns.

Who does what

3The City Cleaning Index: how it is calculated

The City Cleaning Index measures how close a city's public spaces come to the best grade, from 0 to 100. It is built in three steps: criteria roll up into elements, elements into a score for each zone, and zones into the city index.

The grading scale

The index grades outcomes only: what a resident can see. Eight elements cover containers, pavements, streets, verges, public furniture, illegal dumping and graffiti. Every criterion is graded on five levels. Each level has a written definition and a reference photo in a visual standard, so two inspectors at the same site give the same grade. Grade B is the service standard written into the contract, and it scores 75.

The minimum standard is grade B or better at 90% of sampled sites, reported each month as the compliance rate beside the index. Any C, D or E grade triggers corrective work within a set time (section 5).

Scoring B at 75 means a contractor who meets the standard everywhere earns full payment, and five grades keep an even 25-point step between them, so no single grade change causes a cliff.

The weights

Element weights reflect what residents notice most, measured in a baseline satisfaction survey. A survey matters because what residents notice is not what occurs most often. For instance, Keep Britain Tidy's 2012 survey found dog fouling at about 9% of surveyed sites, yet 64% of people named it the most offensive item; detritus failed at 39% of sites and ranked as a low public priority.

Eight outcome elements make up 100% of the index. Weights are illustrative and should be set from a local survey.

The calculation

A worked example

Take one container site. Its grades give a site score of 67.5, below the standard of 75: waste around the container and a dirty container pull it down.

Averaging all sampled container sites in a zone gives that zone's container score, and the eight element scores give each zone's index. The city index then weights zones by population. In this illustrative month the city scores 73.0:

Zone scores are illustrative. Hot spots and informal settlements pull the index down most. Weighting by area instead would let large, lightly populated zones dominate.

By element, containers score lowest (67.6) and main streets highest (81.2); because containers carry 30% of the weight, they cost the city the most points. Zones 2 and 4 and the hot spots are below 75: if they stay there for two months running, each triggers a corrective plan even if the city index recovers (section 4).

4From index to payment

The index sets the monthly payment on a sliding scale. There are no fixed bands, so a fraction of a point never moves a whole band of money.

Full payment from 75 to 85; 1% deducted for each point below 75, up to 25%; a bonus of 0.5% for each point above 85, up to 5%. Parameters are illustrative.

Monthly payment, three months of evidence

Contractors submit monthly invoices, but one month's sample is too small to pay on, and a full quarter is too slow to correct. We recommend auditing continuously and paying on a rolling average:

• Audit every week, at random sites and times, including evenings and weekends.
• Pay each month on the average of the last three monthly indices, which carries a full quarter's sample.
• Charge missed correction deadlines in the month they occur.
• Publish the index, compliance rate and satisfaction by zone every quarter, and recalibrate weights and visual standards every year.

Giving the benefit of the doubt

Every index is published with its margin of error. Deductions are calculated on the upper end of that margin and bonuses on the lower end, so sampling noise never costs the contractor money or pays a bonus the streets have not earned.

Caps and persistent failure

Deductions must be large enough to change behaviour and small enough to stay proportionate. Neither the UK nor Australia sets a statutory cap; contracts set their own, and caps set too low can leave most of the fee payable even when service stops. We recommend three limits, calibrated to the contract's margin:

• An index deduction of up to 25% of the monthly invoice, reached at an index of 50.
• Penalty points worth up to 10%.
• A combined cap of 30%. Beyond it, deductions stop growing and the contract moves to remedies: a formal warning when the index is below 50 or the cap is reached, and a right to step in or terminate if that happens in three of any six months.

Deductions should be written as a proportionate adjustment for service not delivered, not as a penalty.

Floors that stop the index hiding failure

One city number can let clean districts mask a failing one. Three rules prevent it:

• Report scores by zone and element to the authority every month and publish them every quarter.
• Any zone or element below 75 for two months running triggers a corrective plan, whatever the city index says.
• Pay the bonus only when no zone is below 75 and at least 90% of sites reach grade B.

5Correcting failures on the ground

A grade is also a work order. When a site falls to C, D or E, the authority intervenes and the contractor must restore it within a set time, shorter where more people pass and where the failure is worse:

Response times are illustrative.

The inspector's app opens the work order and the contractor closes it with a photo as evidence. A missed deadline earns penalty points: 2 for a first miss, 3 for a repeat at the same site. Points convert to a deduction on a sliding scale, for example 0.1% of the monthly invoice for each point above a monthly allowance of 20, up to the 10% cap. Points are recorded automatically, which removes the manual follow-up that made old daily fines slow and loosely applied. The grade itself affects only the index, so the same failure is never charged twice.

6Sampling: random, zoned and weighted by population

The sample has to be large enough to be fair, random enough that the contractor cannot clean only the audit route, and weighted to where people live and complain.

• Random within zones: Sites and times are drawn afresh each week, so no street can be cleaned "for the inspector".
• Hot spots are a group of their own: Their sites are sampled and scored separately, so the extra visits do not drag down the zone around them, and the hot-spot score is published on its own.
• Hot spots move: When an area falls out of the top quartile, it returns to its zone's normal sample.
• Samples follow performance: Where results stay stable and good, samples can shrink; where they slip, they grow.

Why population, and its traps

Weighting by population makes the index reflect what residents experience; weighting by area would let large, lightly populated zones dominate. Population weights have their own traps, and each has a fix:

• Central areas have few residents but many visitors: Weight the central zone by daytime population or footfall, not by residents.
• Population data goes stale: Update weights yearly from census or utility data, and review zone boundaries as the city grows.
• Small zones give volatile scores: Set a minimum number of sites per zone, whatever its weight.
• Contractors may favour heavily weighted zones: The zone floor in section 4 stops a low-weight zone being neglected.

Fairness, seasons and events

Complaint data is not neutral. Residents of poorer districts report problems less often even where conditions are worse. Hot spots are therefore defined from complaints per resident and failed audits together, and informal settlements keep a minimum sample whatever their complaint count. The contract should also say which seasons, festivals and weather events, such as sandstorms, adjust the response times or suspend the score.

7Benchmarks: what to borrow and what to adapt

The grading idea is not new. England's Code of Practice on Litter and Refuse (Defra, fourth edition, 2026) grades areas from A to D and sets response times by intensity of use. England's national survey reports both an average grade and the share of sites below grade B, across litter, detritus, weeds, staining, graffiti and fly-posting. New York's Scorecard counts only its two cleanest ratings as acceptably clean. In each case the acceptable grade is the pass mark. The system described here borrows that logic and extends it from a duty standard to a payment mechanism.

Sources: Defra, Code of Practice on Litter and Refuse, fourth edition (March 2026), England; Local Environmental Quality Survey of England methodology; Office of the New York State Comptroller, Street and Sidewalk Cleanliness audit (2020).

Three rules for adapting a benchmark

1. Borrow the scale, calibrate the standard: Take the graded logic, but create your local visual standards. In arid cities, sand and grit need their own criterion.
2. Borrow the zones, set your own samples: Response times suit a duty standard; sample sizes and weights suit a payment mechanism. A contract needs both.
3. Do not import thresholds: The 90% compliance target, the payment slope and the caps should be set from a local baseline, measured during the shadow period (section 9).

From samples to census

Brussels now scores streets with cameras on refuse trucks: in June 2026 its cleanliness agency expanded a trial to 16 trucks and 700 km of roads, feeding a 0–5 Clean City Index. Los Angeles set out in 2016 to rate every street each quarter from dash-cam video. Automation widens coverage and improves accuracy, but it does not replace the visual standard: reference photos for each grade are what cameras are trained and audited against, and photo evidence still settles disputes.

8Ten mistakes that undermine city cleaning quality assurance

Most failures come from what the contract measures and who measures it, not from the formula. Each mistake below appeared in, or was designed out of, the city cleaning contracts and practices this paper draws on.

A contractor can meet every quantity in the contract while the street outside stays dirty.

9Getting started: a shadow period and the tender cycle

Start in shadow: Run the full system for six months with no money attached: audits, index, work orders and reports. Use the period to test whether two inspectors give the same grade, refine the visual standards, size the samples and measure the baseline from which the 90% target, the payment slope and the caps are set. Share the shadow results with the contractor, so the first paid month holds no surprises.

Time it to the tender: City cleaning contracts often run three to seven years. When the national system in this paper was ready, most regional authorities had already tendered their contracts, so only a minority could adopt it at the next round. Authorities mid-contract can agree a variation, start with the bonus side only, or run the index in shadow until renewal.

10What to do: a checklist for city authorities

Settle these twelve points before and during a cleaning contract. They are ordered as a programme runs.