Guides
Nudge units and public messaging: what New York City behavioural insights work shows
What New York City's Behavioral Design Team reports from redesigned letters, emails and reminders, including the trials that worked, the ones that did not and how to read the numbers.
What to take away
- New York City runs a Behavioral Design Team (BDT) through NYC Opportunity, the Mayor's Office for Economic Opportunity, in partnership with the nonprofit ideas42. It launched in 2016 and has worked with more than a dozen City agencies.
- Much of its work is public messagingredesigned letters, notices, envelopes, emails and text reminders, tested with randomized controlled trials (RCTs).
- The City reports real gains in several trials, for example misfiled 311 requests falling from 59% to 9%, and some trials with no significant effect.
- Reported figures mix relative changes and percentage-point changes. Do not compare them without checking which one is meant.
- A tested message is stronger evidence than a persuasive-sounding one. The null results are as useful as the wins.
What a city nudge unit is
A nudge unit applies behavioral science to the way a government communicates and designs services. The idea is that small changes to a letter, a form or a reminder can change what people do, without changing the rules or the incentives. The comparison of nudge theory and persuasion separates a nudge, which changes the choice environment, from persuasion, which tries to change belief.
New York City's version is the BDT. NYC Opportunity describes its approach as learning from residents about how they use government services, adapting services to their context and to common human decision-making biases, and evaluating the changes with rigorous methods. The methods it lists include RCTs, redesigned printed and digital communications, text and automated phone reminders, and qualitative work such as interviews, surveys and communications audits.
The techniques the City names are loss aversion, social norms, plan-making prompts, enhanced active choice, defaults, self-affirmation and simplified steps or icons. Notice that most of these concern wording, order and timing. That is why the work is relevant to anyone who writes public-facing messages.
What the results show
The BDT page lists project results by agency. The selection below shows both the range of messages tested and the size of the reported effects.
| Project | What was tested | Reported result |
|---|---|---|
| 311 mobile app | Redesigned menu and feedback language | Misfiled service requests fell from 59% to 9% |
| Finance, parking tickets | Redesigned envelope at 90 days | Drivers were 9.7 percentage points less likely to enter judgment |
| Finance, ECB fines | Redesigned letter | Recipients were 6.7 times more likely to take steps to resolve their debt |
| Finance, vehicle booting | Redesigned warning email | Reduced boot eligibility by 10.4% |
| Human Resources Administration, SNAP recertification | Reminder | Reduced failure to submit forms by 5.5% |
| Human Resources Administration, SNAP periodic report | Enhanced active choice email | Raised engagement by 30.1% |
| Education, Gifted & Talented testing | Redesigned emails | Raised testing requests by 5.0% overall and 9.0% in low-income districts |
Two points follow from the table.
Different units. A 9.7 percentage point change in judgment is a difference between two rates. A 10.4% reduction in boot eligibility is a relative change. A reader who treats both as the same kind of number will misjudge the size of the effect.
Different baselines. The same relative change means more where the problem is common. A message that changes the behavior of a large group by a small amount can matter more than a large effect on a few people.
The trials that did not work
The BDT page also reports results that were null or inconclusive:
- A redesigned Universal Access to Counsel flyer showed no statistically significant difference in uptake of legal assistance compared with the original.
- A redesigned housing assistance notice showed no significant differences from the original.
- An RCT on licensed businesses filing online taxes found no statistically significant effect, although about 30 additional businesses registered after receiving the behavioral email compared with the usual one.
- A Taxi and Limousine Commission mailing led to drivers being 15% more likely to enroll in a wheelchair accessible vehicle incentive, but the page says the difference was not statistically significant.
- A tax preparation portal trial was underpowered, though the redesigned website was scaled citywide.
- In an FDNY residency points trial, social norms and loss aversion messages did not differ significantly.
These entries matter for two reasons. First, they show that a message redesign is a hypothesis, not a guarantee. Second, they show that the City reports tests that did not confirm the idea, which is what you want from a unit that is supposed to be learning.
What this means for public messaging
Test before you scale
Many entries on the BDT page record a scale decision after a trial: a mailer scaled citywide, a notice rolled out to all eligible households, a second email scaled across a program. The pattern is run a small randomized test, read the result, then change the standard communication. That sequence is the strongest practice the page models.
Change one thing at a time
Where the page describes trials, the arms are original and redesigned versions. When two versions differ in several ways, you cannot tell which change did the work. Keep the test narrow if you want a result you can reuse.
Read the unit of measurement
Before quoting a result, check whether it is a rate, a percentage point change, a relative change or a multiple. The ECB result, 6.7 times more likely to take steps, is a ratio of likelihoods and says nothing about how many people took steps in total.
Match the message to a clear action
Several of the successes concern a specific next step: file this form, respond to this survey, resolve this debt. The Mayor's Office of Resiliency "last chance" survey letter, for example, increased survey completion by 4.6 percentage points according to the page. Messages that ask for one thing give the trial a clear outcome to measure.
Do not stop at the message
Some projects on the page changed tools and process rather than wording, such as the Human Resources Administration staff dashboards and the Housing Preservation and Development recommendations that informed a new housing portal. Treat messaging as one part of the service, not the whole of it.
Limits of the evidence
- Reported by the sponsor.The figures come from the City's own page. They are useful, but they are summaries, and the full case studies carry more detail on trial design.
- Context-specific.A redesigned parking ticket envelope in New York tells you about drivers in New York receiving that notice. It does not guarantee the same effect on a different audience.
- Short-term outcomes.Most reported results measure a response soon after the message. They do not show whether the effect lasts.
- Ethics.A nudge that steers people toward an outcome they would not choose if they understood the design is a different matter. The dark patterns versus persuasion article covers where the legal and ethical line sits.
For private organizations, the practical lesson is the same: write the message, run a controlled comparison, report the null results and then change the standard version. Social cues are one tool in that kit, and social proof is a related principle that sits under the "social norms" heading the City lists.
