Case Study 02 — Resident opinion surveys
Can AI personas reproduce
municipal resident opinion surveys?
We reproduced 8 questions from real resident opinion surveys in Yokohama, Fukuoka, Sapporo, and Nagoya
with 200 AI personas each and validated the accuracy.
The results on this page were produced with the latest PersonaChoice v3.0. Evaluation uses a unified standard: “how highly PersonaChoice ranks the items that performed best in the real survey” (Top-K agreement, Top-1 match).
70–80%
Reproduction accuracy for top concerns
4 cities
Yokohama, Fukuoka, Sapporo, Nagoya
8 questions
Questions validated
100%
Rated B or above (8/8)
This page presents validation based on resident opinion surveys officially conducted and published by four municipalities across Japan. By comparing how closely PersonaChoice’s AI simulation matches actual resident responses, we assess its usefulness and reliability as a complementary tool for municipal resident surveys.
- This validation uses 200 AI personas per task as respondents. The AI personas are generated by a proprietary method from official statistics such as the census, with age, gender, occupation, income, municipality, and personality traits.
- The original resident surveys allow multiple answers (e.g., “choose up to 3”), but PersonaChoice converts them into a “choose the single most important” format. Therefore, the comparison uses “overlap among the top-ranked items (Top-K agreement)” rather than the absolute selection rates as the main metric.
- All simulation result reports run during the validation are public. From each city’s “task report” link, you can view the PersonaChoice report screen directly (reports are in Japanese).
Evaluation criteria
| Grade | Criteria |
|---|---|
| A | Agreement ≥ 80% + Top-1 match, or agreement 100% |
| B+ | Agreement ≥ 80%, or agreement ≥ 60% + Top-1 match |
| B | Agreement ≥ 60% |
| C | Agreement ≥ 40% |
| D | Below the above |
* To rule out chance matches, the rank K used for evaluation varies with the number of options (rule of thumb: K / number of options ≤ 50%). Top-5 for 10+ options, Top-3 for 6–9 options (e.g., 6- and 8-option questions use Top-3).
Yokohama R5 citizen opinion survey
Source: City of Yokohama / 2,572 valid responses / 200 AI personas (18 wards)
Result: View the detailed report (JA)
Q1: Worries (original Q9, 13 options)
Grade AQuestionChoose the single thing you have worried about or struggled with most regarding your own or your family’s life recently.
Options(1) Own illness/health and old age (2) Family illness/health and life issues (3) Unemployment/bankruptcy or falling income (4) Work and workplace (5) The economy and cost of living (6) Childcare and education (7) Children’s marriage and employment (8) Odor/noise from neighbors (9) Environmental issues (10) Accidents and disasters (11) Crime and safety (12) Housing (13) No worries or troubles
| Rank | Real survey top 5 | PersonaChoice top 5 (Top-K agreement) | Match |
|---|---|---|---|
| 1 | Own health & old age | Own health & old age Top-1 match | ✓ |
| 2 | Family health | Work & workplace | ✓ |
| 3 | Economy & cost of living | Economy & cost of living | ✓ |
| 4 | Work & workplace | Accidents & disasters | |
| 5 | Childcare & education | Family health | ✓ |
✓ = item present in both the real survey’s Top 5 and PersonaChoice’s Top 5 (regardless of rank)
Top-5 agreement: 80% (4/5) / Top-1: match
From 13 options, PersonaChoice correctly identified 4 of the real survey’s top 5, with an exact match on #1 “own health & old age.” A good example where the AI personas reproduced how worries differ by age (younger → work, older → health).
Q2: Global skills (original Q16, 6 options)
Grade AQuestionChoose the single skill you consider most necessary to be active globally.
Options(1) Language and communication skills (2) Openness to exchange and social participation with foreigners (3) Understanding of diversity and other cultures (4) Ability to convey attachment to and appeal of Yokohama (5) Broad knowledge of international affairs (6) A sense of ownership of global issues toward a decarbonized society
| Rank | Real survey top 3 | PersonaChoice top 3 (Top-K agreement) | Match |
|---|---|---|---|
| 1 | Language & communication | Language & communication Top-1 match | ✓ |
| 2 | Understanding diversity & other cultures | Understanding diversity & other cultures | ✓ |
| 3 | Broad knowledge of international affairs | Broad knowledge of international affairs | ✓ |
Top-3 agreement: 100% (3/3) / Top-1: match
The top 3 ranks matched exactly. Even with only 6 options, the ranking reproduction was perfect.
Fukuoka R7 1st city survey
Source: City of Fukuoka / 559 valid responses / 200 AI personas (7 wards)
Result: View the detailed report (JA)
Q1: Scenery (original Q23, 8 options)
Grade BQuestionChoose the single scenery/view of Fukuoka you think is best.
Options(1) Urban scenery (office and commercial buildings) (2) Historic scenery (shrines, temples, historic sites) (3) Artistic scenery (outdoor art, monuments) (4) Local tradition and culture (festivals, events) (5) People and nature (sea, mountains, parks, farmland) (6) Night scenery (light-ups, illuminations) (7) Outdoor advertising (signs, signage) (8) None in particular
| Rank | Real survey top 3 | PersonaChoice top 3 (Top-K agreement) | Match |
|---|---|---|---|
| 1 | Urban scenery | People and nature | ✓ |
| 2 | People and nature | Local tradition & culture | ✓ |
| 3 | Local tradition & culture | Historic scenery |
Top-3 agreement: 67% (2/3)
In the real survey, “urban” and “natural” scenery were almost tied (62.1% / 60.3%). With 8 options, we evaluate at Top-3 to avoid chance matches, and identified 2 of the top 3 (people-and-nature, local tradition/culture).
Q2: Expectations for parks (original Q18, 12 options)
Grade B+QuestionChoose the single thing you most want for Maizuru Park / Ohori Park.
Options(1) Easier-to-walk paths (2) Brighter park (trees make it dark) (3) Better management of the waterside (moat) (4) Better views and sightlines (5) More facility/history signage and guidance (6) More shops and restaurants (7) A landmark attraction as a tourism hub (8) Clearer entrances/exits (9) More events (10) More parking (11) More benches (rest space) (12) None in particular
| Rank | Real survey top 5 | PersonaChoice top 5 (Top-K agreement) | Match |
|---|---|---|---|
| 1 | Benches (rest space) | Benches (rest space) Top-1 match | ✓ |
| 2 | Shops and restaurants | Easier-to-walk paths | ✓ |
| 3 | Parking | More signage/guidance | |
| 4 | Waterside management | Waterside management | ✓ |
| 5 | Easier-to-walk paths | Views and sightlines |
Top-5 agreement: 60% (3/5) / Top-1: match
PersonaChoice also hit the real survey’s #1 “benches (rest space).” Meanwhile, everyday conveniences residents value such as “shops” and “parking” did not make PersonaChoice’s top ranks — an informative gap for understanding the AI’s characteristics.
Sapporo R6 2nd / R7 4th citizen surveys
Source: City of Sapporo (R6) / (R7) / 200 AI personas (10 wards)
Result: View the detailed report (JA)
Q1: Priority PR themes (R6 Q2, 14 options)
Grade B+QuestionChoose the single theme you most want the city PR magazine “Koho Sapporo” to focus on.
Options(1) Disaster prevention (2) Public facilities (3) Elderly welfare (4) Health & food (5) Community development activities (6) Culture & arts (7) Garbage (8) Children & childcare (9) Environment & energy (10) Education (11) Sports (12) Welfare for people with disabilities (13) Volunteering (14) None in particular
| Rank | Real survey top 5 | PersonaChoice top 5 (Top-K agreement) | Match |
|---|---|---|---|
| 1 | Disaster prevention | Disaster prevention Top-1 match | ✓ |
| 2 | Public facilities | Health & food | ✓ |
| 3 | Elderly welfare | Garbage | |
| 4 | Health & food | Children & childcare | |
| 5 | Community development | Elderly welfare | ✓ |
Top-5 agreement: 60% (3/5) / Top-1: match
PersonaChoice also hit the real survey’s #1 “disaster prevention.” Responses were naturally distributed across all 14 options.
Q2: Key job conditions (R7 Q19, 13 options)
Grade BQuestionChoose the single most important condition when choosing a company to work for.
Options(1) Job content (2) Salary (income) (3) Days off/leave (4) Working hours (5) Benefits (6) Work location (7) Whether transfers are required (8) Financial health (9) Number of employees (10) Post-hire training environment (11) Corporate philosophy (12) Workplace atmosphere (13) None / don’t know
| Rank | Real survey top 5 | PersonaChoice top 5 (Top-K agreement) | Match |
|---|---|---|---|
| 1 | Salary (income) | Workplace atmosphere | ✓ |
| 2 | Job content | Work location | |
| 3 | Days off/leave | Job content | ✓ |
| 4 | Workplace atmosphere | Working hours | ✓ |
| 5 | Working hours | Whether transfers are required |
Top-5 agreement: 60% (3/5)
Responses were distributed across nearly all 13 options — the most even distribution among all questions. It captured 3 of the top 5, covering conditions related to treatment and work style (job content, days off, workplace atmosphere).
Nagoya R6 63rd public opinion survey
Source: City of Nagoya / 911 valid responses / 200 AI personas (16 wards)
Result: View the detailed report (JA)
Q1: What makes the city proud (Q6, 13 options)
Grade BQuestionChoose the single thing that best fits what you are proud of / like about Nagoya.
Options(1) Geographically convenient access across Japan (2) Nagoya dialect (3) A cultural/artistic atmosphere (4) Signature local foods (“Nagoya meshi”) (5) History tied to the three unifiers (6) High technical level as a manufacturing hub (7) Active economic activity (8) Tasty tap water (9) Beautiful cityscape (10) Wide, well-maintained roads (11) Advanced environmental efforts (12) Thriving sports (13) None in particular
| Rank | Real survey top 5 | PersonaChoice top 5 (Top-K agreement) | Match |
|---|---|---|---|
| 1 | Convenient access | Manufacturing tech level | |
| 2 | Nagoya meshi | Wide, well-maintained roads | ✓ |
| 3 | Wide, well-maintained roads | Convenient access | ✓ |
| 4 | Tasty tap water | Nagoya meshi | ✓ |
| 5 | History of the three unifiers | Active economic activity |
Top-5 agreement: 60% (3/5)
It identified Nagoya meshi, road maintenance, and convenient access, capturing even the locally specific “Nagoya meshi” in the top ranks.
Q2: Worries and anxieties (Q11, 11 options)
Grade B+QuestionChoose the single thing you feel most worried or anxious about.
Options(1) Own health (2) Family health (3) Own life issues (education, jobs, marriage, childcare) (4) Family life issues (5) Current income/assets (6) Outlook for future income/assets (7) Retirement life planning (8) Relationships among family/relatives (9) Relationships with neighbors/community (10) Work and relationships at the workplace (11) Business management issues
| Rank | Real survey top 5 | PersonaChoice top 5 (Top-K agreement) | Match |
|---|---|---|---|
| 1 | Own health | Retirement life planning | ✓ |
| 2 | Retirement life planning | Own health | ✓ |
| 3 | Family health / income outlook | Future income/asset outlook | ✓ |
| 4 | (tied for 3rd) | Current income/assets | ✓ |
| 5 | Current income/assets | Own life issues |
Top-5 agreement: 80% (4/5)
In the real survey, “own health” (61.2%) and “retirement” (57.5%) were close. The members of the top 5 matched 4/5.
Overall results across all 8 questions
| # | City | Question | Top-1 | Top-K agreement | Grade |
|---|---|---|---|---|---|
| 1 | Yokohama | Worries | ✓ | Top-5: 80% | A |
| 2 | Yokohama | Global skills | ✓ | Top-3: 100% | A |
| 3 | Fukuoka | Scenery | Top-3: 67% | B | |
| 4 | Fukuoka | Park expectations | ✓ | Top-5: 60% | B+ |
| 5 | Sapporo | Priority PR themes | ✓ | Top-5: 60% | B+ |
| 6 | Sapporo | Job conditions | Top-5: 60% | B | |
| 7 | Nagoya | City pride | Top-5: 60% | B | |
| 8 | Nagoya | Worries & anxieties | Top-5: 80% | B+ |
Grade distribution: A × 2 / B+ × 3 / B × 3 (B or above: 8/8 = 100%, B+ or above: 5/8 = 62.5%)
The agreement rate for top items averages 75% overall, reaching 100% in the best case (Yokohama, global skills — exact match).
Question types PersonaChoice is suited for — and less suited for
Well-suited patterns
- ✓“Choose the single most ___” type
Choosing the most important from multiple options - ✓Preference-split questions with no right answer
Questions where answers split by taste or values - ✓Naturally split by attributes
Themes where age/gender/income change the choice - ✓Region-specific themes
Nagoya meshi, the three unifiers, summer heat, etc.
Less-suited patterns
- ×Evaluating residents’ personal reality
Satisfaction with family relationships, etc., where AI has no “reality” - ×Behavior volume, awareness, screening status
“Did you read the PR magazine?” “Did you get a check-up?” etc. - ×Questions with a “correct” answer
Satisfaction, attachment, intent to stay — where a socially desirable answer is obvious
Conclusion
PersonaChoice accuracy for resident opinion surveys: 70–80%
75%
Top-item agreement
(overall average)
100%
Agreement
(best case)
100%
Rated B or above (8/8)
For resident opinion surveys, it was demonstrated that PersonaChoice can correctly identify 70–80% of residents’ top concerns. Top-item agreement averages 75% overall, 100% in the best case, with a grade of B or above on all 8 questions.
For product preference and marketing surveys, there is an accuracy track record of 80–90%.
Where PersonaChoice fits
PersonaChoice is not a replacement for real surveys — it is a complement.
Whereas a real survey costs millions to tens of millions of yen and takes months,
PersonaChoice delivers, at low cost, with same-day turnaround, results that reproduce a real survey’s top concerns with a high degree of agreement.
Why not try it for free first?
Experience a resident-opinion simulation right now with 3,000 bonus points (free) for first-time sign-ups.