Survey Sample Size for Small Towns: Why 385 Can Mislead

For a simple yes/no question, sample size depends on your desired margin of error, confidence level, and survey design. That calculation is a starting point, not a finish line. Check your confidence level, your margin of error, and your design effect before you set a target. Population size can affect the calculation, especially for a small population, where a finite-population correction may lower the required sample.

Key Takeaways

Accurate survey sample size depends on key inputs like population size, confidence level, margin of error, and response rate, which must be carefully planned for reliable results.

Table of Contents

• What inputs do you need to calculate survey sample size?
• Which formulas and calculators should you use?
• How do worked examples look for three common survey projects?
• When should you increase sample size for design effect, clustering, and weighting?
• How do you adjust for response rate and nonresponse bias?
• How do you size a survey for subgroups and group comparisons?
• What do common rules of thumb get right and wrong?
• What is the checklist for deciding your sample size?
• What I have learned about sample size in small communities
• How can Southwind Marketing help you plan and run your survey?
• FAQ
• Sources

What inputs do you need to calculate survey sample size?

Every sample-size calculation runs on the same handful of inputs. Pick them on purpose, because each one moves the final number.

• Population size. For large populations, the required sample changes little as the population grows. It matters more when you survey a small group, such as a chamber or a town. Standard calculators, including CDC Epi Info StatCalc, ask for it and apply the correction for you.
• Confidence level. This is how sure you want to be that the true value falls inside your margin of error. Higher confidence generally requires a larger sample.
• Margin of error. This is your precision. A tighter margin generally requires more responses, while a wider margin may suit an early read. Every step toward tighter precision is paid for in budget.
• Expected proportion. If you do not know how people will answer, use 50%. It produces the largest sample, so it is the safe default. Researchers commonly use p = 0.5 when the true prevalence is unknown.
• Design effect (DEFF). This is a multiplier for clustering and weighting. A simple random sample has a DEFF of 1.0. Anything more complex pushes it higher.
• Response rate. The formulas give you completed surveys. You need to invite more people than that, because not everyone answers.

Plan with all six in hand before you ask a vendor for a quote. Missing inputs are the most common reason early estimates come in low.

Which formulas and calculators should you use?

You do not need to do this math by hand, but you should know what the calculator is doing. Here is the logic, step by step.

1. Start with the proportion formula. n = (Z² × p(1 − p)) / d². Z is the z-score for your confidence level, p is the expected proportion, and d is the margin of error as a decimal. This is the standard prevalence formula used across survey and health research.
2. Run it for the baseline. With Z = 1.96, p = 0.5, and d = 0.05, you get 1.96² × 0.25 / 0.0025 = 384.16, which rounds up to 385. Most people quote 384.
3. Apply the finite-population correction if your population is small. The adjusted sample is n / (1 + (n − 1) / N), where N is the population size. A small population can reduce the required sample.
4. Use the mean formula for numeric outcomes. When you measure something like dollars spent or minutes traveled, use n = (Z × σ / d)². Here σ is the standard deviation, and it matters a lot. A noisier measure needs more responses.
5. Multiply by your design effect. A design effect above 1 increases the required sample to account for added variance.
6. Divide by your expected response rate. That gives the number of people to invite.

Then treat the result as a guide. As the CDC guidance cautions, cost, nonresponse, and feasibility all affect what you can actually field.

How do worked examples look for three common survey projects?

The table below runs three projects through the same steps. All inputs are illustrative, not market data.

Nonprofit membership survey. For a small membership, the finite-population correction can reduce the required sample. Estimate invitations from your expected response rate, then consider whether surveying the whole membership is practical.

Community satisfaction survey. For a larger town, the finite-population correction may have little effect. Clustering by neighborhood or weighting to match census figures can increase the required sample. Estimate invitations using a realistic response rate, and account for the cost and feasibility of fieldwork.

Regional survey with subgroups. Calculate the required completes for each region using the precision you need, then add the group totals. Estimate invitations separately for each region using its expected response rate. Planning only for the overall total can leave individual regions with too few responses.

Pro Tip: Build your invitation list for the worst response rate you can imagine, not the best. Running short mid-field is far more expensive than inviting extra people up front.

When should you increase sample size for design effect, clustering, and weighting?

A design effect tells you how much extra variance your sampling plan adds compared to a simple random sample. If you sample whole neighborhoods, precincts, or schools, people in the same cluster tend to answer alike. That similarity means each added response from the same cluster tells you less than a fully independent one. Weighting adds variance too, because some respondents count for more than others.

Pew Research shows that weighting and survey design can substantially affect the margin of error, and that the effect should be built into planning rather than discovered afterward.

Weighting and design choices can raise a survey's margin of error well above the simple textbook figure. That is why a sample sized for a perfect random draw often falls short in the field. (Pew Research Center)

Practical ways to set your DEFF:

• Use a prior survey from the same community if you have one, and borrow its design effect.
• When no local data exist, practitioners often assume a DEFF of about 2.0 for cluster surveys, since ignoring clustering can leave a study underpowered.
• For civic surveys that use light weighting to match age and gender, a multiplier between 1.5 and 2.0 is a cautious range.
• Run a small pilot and measure how much your estimates vary before you lock the final target.
• Use 1.0 only for a true simple random sample with no weighting.

Being conservative here costs you some extra responses. Being optimistic can cost you a survey you cannot defend.

How do you adjust for response rate and nonresponse bias?

The number your formula gives is the number of completed surveys. To get the number of invitations, divide the number of completes you need by your expected response rate.

Be careful about which rate you use. AAPOR has warned that response rates have declined and are often misused for nonprobability samples, so opt-in panels and open web links should report participation rates and follow AAPOR reporting guidance. Be honest about how your sample was built.

Ways to protect your response rate:

• Send reminders at spaced intervals.
• Offer more than one mode, such as online plus a paper or phone option, so you reach residents without reliable internet.
• Keep the survey short and say how long it takes.
• Use a modest incentive when your budget allows.
• Have a trusted local name sign the invitation, such as the mayor, chamber director, or school principal.

A high count of responses does not fix bias. If the people who answer differ from the people who do not, your results are skewed no matter how large n is. When response is low or lopsided, compare your respondents to known population figures and report the gaps.

Pro Tip: Track response by neighborhood or member group while the survey is still open. If one area is lagging, you can send extra reminders there before the field period ends.

How do you size a survey for subgroups and group comparisons?

Precision drops fast as groups get smaller. A sample that gives a precise overall estimate may give a much less precise estimate for a smaller subgroup. If a result for a specific group matters, size for that group directly.

1. List the subgroups you must report. Age bands, districts, business types, and customer tiers all count.
2. Pick the margin of error you can accept for each one. Smaller groups usually get a looser target.
3. Calculate n per group. Run the proportion formula for each subgroup, then add them up.
4. Plan for comparisons separately. To detect a difference between groups, you need enough people in each. Design guidance explains how power and the expected difference inform sample-size calculations. Equal-sized groups can be a practical planning assumption.
5. Decide how to handle small groups. You can oversample them, combine two survey waves, or accept lower precision and say so in your report.

Always compute the gross invitations per group, not just overall, because small groups often respond at different rates.

What do common rules of thumb get right and wrong?

Rules of thumb help in early planning. They also get misused.

• Small samples. A small sample may be useful for exploratory work, but it can leave a wide margin of error for a percentage.
• A baseline sample. The required sample depends on the confidence level, margin of error, expected proportion, and survey design.
• Larger samples. A larger sample can improve precision, but it also costs more and may exceed what a local project needs.
• Where they break down. Clustered designs need a DEFF multiplier. Extreme proportions, such as 95% approval, behave differently from 50%. And any subgroup you must report needs its own count.

Use a rule of thumb for a first budget conversation. Switch to a real calculation the moment you sign a scope of work, report by subgroup, or use any clustering or weighting.

What is the checklist for deciding your sample size?

Walk through these steps in order before you brief a vendor or set a budget.

1. Define the primary estimate you care about and the precision you need.
2. Choose a confidence level and set the expected proportion or standard deviation for a mean.
3. Decide on your design effect and realistic response rate.
4. Calculate the raw sample, apply the finite-population correction if appropriate, account for design effects, then divide by the response rate to estimate invitations.
5. Pilot the survey, check early response, and adjust the target and reminders.

Write each assumption down. When results come back, anyone can check how you got the number.

What I have learned about sample size in small communities

At Southwind Marketing we plan community surveys for rural towns, chambers, and economic development organizations, and the textbook rarely fits the situation. The population is small, the budget is tight, and a board wants one clear number. The fix is rarely a bigger sample. Usually it is a better frame: start from a clean contact list, consider the finite-population correction, and choose a fieldwork plan that fits the community. Mixed modes help reach residents who will never click an emailed link. If a group is hard to reach, combine an administrative list with a targeted oversample. A smaller, well-designed survey that you can defend beats a large one nobody trusts.

How can Southwind Marketing help you plan and run your survey?

Southwind Marketing serves Rural America nationwide, working with businesses, economic development organizations, local governments, chambers, tourism organizations, Main Street programs, nonprofits, and community organizations. We run community and stakeholder surveys from start to finish: we set your sample size, manage fieldwork, and turn results into dashboards your board can read.

Our Southwind Connect℠ supports CRM and marketing automation. Southwind Connect can support supplemental crisis communications, but it does not replace official emergency alert systems. We pair it with Southwind Signal℠ and Southwind Guardian℠ where outreach and site support matter, and with data dashboards and Civic Intelligence℠ so results turn into decisions. Tell us your population, your precision goal, and your timeline, and we will send back a sample-size plan and a quote. Request a consultation and we will start with your numbers.

FAQ

Is 20 respondents enough for a survey?

Twenty respondents is rarely enough for a survey that guides decisions, because the margin of error is very wide. It can work for a small pilot or a few interviews. For percentages you plan to report, aim for the number your sample-size calculation gives.

How do I calculate the sample size for a survey?

Use n = Z² × p(1 − p) / d², where Z is the z-score for your confidence level, p is the expected proportion, and d is your margin of error. Adjust for a small population, multiply by your design effect, then divide by your response rate. This standard formula is the same one most calculators use.

Why is 30 the minimum sample size?

Thirty is a rule of thumb from statistics for estimating a mean when the data are reasonably well behaved. It is a floor, not a target. For survey percentages with useful precision, you usually need several hundred responses.

Sources

• Variability of survey estimates | Pew Research Center
• Aapor
• Design and sample size decisions
• Population Survey or Descriptive Study | StatCalc | User Guide | Support | Epi Info™ | CDC
• Sample size calculation in clinical research