AP Statistics Review- Sampling Methods and Bias

What You Need to Know About Sampling in AP Statistics

If you're prepping for the AP Statistics exam, sampling methods and bias are guaranteed to show up. This isn't optional material. Questions about flawed sampling show up in both the multiple choice and free response sections, and missing them costs students points they shouldn't lose.

Here's what actually matters.

The Core Problem: Why Sampling Exists

You can't survey 300 million people every time you want to know something about the US population. So statisticians look at a small group and use that to make claims about the whole population. The question is whether that small group actually represents the whole.

That's the entire game. Get the sample right, and your conclusions hold up. Get it wrong, and you're just guessing with extra steps.

Sampling Methods You Must Know

Simple Random Sample (SRS)

Every member of the population has an equal chance of being selected. No favoritism, no patterns. You assign numbers to everyone, then use a random number generator to pick your sample.

This is the gold standard. It's the baseline all other methods get compared to.

Stratified Sampling

You divide the population into subgroups (strata) that share characteristics, then randomly sample from each stratum proportionally.

Example: Survey 1000 students by taking 250 freshmen, 250 sophomores, 250 juniors, and 250 seniors. Each grade is a stratum.

This works when you want to ensure each subgroup is adequately represented and when there's real variation between groups.

Cluster Sampling

You divide the population into clusters, randomly select some clusters, and then sample everyone within those chosen clusters.

Example: Randomly pick 5 schools from a district, then survey every student in those 5 schools.

Cluster sampling saves time and money. The downside is that if clusters are similar to each other, you might miss important variation.

Systematic Sampling

You pick every nth member of the population after a random starting point.

Example: Survey every 10th person who walks into a store, starting from person #3.

This can work well if there's no hidden pattern in how people are ordered. But if there's a cycle in the ordering, you introduce bias fast.

Convenience Sampling

You grab whoever is easiest to reach. Online polls, street corner surveys, asking your friends—these are all convenience samples.

These are almost always biased. The people you can easily reach often differ systematically from people you can't reach. AP Stats will ask you to identify why these are problematic. The answer is usually that they don't represent the full population.

The Bias Problem: Where Students Lose Points

Bias isn't complicated. It's systematic error that skews results in a particular direction. The math doesn't save you here—more data from a biased sample doesn't fix the problem. It just gives you more precise wrong answers.

Selection Bias

Some members of the population are more likely to be selected than others, and it's not random.

Example: Surveying only people at a gym about exercise habits. People who don't work out won't show up in your sample.

Nonresponse Bias

People selected for the sample don't respond, and those people differ systematically from those who do.

Example: Mailed surveys. People with strong opinions (positive or negative) respond. Everyone else throws the survey away. Your results are skewed toward the extreme views.

Response Bias

Respondents give inaccurate answers, usually because of how the question was asked or the setting.

Example: "Do you support funding public schools?" gets different answers than "Do you support increasing property taxes to fund schools?" Same idea, different wording, different results.

Voluntary Response Bias

People choose themselves into the sample by responding to a call for input.

Call-in polls, online surveys where people click to participate—these attract people with strong feelings. The quiet majority never calls in. Your sample overrepresents extreme opinions.

How to Spot Bias on the Exam

Read the scenario and ask: "Who is being left out, and would they answer differently?"

• Phone surveys miss people without phones. That's selection bias.
• Online polls miss people without internet access. That's selection bias.
• Surveys at one location miss people who don't go there. That's selection bias.
• Low response rates mean nonresponse bias is likely.
• Loaded question wording means response bias is likely.

The AP exam loves scenarios where students have to identify which sampling method was used AND what bias it introduces. Know both parts.

Sampling Methods at a Glance

Getting Started: How to Approach These Questions

Step 1: Identify the sampling method described. Look for whether selection was random, if groups were defined first, or if it was just whoever was available.

Step 2: Ask who is being left out. Every flawed method leaves someone out. Name them.

Step 3: Ask if those people would answer differently. If yes, bias exists.

Step 4: Name the specific bias. Selection, nonresponse, response, or voluntary response. Know the difference.

Step 5: For free response questions, explain why the method fails to generalizable and suggest an improvement. "Use simple random sampling" is usually the fix.

What Doesn't Fix Bias

Increasing sample size doesn't fix bias. A large biased sample is still biased. Random selection fixes bias. That's the only tool that actually works.

Also, convenience samples sometimes get defended with "but it's a large sample." Ignore that argument. Size doesn't compensate for systematic error. The math doesn't work that way.

The Bottom Line

Random selection is what separates statistics from guessing. Know your sampling methods. Know your biases. Be able to spot both in a paragraph of text. That's what the exam expects.

If you can identify the method, spot who's missing, name the bias, and suggest a fix, you've got the skill down. Practice with old free response questions. The patterns repeat.