Sampling Variation SOCL 3633 Research Methods · Lab 5 · Midwestern State University

Today you treat the 21,483 people in our GSS data as a whole population, draw random samples from it, and watch what happens to the answer. Every student will get different numbers from the random samples. That is what the lab is about, not a mistake. Keep this page open next to RStudio and work through the tabs in order. Before you leave, you submit two things in the Lab 5 submission on D2L: your response sheet and the picture file of your own chart.

Get set up

Two new tools today

slice_sample()
Picks rows at random. slice_sample(gss, n = 50) gives a random sample of 50 people. Each time you run it, you get a different 50.
replicate()
Repeats a command many times and keeps every result. replicate(500, ...) draws 500 samples in a few seconds.

One sample at a time: Steps 2 to 4

Step 2: the answer in the full data

Run mean(gss$relig == "None"). The $ sign picks one column out of the data, so gss$relig is the religion column. As in Lab 1, the mean of TRUE and FALSE answers is the share for whom it is TRUE.

Checkpoint. The Console shows [1] 0.1639901: 16.4 percent of the 21,483 people report no religion. Write it down. Today it is the true value you compare every sample to.

Step 3: one random sample of 50

  1. Put 50 in the first blank and run the line. Nothing prints. R stored 50 randomly chosen people as my_sample.
  2. Put my_sample in the next blank and run it. Write down the share.

Step 4: two more samples

Run the two Step 3 lines again, twice. Each time R draws a new random sample of 50 and gives a new share. Write down all three. Compare them with the person next to you. Your numbers will not match, and neither of you made an error.

If something goes wrong

What you seeWhat to do
"object 'my_sample' not found"The slice_sample() line did not run. Run it first, then the mean() line.
The share is 0 or 1 exactlyCheck the spelling and capital letter in "None". A misspelled category matches no one.
A + sign in the ConsoleA parenthesis is missing. Press Escape in the Console, fix the line, and run it again.
A chart block takes a long timeDrawing 500 samples takes a few seconds. Wait for the > prompt to come back before running anything else.

500 samples: Steps 5 and 6

Step 5: 500 samples of 50

Step 5 has no blanks. Select every line from shares_50 <- down to print(chart_50) and run them together. Each bar shows how many of the 500 samples gave a share in that range. The dashed line is the true value, 16.4 percent.

Histogram of the share reporting no religion in 500 random samples of 50 people. The samples spread from about 2 percent to about 32 percent, with most between about 10 and 22 percent, centered near the dashed line at 16.4 percent.
Your chart will look a little different from this one, because your 500 samples are different. The shape will be about the same: centered near the dashed line, with a wide spread.

Step 6: 500 samples of 1,000

  1. Put 1000 in the blank (no comma), then select every line from shares_1000 <- down to print(chart_1000) and run them.
  2. Compare the two charts. Both use the same horizontal axis, so you can see the difference directly.
Histogram of 500 random samples of 1,000 people. The shares fall in a narrow range from about 13 to 20 percent, tightly around the dashed line at 16.4 percent.
The samples of 1,000 still differ from each other, but far less. This spread from sample to sample is called sampling variation. It shrinks as samples get bigger, and it is the reason a poll reports a margin of error.

A list that leaves people out: Step 7

Suppose the only list you could sample from included people aged 40 and older, the way the Literary Digest's lists in 1936 left out many people. You draw 500 samples of 2,000 people from that list, twice the size of Step 6.

Before you run Step 7: where will these 500 samples of 2,000 center? The full-data share is 16.4 percent.
16.5%
  1. Put 40 in the blank and run the frame_40 line. The >= sign means "greater than or equal to."
  2. Select every line from shares_frame <- down to print(chart_frame) and run them.
Histogram of 500 samples of 2,000 people drawn only from people aged 40 and older. The shares fall in a narrow range from about 10 to 14 percent, all to the left of the dashed line at 16.4 percent. None of the samples reaches the true value.
The samples are large and agree closely with each other, and every one of them is wrong in the same direction. Younger adults are more likely to report no religion, and the list left them out. A bigger sample shrinks sampling variation. It cannot bring back people the list never included.

On your own: Step 8

  1. Run the count() line to see the religion groups.
  2. In the three places that say "Catholic", put a different group, typed exactly as it appears in the list, inside quotation marks. Pick a group with at least 500 people, or try a very small group and see what happens.
  3. Run the lines from my_group_true <- down to print(my_chart).
  4. Replace "Write your title here" with a complete sentence of about ten words that says what the chart shows. Run the chart lines again.
  5. Write one ggsave() line that saves my_chart with your last name, for example smith_lab5.png, and run it.
Here is one example, using Protestant. Yours will use your own group and title.
Example histogram titled Small samples give a wide range of Protestant shares. Across 500 samples of 50 people, the Protestant share ranges from about 30 to about 74 percent, centered near the dashed line at about 50 percent.
Before you leave: save the script in your SOCL3633 folder (File, then Save).

Lab 5 response sheet

Answer in complete sentences. Then assemble your answers and paste them into the Lab 5 submission in D2L. In the same submission, upload the picture file of your own chart. Your answers stay on this page and are not sent anywhere until you paste them.

Your name
1. Report your three sample shares from Steps 3 and 4 and the full-data share. Why do they differ when no one made a mistake?
The four numbers and two to three sentences.
2. Compare the charts for samples of 50 and samples of 1,000. What changed, and what stayed the same?
Two to three sentences.
3. The samples drawn only from people 40 and older stayed wrong even with 2,000 people. Why doesn't a bigger sample fix this? Connect your answer to the 1936 Literary Digest poll.
Three to four sentences.
4. Random sampling variation shrinks as samples grow. Name one kind of error that does not shrink however large the sample is, and give an example from this course.
Two to three sentences.

Sources

NORC at the University of Chicago. General Social Survey, 2000 to 2014. Accessed through the gss_cat extract in the forcats package for R. https://gss.norc.org
Lusinchi, Dominic. 2012. "President Landon and the 1936 Literary Digest Poll: Were Automobile and Telephone Owners to Blame?" Social Science History 36(1):23-50.
Wickham, Hadley, Romain François, Lionel Henry, Kirill Müller, and Davis Vaughan. 2023. dplyr: A Grammar of Data Manipulation. R package. https://dplyr.tidyverse.org
Wickham, Hadley. 2016. ggplot2: Elegant Graphics for Data Analysis. New York: Springer.

The charts on this page come from one run of the lab script. Because samples are random, your charts will differ a little. The full-data shares are unweighted shares of the gss_cat extract.