Survey Investigator Speaks Out
Posted by: Craig Woolheater on October 30th, 2006
Christopher D. Bader, an Assistant Professor of Sociology at Baylor University, was one of the principal investigators of the study that is being discussed here on Cryptomundo.
After reading the post on Cryptomundo and the comments from readers, Chris had this to say:
There seems to be an assumption in the comments that the survey was either
a.) somehow (it is never exactly specified) biased since it is from Baylor and
b.) probably administered to a bunch of other Southern Baptists.
The survey was administered to a random, national sample of U.S. citizens. In other words, any one in the U.S. had an equal chance of being selected. Our respondents are from all regions of the country, all races, all levels of education, all religions (or not religious) and so on. Gallup, in fact, administered the survey.
As for bias – some of the commenters are correct- you can mold or shape survey questions if you are not careful. Polls conducted by lobbyists, etc are notorious for this. They will only administer a poll to certain types of people, limit potential responses, find clever reasons to remove cases and so on. We spent a lot of time (and consulted many survey experts) to attempt to remove obvious sources of bias. And will continue to do so in the future. As social scientists we want to find out what Americans believe – not what we hope or think they believe. The argument that because polls can be twisted all polls are useless is specious.
Reminds me of the argument that since it is possible to hoax Bigfoot sightings, all Bigfoot sightings must be hoaxes.
No one on the Baylor Religion Survey team is Southern Baptist by the way. 
We are used to getting heat about the survey – getting criticized by others about our work is what academics do for a living. Not offended in any way and find thoughtful debate fun. So feel free to post or not post my response if you think it will be interesting to your readers.
Love the site!
Christopher D. Bader
Assistant Professor
Department of Sociology
One Bear Place #97326
Baylor University
Waco, TX 76798-7326
Co-founder of Cryptomundo in 2005.
I have appeared in or contributed to the following TV programs, documentaries and films:
OLN's Mysterious Encounters: "Caddo Critter", Southern Fried Bigfoot, Travel Channel's Weird Travels: "Bigfoot", History Channel's MonsterQuest: "Swamp Stalker", The Wild Man of the Navidad, Destination America's Monsters and Mysteries in America: Texas Terror - Lake Worth Monster, Animal Planet's Finding Bigfoot: Return to Boggy Creek and Beast of the Bayou.
Well, irregardless of what I thought about the survey, at least Christopher was a standup guy and responded to his critics. So many people (ie, politicians on both sides of the aisle) never respond to their critics, only their supporters. Thanks Christopher.
I’m not overly critical of surveys so much as I am on what people ‘believe’…beliefs are very overrated as I eluded on the other post.
For me I would say that I have ‘an interest’ in the paranormal and most things strange and mysterious. Belief has no room here. My interest is an interest to know (more).
Remember the poster in Fox Mulders basement office from the X-files…’I want to believe’….that would be more MY stance. Though I would much rather KNOW. Does that make sense? So many just put the believing before the knowing and that’s where we put our heads in the neuce…
Things that exist exist whether we believe in them or not.
Believing in something doesn’t necessarily make it so…
I have to say I spoke before I completely digested what I read.
My comments seem to have been completely unfounded in regards to this survey.
While in the situation I mentioned they may have bearing they were completely useless as I made a couple assumptions about the survey. You know what thy say about assumptions…
Surveys that attempt to quantify belief in things like Bigfoot, UFO’s, and Ghosts come down to one thing, personal experiences. No sane person would blindly embrace such a radical position without the courage of their own convictions.
It requires more than a passing interest in the paranormal to go to school as a 12 year old and admit to seeing a ghost. It takes a lead pipe belief in what your own eyes and ears experienced in order to go against the ‘conventional wisdom’ of the masses. I was sent to the school psychologist because I dared challenge the teacher’s assertion that ghosts were nothing more than the products of superstition. No matter what proof I could offer, the details of my story, or the presence of a witness were not enough to open the minds of my ‘superiors’. I learned a few lessons then that stayed with me through the years.
1) The ‘improbable’ is light years away from the ‘impossible’.
2) Someone who claims to have seen something extraordinary isn’t always wrong.
3) Seeing is believing.
I am actually surprised that 16% of the people expect to one day find bigfoot. I think that’s higher than I would have guessed. With over one third of the people accepting ghosts and hauntings as believable tells me that there is more going on in the paranormal than anyone would have ever imagined.
This study seems like grounds for celebration to me. What I glean from the numbers is that people are willing to think for themselves and entertain the possibilities of an alternate set of ‘facts’. At the heart of the numbers it appears that the skeptics are doing a poor job of getting people to disavow their own experiences.
Thanks for writing to us Mr. Bader. We appreciate the extra details you gave.
Good response, now on to Big Foot in Texas. It’s a big state, it has to be loaded with primates with big feet.
No doubt about it, we are still the Planet of the Apes
I fail to see any validity in tracking the political affiliations, and/or voting practices, of the survey’s participants.
Any attempts to justify this with the old, “we’re trying to group our participants”, or “we are gathering data about specific groups” merely points out how un-scientific the results are. The study should be totally blind, the questions should be designed to be as non-ambiguous as possible, and conducted without a pollster in order to provide any usable data. And still, whomever compiles said data, may impart a bias to said results intentionally or not.
Hence, the very not scientific nature of the poll/survey.
Where in your numbers do you account for the people who merely gave your pollsters a dirty look and walked away without a word? How did you classify those according to political affiliation? I can go on with a list of serious or fatal (to the survey) flaws.
Christopher, I thought your survey was interesting. Personally I didn’t see anything controversial. I figure any exposure on the bigfoot topic is good, so for that, I thank you.
Craig invited me to join in on the discussion. Thanks for the comments. I appreciate your feedback and will be pushing for more questions on cryptozoology on the next wave of the Baylor Religion Survey.
I even appreciate your feedback Mr Inspector – I’m sure you could go “on and on” with your list of serious flaws. It is easy to do so when you have no idea what you are talking about. You are making enormous assumptions about the survey that are simply not correct. The survey was not conducted in the presence of a pollster. It was mailed to respondents to complete. We know the numbers for people who returned and did not return the questionnaire and did not have higher non-response from a particular political group, etc. And I’m sorry if a pollster has hurt you in some way.
If any of you have any questions or comments feel free to write me at any time or post here. I’ll respond in a few days when I return from a conference. It will be at least entertaining to see how much I have annoyed “Mr Inspector.”
Thanks again for your interest and keep up the good work!
Mr.Inspector, you’re making some odd presumptions about this survey. I believe you said in another thread that you “studied” statistics at some point in school. Well, I work in market research, and deal with questionnaire design and creation, survey results, and analysis of qualitative & quantitative data on a daily basis.
I fail to see what your point is re: asking someone’s political affiliation. Asking questions that determine an audience’s demographic makeup is an absolutely accepted and standard practice in the polling and research world.
Also, regarding your question about people that just “walked away,” I would be willing to bet that the poll was done via the phone, with perhaps an online component, as well. If people don’t complete a survey because they “walk away,” hang up the phone, etc., they’re not included in the sample. Why WOULD they be?
Ummm, one of the very first things they tell you when taking a market research course is that you ask about age/gender/race/political party/religion/etc — specifically so you can be sure you haven’t inadvertently picked a statistically skewed sample. Can you imagine how badly the results would be skewed if you happened to be doing a walk-by survey (yes, this was a mail survey, but work with me) on a city street where one hotel in the city was hosting a science-fiction convention and another was hosting a meeting of the Flat Earth Society?
And non-responses are only mentioned as an aside to the research report (e.g., 1000 surveys were mailed, of which 400 were returned), because non-responses are exactly that, non-responses.
Sunny and jjames – you two really know your stuff. If you ever feel so inclined, take a look at the initial report from the Baylor Religion Survey.
This talks about our methodology, shows the full questionnaire, etc. We are preparing Wave 2 – so any comments, advice or critiques you might have would be most useful.
What I find very unfortunate is how much politics and social views affect science, rather than the truth regardless of the implications. A photo taken by Dr X with PHD’s etc out the ying yang is it seems seldom ever scrutinized to the degree of a photo by John Doe, why? A lot of discoveries, inventions etc are done by John Does rather than Dr Xs, but its the Dr Xs who write the papers, get the credit etc, which really aren’t fair. History is full of the Scientists saying things that became unproven ie mustard gas could never be used, would kill everyone, flying wasn’t possible, and travel at speeds of over 30mph wasn’t possible as the human body couldn’t withstand the pressure, to name a few. It seems to me further that the more controversial the subject ie for example is, no well founded credited Dr would ever publish a photo he personally too for fear of reputation damage, not to mention that the Government has views on what society should and shouldn’t know. The end result here I think is that it takes a dead one dragged in to say here ya go explain this irrefutable evidence.The problem with this types of surveys is not only the above but sorting out eye witness’s, people who know and believe these people, along with I want to believe types to the average person, the survey is indeed biased by sampling this site right off, as it would be sampling a scientific site.
Chris, while I took the class (and did well at it, including a nightmarish run-in with SPSS) I can’t claim to have much more experience than that — jjames could tell you much more, and give far more creative input than I could.
I did try to take a look at the report, as I’m always interested in things like this — but the pdf file is broken at this point.
Drop a message on the board when it’s working again — I’d like to read the report in depth (as it’s always, *interesting* to see how the popular press interprets things like this…!)