Should we trust the surveys saying students strongly prefer on-campus to on-line recruiting?

Earlier today, I participated in a discussion in a listserv owned and moderated by the National Association of Colleges and Employers (NACE). The discussion was started by a consultant sharing a summary of results of a recent survey that she conducted. The results of that survey seemed to indicate that students strongly preferred to engage with prospective employers through traditional, on-campus recruiting instead of other, virtual methods such as video, email, chat, or text.

It was impossible to see from the summary what methodology was used to collect the data or even who was sampled. I suspected that the methodology was the same as previous surveys that I’ve seen shared within the group. If so, it was probably convenience instead of random sampling. Convenience sampling has become quite popular in the era of email, online polls, SurveyMonkey, etc. It can take many forms but often a group of people receive an email with a link and request to complete a short survey. Some members of that group comply. Both the group receiving the request and the subset completing the survey are not randomly selected.

According to statisticians, the results of convenience samples cannot be generalized to a target population because of the inherent bias of the technique. The bias occurs as a result of the inevitable underrepresentation of subgroups in the sample. Unfortunately, the bias of the sample cannot be measured. 

For example, let’s say that a survey taker partners with a few dozen college career service offices to distribute the survey to students who have registered with the career service office. Right away, you run into tremendous bias. Instead of sampling all students at the schools, you’re only surveying those who are biased toward working with their career service office. To be clear, students should work with career services, but the reality is that many don’t and, at some campuses, most don’t. Some schools require students to register with career services in order to receive a transcript and so the numbers at those schools appear to be quite high, but when you scratch the surface many of those students are hardly engaged with those offices. 

To draw any reliable conclusions about what a broad crosssection of students at a broad crosssection of schools want, you need to randomly survey those students in large enough numbers that the sample size will be statistically valid. Otherwise, what you’re doing is collecting data that is inherently unreliable. Some might say that some data is better than no data. I beg to differ. You’re actually better off making decisions without the benefit of data than with the benefit, if you can call it that, of unreliable data because you’re more likely to understand and account for the limitations of the data you’re using.

Now, all that said, do many and perhaps most students prefer in-person interactions with employers? Absolutely. Is the same true in reverse? Absolutely. Does that mean that employers should or even will revert back to their mostly in-person process starting this fall? Absolutely not, at least that’s my prediction. Covid forced the pendulum to swing from an almost entirely in-person, on-campus process that hadn’t changed much since 1952 to an almost entirely virtual process. But that pendulum swing also allowed (forced?) many employers to acknowledge that they could accomplish far more with far less. They could engage with a far more diverse group of students far more quickly, far more equitably, and far more efficiently through a more virtual model.

Do I think that employers will abandon the in-person, on-campus model? No. But do I think that the more and probably most will shift resources so that a significant amount of their efforts are virtual? Absolutely.