University of North Carolina Wilmington (UNCW) officials in recent years created a minority mentoring program to connect students of color with business leaders in the community. The university recruited minority staff and students to work in its career center. It hired an adviser to work with minority student groups to ensure campus events reflected their interests.
All of these changes came about due to data—survey data that, for instance, revealed minority students were not taking advantage of the career services center.
“There were certain sub-populations on campus that felt marginalized,” says Dr. Nathan Lindsay, director of student life assessment at the University of North Carolina Wilmington that has an 11 percent minority undergraduate enrollment. “Students who are Native America and Asian consistently across several different surveys produced lower scores in terms of their satisfaction level with services, in terms of feelings of being included or represented. It gave us greater cause to address those things.”
UNCW is one of 1,400 institutions that have participated in the National Survey of Student Engagement (NSSE), the survey instrument that signaled the need for changes in the career services center to better meet minority students’ needs. Administered to first-year and senior students at universities that pay for it, the survey gauges to what extent students are participating in activities known to be effective best educational practices, such as a high level of interaction with faculty.
Founded 11 years ago, the NSSE program based at Indiana University has grown in size and influence. Its annual results are widely reported in the media. Researchers have used the data for a variety of studies. Also, it is one of the tools institutions can use as part of the Voluntary System of Accountability, an initiative among public four-year institutions designed to demonstrate transparency and good stewardship founded by the Association of Land-grant and Public Universities and the American Association of State Colleges and Universities.
“We have two overarching goals,” says NSSE director Alex McCormick. “One of them is to enrich the national conversation about college quality, which has been unreasonably obsessed with reputation and student test scores and pretty much bereft of any discussion about teaching and learning. Unlike other approaches that focus on outcomes, we’re focused on process. We know about what works. We go straight to the students and ask them about the extent to which they are involved in those practices.”
“Our other big goal is to use that information and feed it back to institutions in the form of diagnostic information on how they can reform,” McCormick adds. “We’re trying to encourage institutions to try to understand whom their least engaged students are and what we can do to improve their experience.”
UNCW’s career services changes provide one of many examples of schools making data-based campus improvements. NSSE has also been involved with the Building Engagement and Attainment for Minority Students, an initiative to help minority-serving institutions use NSSE data and campus assessments to improvements students’ experiences. Jackson State University, a historically Black institution in Mississippi, for example, saw a 46 percent increase in students using tutorial services after it realigned student support services in response to data indicating an underutilization of services by students who really needed them.
There are few critics of NSSE, which is regarded by many researchers who study minority student experiences as a useful tool, along with data from the federal government and from UCLA’s Higher Education Research Institute, which administers the Cooperative Institutional Research Program’s annual survey of freshmen.
“The questions that NSSE asks about the nature of students experiences are very useful for understanding how minority and other students are engaged in college. The questions are well grounded in relevant theory and offer a comprehensive assessment of student engagement,” says Dr. Laura Perna, a professor in the Graduate School of Education at Pennsylvania State University whose research focuses on understanding how to improve college access and success for students from historically underrepresented groups. Although she has not used NSSE data in her research, she notes the numerous studies on minority student success that have relied on it.
Those who have concerns about NSSE note its limitations, which include the narrow pool of students surveyed. Surveying seniors, or the “survivors,” means they’ve missed students who have dropped out, students the institutions really need to hear from, wrote Mark Schneider, vice president at the American Institutes for Research, in an op-ed that questioned the use of NSSE data in holding institutions accountable for student success.
Still, NSSE is cited frequently as a useful tool for informing minority student success. In addition to looking for differences in the responses of subgroups and the broader student population to questions about various student services, the NSSE survey has questions dealing directly with diversity issues. These include, “In your experience at your institution during the current school year, about how often have you had serious conversations with students or a different race or ethnicity than your own?”
Minority Student Experiences
NSSE findings have been used to compare the experiences of African-American and Hispanic students at minority-serving institutions and their predominantly White counterparts as well as to gauge students’ perceived benefits to interacting across different cultures. The findings provide a starting point, however, on issues that require further study.
For instance, using data from the 2003 survey of 147,166 undergraduates, researchers have found that African-American seniors at historically Black institutions were more likely to report participation in active and collaborative learning environments and greater student-faculty interaction than Black students at PWIs. However, Hispanic students report no greater advantage attending a Hispanic-serving institution than a PWI. The scores for Hispanic seniors at HSIs on those measures were similar to Hispanic seniors at PWIs.
“We found that African-American students at HBCUs were more engaged and gained more from their experiences compared to African-Americans at predominantly White institutions. These are cultures that have been built to encourage good academics among African-American students. It’s generally a faculty and an administration that believe in the education of those students, and they’re working hard to make it the best it can be,” says Thomas Nelson Laird, co-author of the study and an assistant professor of higher education at Indiana University.
HSIs, however, don’t have the long history and mission of HBCUs, which may account for the differences in Hispanic student responses, says Nelson Laird. “They weren’t generally founded to be Hispanic serving; they’ve become Hispanic-serving over time. They don’t have a faculty or administration that has fully taken on that mission.”
In another study, Nelson Laird and colleagues found that students generally viewed their educational environment more positively the more they interacted across differences, including racial, political and socioeconomic differences.
“The strength of that relationship depends on your race and ethnicity,” Nelson Laird says. “African-American students see the environment more positively the more they interact across differences. It’s positive for all students but more positive for some and less positive for others.”
“Should it be different for these different groups? Institutions need to wrestle with that. The really important message that’s embedded here is that a lot of institutions have bought into the idea that diversity is important for their campus,” Nelson Laird says. “This message is we can’t do that blindly. We can’t go in and pick these things and believe that it’s going to be a universal benefit for everybody in the exact same way. It’s going to have a different effect on different students.”
The Accountability Question
Dr. Mitchell Chang, a professor of higher educational and organizational change at the University of California, Los Angeles, says he holds NSSE in high regard but notes there is some redundancy in what NSSE and the much older CIRP assesses. In many ways, they’re competing with each other for institutional participation as student survey fatigue rises.
Still, he says that NSEE is valuable to institutions in responding to minority students’ needs and that surveys like NSSE and CIRP are a better barometer of institutional performance than traditional measures.
Using NSSE as an accountability measure “is a very good move because without those surveys you’re looking at institutional level data, such as retention rates and loan default and transfer rates, but you don’t get a good sense of how students have benefited from being on those campuses over time, the added value of higher education,” Chang says.
At UNCW, Lindsay says it’s important for institutions to use NSSE in combination with other assessments. Faculty may debunk NSSE results on a particular measure as invalid or question how the data can be useful, Lindsay says, but he always makes sure the results are corroborated by another source. “We’re not just seeing it in NSSE; we’re seeing it in other instruments. It’s a lot more powerful,” Lindsay says.
McCormick says it’s fair for institutions to use NSSE data to hold themselves accountable, but there are limits.
“We’re the first to say it’s no silver bullet. Just signing up for NSSE in and of itself is not going to do anything. The rubber hits the road when the institutions grapple with the results and start to think about what kinds of things we provide to our students,” McCormick says. “It would be unfortunate if NSSE results alone are used to reward or punish institutions. I do worry about NSSE becoming a high-stakes test.”
