from VCCA Journal, Volume 5, Number 1, Summer 1990, 6-9
© Copyright 1990 VCCA Journal
[At the 1989 annual VCCA convention held in Roanoke, I predicted that the college outcomes assessment movement is likely to become even more pervasive in the 1990s. I warned that federal "track record disclosure" requirements may foreshadow a chilling new era of increasing accountability for higher education. The following is an elaboration of that warning. D.H.]
As in other states, Virginia's institutions of higher education have been responding to recent mandated guidelines for systematic and comprehensive assessment of student outcomes. Attention has focused on two external sources of institutional action on outcomes assessment: a 1985 General Assembly resolution (and subsequent State Council of Higher Education guidelines) and "institutional effectiveness" criteria added to postsecondary accreditation standards by the Southern Association of Colleges and Schools (SACS).
Though fairly specific about what colleges must do to be in compliance, both State Council of Higher Education and SACS requirements recognize the importance of institutional diversity and autonomy in higher education. They also emphasize that assessment should be, above all, a self-directed process of evaluating and improving institutional performance. The assessment movement in Virginia has avoided the kind of lock-step tracking, testing, and numbers games that have plagued colleges in other states. Instead of a burdensome intrusion on faculty, a barrier to students, or a meaningless exercise in bureaucratic busywork, assessment in Virginia appears to have been creative, stimulating, and rewarding.
Within the last year, however, a new player, the U.S. Department of Education, has quietly entered the assessment arena with an unsettling game plan. National interest in upgrading the quality of higher education is nothing new, but until recently the federal role had been essentially limited to stirring up attention through various well-publicized studies such as A Nation At Risk, and to stimulating voluntary reforms through FIPSE grants. Then, in July 1988 the Secretary of Education announced in the Federal Register a new requirement that regional accrediting agencies must include a focus on outcomes assessment. This was followed a year later by "track record disclosure" regulations for postsecondary institutions offering undergraduate nonbaccalaureate programs. These regulations (34 CFR 668.44), however well-intentioned, could lead to disastrous consequences for community colleges. Proposed as part of the Secretary's assault on the problem of student loan defaults, the "consumer information" regulations took effect in December of 1989. Briefly summarized, the regulations require institutions to report to prospective students (and to the Department of Education) specific graduation rates, employment rates, and licensure examination pass rates for nonbaccalaureate vocational programs. The purpose of this "track record disclosure" information is to allow prospective students to make informed consumer choices about academic programs and thereby help reduce the default rate on federally guaranteed student loans.
Aside from the fundamental question of whether students' opportunities for academic success and their likelihood of subsequent employment and loan repayment are significantly enhanced by such disclosure, there are numerous technical problems with the data requirements specified in the regulations. In essence, institutions are required to calculate program completion rates as the percentage of students who enter a program and graduate on schedule and/or obtain employment in their career field. Projected completion dates are based on the length of the program and each student's credit load (pro-rated for part-time students). To obtain this information, institutions would need a sophisticated tracking system that follows each student from entry to exit and on into the work force. Obtaining the information is but one of many sources of concern. Another problem is figuring out how to provide and explain such potentially misleading data to prospective students. Colleges are required to use a standard form entitled "How Our Students Are Doing" for each academic program. Prospective students are supposed to read and sign the form, thereby certifying that they were not recruited under false pretenses.
Perhaps the most frightening aspect of the track record disclosure regulations is the Secretary's stated intent to disseminate this information, compiled and arranged by geographic area and vocation, to high schools and various other entities. Recipients of these lists would be able to see for themselves "how students are doing" in postsecondary vocational/technical education. What is so frightening about public disclosure of comparative information on student success rates? What are colleges afraid of? Won't informed consumer choice simply help to promote healthy competition, stimulate program improvements, and eliminate unproductive or unscrupulous operations? And won't a realistic picture of program performance help protect students' and taxpayers' investment in federal student loan programs?
First, colleges should not fear fair and accurate disclosure of student success rates; but colleges are wise to be concerned about the dissemination of gross and misleading data. Normal community college student enrollment patterns do not fit neatly into the type of tracking system prescribed by the federal regulations. Students commonly change from one program to another; they change their credit load and timetable for program completion because of job changes, family obligations, health and personal developments, and countless other unpredictable nonacademic reasons.
Second, a high rate of student success is not necessarily a measure of program quality. If colleges feel pressured to retain and graduate students at the expense of academic standards, or to prepare students for high employment in low-paying entry level jobs, the disclosure of comparative rates would be counter-productive. Conversely, more selective admission standards would help raise student success rates but at the cost of access, thus restricting access to vocational/technical education and denying equal opportunity to those most in need of improved employment skills.
Colleges are trying to comply as best they can in response to the seemingly impossible demands posed by these regulations. A poll of Virginia's 23 community colleges revealed that none were able to provide all the data required, even after extensive work. Most reported that they felt compelled to use the "How Our Students Are Doing" forms even though they did not have the correct data or any confidence in the forms. One college official explained, "most students were impatient with the explanation and just wanted to know where to sign." Another college representative commented, "It was a nightmare to try to put [the prescribed data] together, and even after all the effort, the data was useless." The following expression of frustration reveals that poor communication by federal officials contributed to the problems:
At first we thought the requirements applied only to schools with a high default rate, and ours was less than five percent. Then we learned it applied to all schools, but we assumed it was just for students in Title IV programs. By the time we understood all that was required, we barely had time to meet the deadline and we couldn't provide much of the required data.
Indeed, it is astonishing that something of such magnitude was mandated with so little communication. All direction from the Department of Education regarding the sweeping consumer disclosure requirements of the default reduction regulations has been through the Federal Register and some last-minute "dear colleague" letters sent to college financial aid officers. This has caused the requirements to be widely regarded as a financial aid matter rather than an assessment issue. Consequently, the technical problems obvious to experienced college assessment personnel have not been resolved.
Worse than poor communication is the apparent lack of any consideration or understanding of how community colleges operate. The regulations are believed to have been directed primarily at proprietary schools which attract students with promises of guaranteed job placement. Public colleges rarely recruit students with specific claims of student success (other than occasional alumni testimonials in brochures). Such tactics are considered unseemly. Studies consistently indicate that students typically enroll in community colleges because of location, open admission, flexibility and low tuition. Those community college programs such as nursing and other allied health fields which do attract mostly traditional, full-time students because of their reputation for high licensure exam pass rates and employment success are atypical. Since most other occupational/technical programs at community colleges largely consist of part-time students who are already employed, the assumptions underlying the consumer disclosure requirements are simply not valid.
The worst-case scenario frequently discussed at assessment conferences is the specter of externally developed and rigidly structured reporting requirements, imposed on colleges by public officials who do not understand the complexities of colleges, who are not sensitive to differences among institutions and academic programs, and who reduce accountability to simple numbers which distort the true nature and purpose of higher education. To the extent that this characterization fits the federal track record disclosure regulations, it is a problem which deserves broader and more serious attention.
David C. Hanson is Director of Instructional Support Services at Virginia Western Community College in Roanoke and a consultant on student outcomes assessment.