Tag Archives: Collegiate Learning Assessment

Our Colleges Are Getting Worse—Three Proposals to Help Save Them

American colleges have been celebrated as the best in the world. But in fact, they have been getting worse – and something must be done about it.

The greatest value of a college education is in enhancing a student’s command of critical thinking and analytical reasoning. The educated person can think. Among other things, she can evaluate arguments from politicians, pundits, salespeople, business associates and others. She may not know whether a particular claim is correct, but she has a meaningful capacity to gauge whether a claim makes sense and to figure out how to investigate it.

For decades, researchers have been measuring how much college enhances these skills. A standard yardstick has been Collegiate Learning Assessment, or as it is known in its current iteration, the CLA+. That test is designed to measure critical thinking, analytical reasoning, problem-solving, and writing. For decades, it has been given to undergraduate students at the beginning of their freshman year and at the end of their sophomore year to see how much their analytic skills have improved.

‘Barely Noticeable Impact’

The CLA+ is an open-ended test; students are given performance and writing tasks, not multiple-choice questions. In one section of the test, students are asked to read a set of documents and write a memorandum presenting their conclusions about questions relating to the material. For example, students may be told they are working for a city mayor who plans to combat rising crime by increasing the number of city police officers. The mayor is running for reelection against a candidate who advocates spending available resources on a drug education program for addicts instead of on more police. Students are asked to read a set of newspaper articles, statistics, and research briefs about crime and drug addiction and then write a memorandum for the mayor that evaluates the validity of both the opponent’s proposal the opponent’s criticisms of the mayor’s plan. The CLA+ does not test general knowledge. Its designers claim that the only way to successfully prepare students for the test is to teach how “to think critically, reason analytically, solve problems, and communicate clearly.”

In one study, researchers administered the CLA in 2005 and again in 2007 to 2,322 students at 24 colleges that varied in admissions selectivity and other factors. Students increased their CLA scores by the equivalent of only seven percentile points over their first two years of college. Thus, on average, students who entered college at the 50thpercentile on the CLA scored at the 57th percentile near the end of their sophomore year. That is considered anemic progress. As the researchers put it, the first two years of college today “have a barely noticeable impact on students’ skills in critical thinking, complex reasoning, and writing.” By contrast, students in the 1980s increased their performance by the equivalent of 34 percentile points – rising from the 50th to the 84thpercentile. Other data confirm those findings

Why Have Colleges Gotten Worse?

Researchers suspect that the principal reason is that colleges have become less rigorous. Colleges more readily tolerate students doing little work, and more readily give good grades for mediocre performance. In the 1960s, students studied, on average, more than 24 hours per week outside of class. Even that is rather low. Undergraduates should be studying at least two hours outside of class for each hour of class. That would mean that for the average load of fifteen hours of class time they should be spending at least 30 hours on homework. But students today study only twelve hours per week on average. More than a third of students report studying less than five hours per week.

Moreover, 20% of students say they frequently go to class unprepared. Yet the average GPA in colleges today is 3.2. According to one recent study, 43% of all grades awarded in American colleges today are A’s. Meanwhile, the percentage of C’s has declined from 35% decades ago to 15% today. Grades and GPAs have become almost entirely meaningless.

Researchers also believe that a factor with a great impact on critical thinking skills is whether students take classes from instructors with high expectations. They define “high expectations” as assigning more than 40 pages of reading per week or more than 20 pages of writing per semester. Half of the students who took the CLA in their sophomore year reported that in the preceding semester they did not take a single class that required more than 20 pages of writing, and a third did not take a course requiring more than 40 pages of reading per week. Researchers observed: “If students are not being asked by their professors to read and write on a regular basis in their coursework, it is hard to imagine how they will improve their capacity to master performance tasks – such as the CLA – that involve critical thinking, complex reasoning, and writing.”

After controlling for many differences, including the selectivity of the college attended, students who reported they had taken high-expectation courses scored 27 points higher on the CLA than did students who had not taken such courses.

Do Away with Letter Grades

What should be done? I have three proposals. First, colleges should designate high-expectation courses in their catalogs and on student transcripts so that graduate schools and employers can see how many of such courses applicants have taken. As someone who often sits on my law school’s admissions committee, I know we would find this information extremely useful. It would indicate both how intellectually ambitious an applicant is and how well his studies prepared him for law school and for a career in the law. Making this information publicly available in the college catalog will also encourage colleges to increase the number of their high-expectation offerings. No college – and few instructors or academic departments – would want to be known as a purveyor of low expectations. Colleges must, of course, adopt mechanisms for ensuring that the designations are honest, and those mechanisms should be reviewed by accrediting agencies.

Second, instead of receiving a letter grade in courses student should receive a number designating the quartile of the class in which the student ranked. Students whose performance placed them in the top quartile of the class would receive a 1, students ranking in the second quartile would receive a 2, and so on. This is would be far more meaningful than our current system. It would also end grade inflation; after all, 43% of students could not be awarded the top grade of 1. Instead of GPAs, colleges should employ AQRs – average quartile ranks. For example, a student who ranked in the top quartile in half of her classes and in the second quartile of the other half her classes would have an AQR of 1.5. Colleges might also consider giving grades earned in high expectation courses a boost for AQR purposes, just as some high schools give a mathematical boost to grades earned in AP classes for the purpose of calculating GPAs.

Third, so-called output assessments are all the rage in education today. What could be a better measure than an instrument such as the CLA+? Colleges should require at least a significant cohort of their students to take such a test both on upon entering and again perhaps two years later, and should be making that information publicly available.

Do I think colleges will leap at my proposals? No, I don’t. Colleges will be resistant to adopting AQRs because doing so will make many students and their families unhappy. Letter grades hide a multitude of sins; performance by quartile is brutally transparent. However, graduate schools could pressure undergraduate colleges to provide quartile ranks and AQRs, if not instead of, then at least in addition to, GPAs. And for their own sake in being able to effectively evaluate applicants, graduate schools have good reason to do so. If graduate schools preferred students from programs that provided this information, undergraduate schools would be pressured to comply.

Designating high-expectation courses should be a bit easier. Some instructors will not like it. There may be something of a tacit agreement between some instructors and their students: the instructor demands little and awards high grades; in return, students reward the instructor with high student evaluations. However, colleges and departments that pride themselves on their rigor will see a competitive advantage in designating high expectation courses. Once some begin doing that others will have a hard time not following suit.

Accrediting agencies could require colleges to use a test like the CLA+ to assess their performance. And if U.S. News began using that information in its ranking calculations, colleges would be scrambling to figure out how to perform better.

Carl T. Bogus is Distinguished Research Professor of Law at Roger Williams University School of Law in Rhode Island. He is the author of Buckley: William F. Buckley, Jr. and the Rise of American Conservativism and Why Lawsuits are Good For America: Disciplined Democracy, Big Business, and the Common Law.

More Bad News about College

What was the most noteworthy finding of the recent Gallup survey of people who have attended college? Half of the 90,000 respondents regretted one significant decision made as an undergrad, such as picking the wrong major. In journalistic terms, this is known as burying the lede — downplaying the major point of a story while elevating some minor point.

The major finding—stunning really–appears under the heading, “Most U.S. Adults Say They Had a High-Quality Postsecondary Education.” Gallup asked respondents whether they received a “high-quality” education, and they answered overwhelmingly in the positive.

Fifty-eight percent of those who earned a bachelor’s degree assigned their school a 5, the highest testimony, while another 31 percent graded them a 4.  Even those who never earned a degree came out at 40 percent a 5 and 30 percent a 4.

But most college students in America today do not receive a high-quality education. Academically Adrift, the 2010 book that struck the world of higher education like a bombshell, proved that only a small number of students make significant gains in critical thinking and problem-solving from freshman to senior year. Every survey of employers, too, shows them complaining about poor reading and writing skills.

Related: College Students Now, the Good and the Bad

One poll is particularly relevant here. When the Association of American Colleges & Universities commissioned a poll of college students and employers that focused on the latter’s workplace readiness, an astonishing gap was revealed. While 65 percent of students said that they were “well prepared” in written communication, only 27 percent of employers agreed. Similarly, wide discrepancies showed up in areas of critical thinking, problem-solving, and critical thinking.

Coincidentally, a week after the Gallup poll was published, The Wall Street Journal published an investigative report from Collegiate Learning Assessment on the scores of dozens of colleges and universities dating back to 2013 titled, “Many Colleges Fail in Teaching How To Think.”  The results are embarrassing, and they reinforce the judgments of Academically Adrift authors Richard Arum and Josipa Roksa.

The Journal reported that at more than half of 200 schools tested, at least a third of seniors were unable to make cohesive arguments, assess the quality of evidence in a document or interpret data in a table. This is a devastating finding. International rankings show U.S. college grads in the middle of the pack on numeracy and literacy and near the bottom when it comes to problem-solving.

The gist paragraph reads, “At some of the most prestigious flagship universities, test results indicate the average graduate shows little or no improvement in critical thinking over four years.”

Related: How to Make College as Bad as High School

The data comes from public records requests, and so the Journal’s findings apply to public institutions, not private ones. The biggest point gain didn’t come from top research universities. Plymouth State University in New Hampshire led the list.  The University of Kentucky and University of Texas-Austin students didn’t show much improvement at all. Those schools no longer use the test.  University of Louisiana—Lafayette scored low as well, and it, too, has dropped the test.

The Journal quotes a 2011 Lafayette graduate who recalls, “I wasn’t as focused as I should have been, but in a lot of classes, we just watched videos and documentaries, then we would talk about them. It wasn’t all that challenging.”  He now works in a local coffee shop.

I have no doubt that any other objective measure of actual learning that takes place from matriculation to graduation—except for the competitive areas of pre-med and STEM fields—will replicate these disappointments. Even if super-selective institutions point to the strong scores that their graduates earn on the CLA, they will not be able to show much value-added impact. That is, their students came in with sound critical thinking skill, and they left with, oh, a bit more of it.

Those data points force another interpretation of the high ratings people give to the quality of higher education. Instead of proving the actual rigor and excellence of undergraduate instruction in the United States, the sanguine estimates evince the low educational standards of American millennials. They just don’t know what actual excellence is. How could they when grade inflation in high school and college has reached such an absurd level that nearly half of all college grades are in the A range. If their teachers awarded them the top mark, well, then, they learned a lot in the course.  If the work that was required of them during the semester seemed suspiciously light, well, that may be due to the sparkling intelligence of the student, not to a cushy workload.

Or, perhaps, the faith that they received a high-quality education only proves their high gullibility. Every college has abundant marketing materials that proclaim the wonderful education they provide, and the students trust those pledges of superiority. It soothes their vanity. After all, the more superb the education they received, the more educated they are. The respondents in the Gallup poll are early in their adult lives, searching for jobs and for spouses, they want to believe in their own special condition.  Acknowledging a crummy education hampers their self-confidence. They need the power of positive thinking.

Millennials have been encouraged ever since kindergarten to overestimate their own abilities. They aren’t going to stop once they graduate. It takes several years of the realities of the American workplace to contain their judgment.