In the fall semester of 2000, Southwest Virginia Community College’s math division implemented the Virginia Community College System Developmental Studies Implementation Task Force recommendations.  One of these recommendations was to ensure that the approved algebra content items were included in our MTH 03 (Basic Algebra I) and MTH 04 (Basic Algebra II) courses.  Accordingly, these two course syllabi were reviewed and reorganized to meet these guidelines. 

We also revised our in-house readiness tests for MTH 03 and MTH 04 – which had been used since the mid 1990’s as a follow-up placement tool to the COMPASS and ASSET tests – to make them consistent with the content actually covered in these courses.  However, no changes or reorganization of content items were performed on the COMPASS or ASSET placement tests. 

Consequently, our recent comparison of the MTH 03 and MTH 04 syllabi with the corresponding COMPASS and ASSET placement tests has revealed major inconsistencies in content coverage.  These differences suggest that students may not be properly placed into developmental algebra courses.  Thus, the purpose of this study was to assess how well the ASSET, COMPASS, and in-house readiness tests actually do in their placement of students into the algebra developmental courses.  Specifically, we collected and analyzed student data from these placement tools and the core final exams for MTH 03 and MTH 04 to determine how well these placement tools predict success in our developmental algebra courses.

Descriptive Statistics 

Descriptive statistics in Table 1 reveal characteristics of the developmental algebra classes.  First, on-campus students account for two-thirds of the students in developmental algebra classes, and female students outnumber male students 2 to 1.  Students for this study were enrolled in Fall 2001, Spring 2002, or Summer 2002 classes.  The fall semester enrolled the largest number (195 or 54%) of the 364 students in the study, whereas spring semester enrolled more that half that many students (133 or 37%), leaving summer semester to enroll only 36 (or 10%) of the total students.  The breakdown by math courses reveals that 261 of the 364 students (or 72%) enrolled in MTH 03, with only 103 students (or 28%) enrolled in MTH 04.

 

Table 1

SwVCC Math Students Categorized According to
Site, Gender, School, Term, and Math Course

 

          Variable Category                                  n                                                           Percent

Site
On-Campus	239	65.7
Off-Campus	125	34.3
Total	364	100.0
Gender
Male	123	33.8
Female	241	66.2
Total	364	100.0
School Term
Fall, 2001	195	53.6
Spring, 2002	133	36.5
Summer, 2002	36	9.9
Total	364	100.0
Course
Math 03	261	71.7
Math 04	103	28.3
Total	364	100.0

 

Although completion of all needed developmental mathematics courses is a prerequisite to enrolling in college-level math courses, these data suggest that many of the developmental students who successfully complete MTH 03 do not go on to take MTH 04. 

What is happening to these students?  Are they bypassing the prerequisite MTH 04 course and enrolling in MTH 151 or MTH 163?  With the implementation of People Soft in Spring 2004, students should not be able to bypass prerequisite courses.  Does this mean that we will see more comparable numbers of students completing both MTH 03 and MTH 04 in future semesters?  We hope that will be the case but, of course, a certain percentage of attrition will still exist from semester to semester.  Regardless, students who remain enrolled and need the developmental math courses should complete the entire MTH 03-04 sequence.

Comparing developmental students by site – on campus and off campus – provides more valuable information.   Table 2 lists the mean scores for the core final exam, with COMPASS, ASSET, age, and GPA shown separately by site for on- and off-campus students in MTH 03 and MTH 04.

 

Table 2

Descriptive Statistics on the Variables—
Core Final, COMPASS, ASSET, Age, and GPA—by Site

 

Variables                                                                 On-Campus                                        Off-Campus

Core Final
N	239	125
Mean	79.2	67.9
SD	11.01	18.67
Range	56	91
COMPASS
N	239	125
Mean	35.6	38.5
SD	7.95	6.93
Range	69	53
ASSET
N	239	125
Mean	37.1	37.8
SD	4.05	3.95
Range	23	21
Age
N	239	125
Mean	24.8	30.2
SD	7.68	9.42
Range	35	48
GPA
N	239	125
Mean	2.6	2.9
SD	.91	.75
Range	3.86	3.43

 

In comparing these means, we find that the most noticeable difference exists with the core final exam.  The on-campus mean of 79.2 is more than 10 percentage points higher than the off-campus mean of 67.9.  This sizeable difference in means on the core final exam prompted a run of t-tests for differences in the means for all these same variables in Math 03 and Math 04. 

First, for MTH 03, Table 3 shows the means for these same five variables, by site, and then   presents the t-values and levels of significance for differences between these means.

 

Table 3

Mean Scores on Math 03
Variables by Site—On Campus vs. Off Campus

 

  Variables                      Mean Score                          SD                                 t                                p

Core Final
On Campus n=168	80.7	11.05	5.16	<.0001
Off Campus n=93	68.7	20.82
COMPASS
On Campus n=168	36.2	8.49	-2.33	.020
Off Campus n=93	38.7	7.11
ASSET
On Campus n=168	36.6	3.43	-2.79	.006
Off Campus n=93	37.9	3.87
Age
On Campus n=168	25.0	7.64	-5.51	<.0001
Off Campus n=93	30.9	9.37
GPA
On Campus n=168	2.52	.92	-4.11	<.0001
Off Campus n=93	2.97	.73

  

Students in MTH 03 on-campus classes outperformed off-campus students on the core final exam at the <.0001 level of significance, even though off-campus students had significantly higher scores on the COMPASS (.02 level) and ASSET (.006 level) tests and higher GPA scores (<.0001 level) overall. 

Table 4 reveals consistent findings for on- and off-campus students in MTH 04.  MTH 04 on- campus students outperformed off-campus students on the core final exam at the <.0001 level also.  This better performance by on-campus MTH 04 students was achieved even when off-campus students had significantly higher (.004 level) COMPASS scores.  However, there were no significant differences in ASSET and GPA between on- and off-campus students in MTH 04.

 

Table 4

Mean Scores on Math 04

Variables by Site—On-Campus vs. Off-Campus

                       

     Variables                   Mean Score           SD                                 t                                  p

Core Final
On-Campus n=71	75.7	10.17	4.73	<.0001
Off-Campus n=32	65.6	10.03
COMPASS
On-Campus n=71	34.1	6.27	-2.921	.004
Off-Campus n=32	38.0	6.46
ASSET
On-Campus n=71	38.1	5.12	.471	.639
Off-Campus n=32	37.6	4.22
Age
On-Campus n=71	24.2	7.79	-2.18	.032
Off-Campus n=32	28.1	9.39
GPA
On-Campus n=71	2.76	.86	- 0.581	.563
Off-Campus n=32	2.86	.81

 

 Another variable worth consideration is that of age.  The data show a significant difference in age for the on- and off-campus students, with off-campus students significantly (<.0001 for MTH 03 and .032 for MTH 04) older than on-campus students.  Thus, off-campus students tend to be older (mean of 30 versus 25 – Table 2), female, and a little better prepared academically than their on-campus counterparts. 

Graphs 1 and 2 provide a representation of the differences in the means for these variables in the MTH 03 and MTH 04 classes.  Graph 1 plots the mean scores for each of the five variables (final core, COMPASS, ASSET, age, and GPA) for MTH 03 by site – on-campus versus off-campus.  Notice that the COMPASS scores switch relative positions, or exhibit disordinal interaction, for these MTH 03 students.  Graph 2 plots the mean scores for each of the same five variables for MTH 04 by site, on- campus versus off-campus.  For MTH 04 students, both COMPASS and ASSET exhibit disordinal interaction.  These graphs simply reiterate the fact that the basic difference between off- and on-campus student performance lies with the core final exams.  Even though the off-campus students had higher or similar COMPASS, ASSET, and GPA scores, their core final exam scores in both MTH 03 and MTH 04 were significantly lower (<.0001 level) than those scores for on-campus students.  Furthermore, with the inconsistent behavior of the COMPASS and ASSET scores of MTH 04 students, it appears that these two placement tests are measuring different aspects of student performance.  Thus, a closer look at these two placement tools is warranted to determine if they are consistent in their placement of students into developmental mathematics courses

 

These findings on performance raise concerns about our policy of collapsing courses (MTH 02, MTH 03, and MTH 04) at off-campus classrooms.  (In some situations, off-campus sections of two or three different math courses, all of which have low enrollment, are collapsed into one section that meets at the same time, in the same room, with the same instructor.  This presents a monumental challenge to these adjunct teachers.)  Currently, our division is aware of this situation and is looking for possible solutions to the problem.  One alternative under consideration would involve collaboration between off- and on-campus faculty in teaching these courses. 

Multiple Regression 

Next, we performed a multiple regression to determine how well the ASSET, COMPASS, and the readiness tests place students into developmental math courses.  Since the independent variables, math class (MTH 03 or MTH 04), sex, GPA, age, and site (on- or off- campus), each had the potential to explain additional variation in the dependent variable, core final exam, they were also included in the regression equation. 

Table 7 reveals that only five of the variables in the regression equation – site, ASSET, age, GPA, and math course – do actually predict a significant percent of the variation in the core final exam.  But even after including all eight variables in the regression equation, only 27.5% (Table 5) of the variance in the dependent variable – core final exam – was explained.  The regression coefficients reflect no good predictor for the core final exam.  This lack of a good predictor variable explains the fluctuation in the variable means displayed earlier in Graphs 1 and 2. 

Since there is no one good predictor, it is imperative that we consider all of these variables along with any additional information available for proper placement into a developmental mathematics course.

 

Table 5

Model Summary:  Regression  of Core Final Grade on

Selected Independent Variables  (Predictors)

 

  Core Final  Predictors                                Multiple  R                                                 R

	.524	.275
Site
Ready Test
COMPASS
ASSET
Age
Gender
GPA
Term 1—Fall, 2001
Term 2—Spring, 2002
Math Course

  

Table 6

Analysis of Variance:  Regression of Core Final Grade
on Selected Independent Variables

 

  Source                       DF                           SS                        MS                           F                             p

Regression	10	22695.21	2269.52	13.38	<.0001
Residual	353	59891.63	169.67

 

Table 7

Predictor Coefficients:  Regression of Core Final Grade
on Selected Independent Variables

      Core Final Predictors                       Predictor Coefficients                             p

Site                          x	-.301	<.0001
Ready Test             x	.141	.062
COMPASS             x	.055	.244
ASSET                    x	.213	<.0001
Age                          x	-.114	.025
Gender                    x	.006	.903
GPA                        x	.279	<.0001
Term 1 Fall, 2001  x	-.030	.537
Term 2 Spr, 2002  x	.056	.252
Math Class            x	-.162p; -.162	.001
(Constant)             C	29.658	.001

 

 

Recommendations

The findings in this study imply the need for future research as well as for changes in our course design for developmental mathematics. 

First, the low enrollment in MTH 04 – a required prerequisite for college-level mathematics courses – justifies a tracking of developmental students upon completion of MTH 03 to determine if large numbers are bypassing this required math course. 

Second, the readiness tests, although close (.062 level), did not predict a significant percentage of the variation in the core final exam.  Still, giving this short test during the first week of classes would assist teachers in correctly placing students who may have been on the borderline between MTH 03 and MTH 04.  In addition, when a student’s ASSET and COMPASS scores produce conflicting recommendations, the student’s performance on a readiness test may provide sufficient additional information for the instructor to determine the proper mathematics class for the student. 

Since the COMPASS test also failed to predict a significant percentage of variation in the core final exam, a closer look at this placement tool is warranted.  In addition, inconsistencies have been noted between the COMPASS and ASSET placement recommendations, perhaps because these two placement tools are measuring different cognitive aspects.  A careful look at the “branching” aspect for the COMPASS test may reveal needed changes in the test format and/or item content.  Otherwise, faculty and counselors may provide better advisement to students by using only the ASSET scores for assistance in placing students into the appropriate developmental algebra course. 

What is truly needed is one placement tool that serves as a good predictor for success on the core final exam for MTH 03 and MTH 04.  Since ASSET was the only placement test in this study that was a significant predictor of success on the core final exam, perhaps aligning the content in MTH 03 and MTH 04 more closely to the content items on the ASSET tests would make it an even better predictor for success in these courses.

 

Note

The author would like to thank Bill Fiess and Dr. Sexton Burkett for their help with the graphs and statistical design, and Tammy McCracken for her assistance with the SPSS package.