-->

Could this site post a breakdown of how the $25 million allocated for SOA is or has been spent?

Are any state funds being provided to local school boards for the purpose of evaluating their curriculums in light of the SOL?

While SOL test results are certainly a wake up call, what steps are being taken to validate the accuracy of the SOL tests before local school boards direct changes to curriculums and leason plans in efforts to improve their future test results?

Dear Mr. Westhoff:

Listed below are your questions regarding the SOL test and my responses.

QUESTION:

While SOL test results are certainly a wake up call, what steps are being taken to validate the accuracy of the SOL tests before local school boards direct changes to curriculums and lesson plans in efforts to improve their future test results?


RESPONSE: The Department of Education takes the construction of the SOL test very seriously and follows the prescribed steps needed to be confident that they are reliable and valid tests. Dr. Susan E. Phillips, professor at Michigan State University and noted testing expert, in commenting on the Virginia program said that "The most important validity evidence for a statewide, criterion-referenced test is evidence that the test items measure the state objectives and that each test form matches the Virginia test specifications. Such evidence usually consists of professional judgments made by representative panels of content experts."

Virginia's test development process employs just such a procedure for the expressed purpose of assuring that each and every SOL test administered is a valid measure of the SOLs for the content area.

Each SOL test is developed with the cooperation of a Content Review Committee composed of Virginia educators who have experience and expertise in the area and grade level of the test. Working with the Department and the testing contractor, the Content Review Committees review each test question before it is field-tested. In order to make it to field test status in Virginia, the question must, in the eyes of the Content Review Committee, meet the following four criteria:

* Does the question measure the SOL it was designed to measure?
* Does the question appropriately measure content or skills that students in Virginia should be expected to learn by the spring of the designated grade level or near the end of the course?
* Is the difficulty of this item appropriate?
* Is the question from content that stereotypes, offends, or unfairly penalized students on the basis of personal characteristics such as gender, ethnicity, religion, or socioeconomic status?"

If the item meets these criteria it will be used as a field-test item with Virginia students during a regular administration of the SOL tests. Once a test question has been field tested, the Content Review Committee is convened again to review the items and answer these same four questions, but this time with field test information about the test questions.

Results from the field-test administration considered in the decision to include questions in the operational test forms and constructing equivalent forms included: traditional item statistics, Rasch item statistics, and differential item functioning (DIF)

Traditional Item Statistics: frequency distributions, item mean (P values), standard deviations, n count, point-biserial correlations, biserial correlation, item reliability index, response distribution for each option for all respondents, for high middle and low ability groups, by gender and by ethnic group. In addition, traditional item difficulty and item discrimination statistics were computed.

Rasch Item Statistics: To supplement the traditional statistics, item difficulty parameter estimates based on Item Response Theory (IRT) were computed. Under the Rasch model of IRT, a common underlying construct is assumed to be measurable and estimable as a function of item or test performance making it possible to estimate item difficulty and item fit.

The Rasch IRT method of computing DIF was also employed to provide item difficulty estimates among demographic groups. Under the assumptions of the Rasch model, the only reason for differences in item difficulty statistics among groups is some group characteristic other than achievement. When the Rasch item difficulty estimates are statistically significant between groups, it is an indicator that further examination is warranted. The Rasch IRT procedure was used to compare white and African American students, white and Hispanic students, and male and female students.

Differential Item Functioning (DIF): DIF statistical procedures such as the Matel-Haenszel Alpha compute the probability that one demographic group is more likely to answer an item correctly than another group. This information is useful in reviewing items and tests for potential bias. High values of the Matel-Haenszel Alpha indicate that an item interacts differently among equally able students in a reference and comparison groups. When the probability is significantly different across groups, an item warrants further examination. The Mantel-Haenszel Alpha procedure was used to compare white and African American students, white and Hispanic students, and male and female students.

Since passing the high school SOL assessments will be a graduation requirement, it is especially important that they be free of factors that unfairly impact a group of students. Therefore, an additional bias review is conducted by a separate Bias Review Committee representing each content area to be tested. Bias Review Committee members are asked to scrutinize items for potential stereotyping or other forms of bias. The purpose of the bias review is to identify any items that appear to have the potential to treat any ethnic, gender or regional group of students differently from other groups. Committee members examine the response distribution for each of the demographic groups identified for the review. The intent of the review is to determine if members of a certain group were drawn to one or more of the answer choices for the item. If a large percent of one group selected a particular response or did not select a particular response, the item is carefully examined.

The training and procedure is similar to those carried out during the Item Review meetings. The committee's task focuses solely on reviewing test items for potential bias after the items have been reviewed by the Content Review Committees. It is the committee's responsibility to ensure that items are fair to all students and that all students would have an equal opportunity to demonstrate achievement regardless of gender, ethnic background, religion, socio-economic status or geographic region.

Once a test question "passes" all the stages of the development process-

* Initial review by the Content Review Committee
* Field testing
* Bias review
* Data review

The question is placed in the item bank and is eligible for use on future SOL tests. All items that appeared on the spring 1998 SOL tests were subjected to this procedure. All items that appear on subsequent forms of the SOL tests will be developed in this same manner and receive this same level of scrutiny.

Each SOL test is constructed according to the specifications of the test blueprint. In preparing a test form for operational use, the testing contractor selects questions from the item bank in accordance with the blueprint specifications. Once draft test forms are constructed, the Content Review Committee is again convened. Committee members assume the task of approving or editing two forms of each grade level or end-of-course test to determine the content validity and equivalency of the test forms as a whole. While the previous committee reviews were concerned with individual questions, the focus of the review is the full operational test forms. At this stage there may be additional edits or revisions.

It is after this final stage that the test forms are ready for use in a SOL test administration.

As you can see, each SOL test is grounded in a complex process that meshes technical requirements for good tests, with practical considerations from everyday teaching. Given this process, you can be confident that the SOL tests administered in Virginia accurately reflect the expectations embodied in the standards themselves and reflect the level of accomplishment of our students. The results of the tests thus become a strong component in decisions to revise curricula, adjust lesson plans, and provide specialized services to students.

If you have further questions, please feel free to call Cameron Harris at the Department of Education at (804) 225-2102.


Virginia's K-12 Education Reform: Virginia's Standards of Learning

Virginia's Standards of Learning: At a Glance


Adopted by the Board of Education in June 1995 to set clear and concise expectations for student learning and achievement

* Rigorous

* Academic

* Measurable

* Clear and Understandable


Outline the minimum knowledge and skills which students should acquire in the four core areas of English, Mathematics, Science, and History and Social Science (History, Geography, Civics, and Economics), as they progress from kindergarten through the 12th grade.


Contain computer technology learning standards which are intended to result in computer literacy for all students by the time they enter high school


Represent minimum curricular requirements


Developed through extensive public participation, including educators, parents, teachers, special interest groups, and the business community

* Lead school divisions built consensus

* More than 5,000 Virginians participated


Product of widespread public review through solicitation of comments by Board of Education

* 30,000 draft copies distributed to the public

* 10 public hearings statewide


Adoption followed by comprehensive information, training, and support activities

* Distributed to all school divisions and interested groups and individuals

* "Share Fairs" of resource information; Standards of Learning working sessions


The Commonwealth provided generous funding to local school divisions to support the implementation of the SOLs, including funds for materials, training, and student instruction (This funding is in addition to the funding provided for the Standards of Quality)

* Fiscal Year (FY) 97: $6.2 million for teaching materials

* FY 97: $1.6 million for teacher training and curriculum development

* FY 97-98: $6.6 million provided for the Early Reading Initiative for students to support K-5 reading SOLs;

* 98-2000 Biennium: $12.1 million provided for Early Reading Initiative;

* 98-2000 Biennium: $25 million for instructional materials

* 98-2000 Biennium: $25.1 million for teacher training and staff development;

* 98-2000 Biennium: $28.3 million for remedial instruction for students and teacher training; and

* $18 million provided to purchase graphing calculators and scientific probes to support Math and Science SOLs.

If further information is needed, please feel free to call Dan Timberlake, assistant superintendent for finance at the Department of Education, (804) 225-2025.

Has the Dept of Education taken into account the impact on students with special needs (many of whom do not perform well on standardized tests,with or w/o accommodations, but who do very well in school)? Does the expert from Michigan State have any background or knowledge regarding this aspect of SOL testing? How have special education practioners and experts here in Va. been involved in the SOL conversation? At the beginning of the 98-99 school year, we were told that 8th graders would be subject to the SOL tests but this would not impact their graduation status. Now we learn that SOLs will be factored into decisions with regard to promotion and grade point averages this year. Will we hear anything else before the end of the year? Was any money allocated to better prepare special education students for SOL testing (beyond accommodations)? Aside from UVA and another Va. institution and this one expert from Mich State, I would appreciate knowing who else has critiqued the SOL content and purpose. For this multi-million dollar effort, I assume the VA Department of Education soliciated ideas from recognized education experts and proponents nationwide. Is that so? I look forward to attending the May 18 meeting in Alexandria re: the purpose and future of SOLs. Thank you for a prompt response to these several questions.

Dear Ms. Lader:

The Department of Education has taken into account that students with disabilities need accommodations and modifications that "level the playing field" when taking the SOL tests. An extensive set of guidelines on SOL testing accommodations has been adopted by the Board of Education and distributed to school divisions, parent information centers, and other agencies/organizations statewide. Special education personnel were deeply involved in the development of these accommodations guidelines.

The use of the 8th grade SOL scores in determining whether or not a student is promoted is a local decision.

Students with disabilities may participate in remedial programs for students who need additional help to pass the SOL tests.

If you need further assistance, please call Mr. H. Douglas Cox, director, Office of Special Education and Student Services at (804) 225-2402.

Who exactly is responsible for introducing and teaching two-digit addition and subtraction using numbers less than 99 with regrouping?
(Example: 27+19 = ) Objective 2.8 is unclear.
The second grade teachers say it is the third grade teachers. The third grade teachers believe it is the second grade teachers. Objective 3.8 however, jumps up to adding and subtracting with/without regrouping up to 9,999. The third grade objectives take a considerable jump in almost all of the content areas. The objectives need to be more specific and they need to increase gradually from grade to grade.

What about the handwriting objective 3.9, students will write legibly in cursive...Who is responsible for introducing and teaching the correct formation of cursive letters? Third grade or second? When I taught second grade, I taught regrouping and introduced cursive and correct letter formation.

Thank you for responding to my May 14 inquiry. I will indeed contact Mr. Cox. I would, however, like to stress that test preparation for special needs students does not necessary equate to remedial classwork. It may have more to do with working with individual students to improve their strategies relative to taking this particular set of tests. That really equates to providing additional funds and/or resource materials at the school level for special education and regular ed teachers. I do not get the impression that SOL funds are earmarked in this way, but would be pleased to hear that they have been and to learn the particulars. I must also say that the listing of accommodations does not address many of the issues faced by children with speech/language difficulties and addressed in IEPs. For example, while my child has done very well in math, he felt the test this year was looking for his reading comprehension skills moreso than his math skills. As these tests are refined, as they must be, listen to the parents and the children faced with these issues. (By the way, my school district received one, repeat one, copy of the booklet on special education accommodations. Someone needs to follow through and determine how better to handle this in the year to come. Additionally, I see no reason why State of Va. cannot translate the parent info on SOLs at least into Spanish, Russian, Arabic, Vietnamese - whatever the largest 2nd language groups may be. This costs you one translation each and can be posted on your website. Finally, I did not receive a response to my question about the breadth and depth of outside review of the SOLs to date. The statistical expert from Mich. referred me back to you. On this point, I feel I am owed a response. Thank you.

Dear Ms. Turner:

I am writing in response to your request regarding the introduction and teaching of two-digit addition and subtraction. The mathematics Standards of Learning which you identified in your note are as follows:

1.8 The student will recall basic addition facts, sums to 10 or less, and the corresponding subtraction facts.

2.6 The student will recall basic addition facts, sums to 18 or less, and the corresponding subtraction facts.

2.7 The student, given two whole numbers whose sum is 99 or less, will
* estimate the sum; and
* find the sum using various methods of calculation (mental computation, concrete materials, and paper and pencil).

2.8 The student, given two whole numbers each 99 or less, will
* estimate the difference; and
* find the difference using various methods of calculation (mental computation, concrete materials, and paper and pencil).

3.8 The student will solve problems involving the sum or difference of two whole numbers, each 9,999 or less, with or without regrouping, using various computational methods, including calculators, paper and pencil, mental computation, and estimation.

As you know, in mathematics, particularly in computation, there is the expectation of a spiraling effect which one skill builds upon a previous one. This is the case in the development of addition and subtraction skills. For example, it is expected that first-grade students will recall the basic addition facts with sums to 10 or less. These skills are essential and are the building blocks to student's successful achievement of finding the sum of two whole numbers (99 or less) in second grade. Without knowing the facts, students cannot demonstrate mastery of addition.

In the Virginia Standards of Learning program, students are expected to demonstrate mastery of the skills identified in the grade level at which they occur. Therefore, in answer to your question, grade-two students are expected to demonstrate paper and pencil mastery of addition of two whole numbers whose sum is 99 or less. This includes problems such as 45 + 36 where regrouping is required, but not problems such as 89 + 64 where the sum is greater than 99.

In subtraction, grade-two students are expected to demonstrate paper and pencil mastery of finding the difference of two whole numbers, each 99 or less. This includes problems such as 94 - 87 which requires regrouping, but not problems such as 137 - 25 where one of the whole numbers is greater than 99.

Addition and subtraction skills in the third grade require computation with whole numbers, each 9,999 or less. Because students will have mastered regrouping by the second grade, this third-grade SOL spirals such that the change is only with respect to place value (e.g. through five-digit numbers -- when you add two four-digit numbers such as 9,876 + 8,765), but there is no change with respect to the regrouping skills that were mastered in second grade.

I hope that this response helps to explain the expectations of the Standards of Learning. If I can be of further assistance, please do not hesitate to contact me, Carol Rezba, mathematics specialist K - 8, Virginia Department of Education, e-mail:{crezba@pen.k12.va.us,crezba@pen.k12.va.us}, Phone: 804-225-2676.