Validity and Reliability Studies
Latest study
DEVELOPMENT AND INITIAL VALIDATION OF THE FURTADO-GALLAGHER COMPUTERIZED OBSERVATIONAL MOVEMENT PATTERN ASSESSMENT SYSTEM – FG-COMPASS
Mastery in fundamental movement skill (FMS) performance (e.g., kicking) is an important factor in preventing unhealthy weight gain (Okely, Booth & Chey, 2004) as well as increasing participation in organized and habitual physical activity among children and adolescents (Foley, Harvey, Chun & Kim, 2008; Hume, Okely, Bagley, Telford, Booth, Crawford & Salmon, 2008; Mazzardo, 2008; ). Since schools are involved in assisting children improve FMSs, assessing FMS development is crucial. Therefore, the purpose of this study was to develop and collect initial validity evidence for a new observational assessment tool (FG-COMPASS) to evaluate FMS development of children 5- to 10 years of age. In Phase I, 110 video clips of children performing 5 locomotor and 6 object-control FMS were developed. In Phase II, rating scales were developed for each FMS. In addition, the efficacy of the rating scales was evaluated by comparing judgments of 30 undergraduate students to a standard. Weighted kappa indicated that the agreement was best for hopping (Kw= .85), striking and batting (Kw= .79), skipping (Kw= .77), overhand throwing (Kw= .74), catching and hand dribbling (Kw= .72), and horizontal jumping (Kw= .70). The poorest agreement was for kicking (Kw= .51), side sliding and leaping (Kw= .61). The proportion of specific agreement (Ps) was calculated for each skill to determine the source of disagreement. Six skills had at least one category (e.g., initial, elementary, mature) with Ps values below .70. After review of the problem statements, the rating statements were modified. Next, a follow-up study will be conducted to verify whether the changes will improve accuracy of classifications. In conclusion, this study provided initial validity evidence indicating that five rating scales developed for the FG-COMPASS can be used to classify individuals based on their FMS development. However, a follow-up study is needed to verify whether changes made to six rating scales will improve observer agreement.
Source: http://etd.library.pitt.edu/ETD/available/etd-08052009-134650/
FURTADO-GALLAGHER MOVEMENT SKILL ASSESSMENT: COLLECTING EVIDENCE FOR CONTENT-RELATED VALIDITY
The National Association for Sport and Physical Education (NASPE) emphasizes in its second edition of the National Standards for Physical Education the importance of linking instruction and assessment in physical education. NASPE standards reflect what students should know and be able to do as a result of a quality physical education (NASPE, 2004). This link is even more beneficial to students and teachers alike when assessment is aligned with the NASPE content standards. Therefore, the purpose of this study was to develop a criterion-referenced assessment tool to test movement concepts and fundamental movement skills of children ages 3 through 10 and provide evidence for content-related validity. An attempt was made to develop an assessment tool that is tied closely to the postulates of the National Standards for Physical Education (NASPE, 2004). The gathering of evidence for content-related validity was accomplished through judgments of a panel of 20 content experts (professors = 8; and PE teachers = 12). An Internet-based item review form was used to collect information from experts. Experts rated content at item level (31 items divided in 7 categories) using a four point likert-type scale where 1 being ‘not important at all’ and 4 ‘very important’. Content was also rated at test level (7 questions) addressing the match between the test content and the test general characteristics (e.g., match between the proposed set of items and the test purpose). This set of questions were rated on a five point likert-type scale with 1 being ‘poor’ and 5 being ‘excellent’. Many of the expert reviewers also provided written feedback on the content of the test. Descriptive statistics (percentage of responses and median) were used along with qualitative procedures for data analysis. The analysis of content at the item level yielded revision of 11 items. Also, three items were considered not appropriate for either 3-6 or 7-10 age groups. Four items were included based on experts written comments. Finally, two items were dropped out of the test. The analysis of the content at the test level showed that questions were rated as ‘very good’ or ‘excellent’ by 80% or more of judges. The information collected from content experts served to validate and confirm the content of this assessment tool while providing helpful feedback with which to make improvements in content and format. Results of this study provided an initial support for the content-related validity of the Furtado-Gallagher Movement Skill Assessment. Content validity is an ongoing process throughout the development of any instrument and further analysis will be needed to confirm the content validity of this assessment tool.
Source: http://etd.library.pitt.edu/ETD/available/etd-12042004-093510/