Test anxiety is a complex phenomenon. Although studies have mostly focused on traditional assessments and test anxiety in this context, there are few studies that have examined students’ test anxiety when taking tests online. To understand college students’ test anxiety levels and help them to cope with this unpleasant and uncomfortable condition, it requires a psychometrically sound, valid and reliable instrument that measures test anxiety levels of college students in online testing that this delivery format only increases across campuses. In the development phase of this study, the psychometric properties of the Revised Test Anxiety-Online (RTA-O) scale were examined, and then a brief, unidimensional form of the RTA-O (RTA-O-SF) scale was created. After creation of the RTA-O-SF, the second phase of this study provided an initial examination of the reliability and validity evidence of the RTA-O-SF scale using Rasch Analysis and the results showed strong validity and reliability evidence of the scale scores for future use. In addition, the results revealed that older, male college students with higher GPAs and students with high self-esteem had lower online test anxiety levels. Also, hours spent studying during midterm/finals weeks and perceived stress were positive predictors of online test anxiety levels. The results obtained from this study provided evidence for validity and reliability of RTA-O-SF scale. However, the use of the RTA-O-SF scale should be done with caution in clinical settings for diagnosis purposes. Implications for researchers, faculty members/university administrators, and counselors/health professionals, and limitations and future directions were presented.