Cultural domain in development and validation of tests on arts to assess the skills of student graphic designers

Abstract

The purpose of the study was to examine whether the presence of items that covered cultural questions in the test оn arts for student graphic designers influenced the fairness of the test across different ethnic and social groups. The reason for the study was to address the gap in the development and validation of tests оn arts that include a cultural domain to assess the skills of undergraduate graphic designers. The research design was based on DIF and DTF methods to examine the factorial structure of test data and to identify whether the factorial structure of the test was the same across ethnic and social groups. A one-factor CFA model was applied to perform measurements for categorised ethnic and social status groups to identify whether the factorial structure was similar or identical for them. The goodness-of-fit measures were calculated using the chi-square statistics, CFI, TLI, and RMSEA to identify how the obtained data is consistent with the hypothesised model. The inclusion of local culture-related questions in the tests оn arts to assess the skills of student graphic designers influences the individuals' latent traits which lead to an increase in DIF values. Though there were detected seven items with the DIF for the whole test, the DTF measurements showed that the DIF effect eliminated each out at test level which is related to the fact the certain ethnic groups performed better in answering some specific test items, while certain status groups performed better in answering some other test items. It is noteworthy that DTF for the Ukrainian art (miscellaneous) section was between 0.07 and 0.14 meaning a moderate DTF effect. However, the DTF variance values for the sections of principles of design and principles of art were lower than 0.07 meaning a small effect. Therefore, it could be concluded that small DTF effects found in both the whole test and each test section separately indicated that DIF effects eliminate each other at the test level.