Validity challenge in GenAI models: Evaluating the validity of content generated by text-to-image models in the context of social studies education

Generative artificial intelligence (GenAI) models have led to many positive changes in educational settings; however, the validity of the content they produce remains a significant topic of academic discussion. This research aims to determine the validity of content produced by text-to-image models within the context of social studies education. For this purpose, different curricular outcomes from the social studies curriculum implemented in Türkiye were identified, and image content related to them was generated using DALL-E 3. The validity of this content was assessed within a panel consisting of social studies teachers. Quantitative analyses revealed inconsistencies, showing that some images are sufficient in terms of scientific accuracy as well as socio-cultural and geographical relevance, while others are not. Intraclass correlation coefficient analyses demonstrated that there was significantly moderate agreement among the panelists regarding these evaluations, indicating that the assessments are reliable. Qualitative analysis, on the other hand, revealed that panelists evaluated some images positively in terms of validity, noting that the content produced by these models has significant potential to enhance the learning of relevant outcomes of social studies. However, it was indicated that certain images exhibit prominent algorithmic biases, provide misleading or incorrect information, include details that could be characterized as hallucinations, and could potentially negatively impact the learning process. Additionally, it was determined that some images, in an attempt to highlight a particular cultural element, displays representations that are either disconnected from reality or overemphasized, which could be termed “Socio-cultural algorithmic exaggerations”. At this point, it is considered essential for all educators, including social studies teachers, to develop a critical perspective on the content created by GenAI models and to use this content only after thorough evaluation.