Creative Thinking Skills in Social Studies Written Exam Questions

In this study, Social Studies teachers tried to examine the questions in their exams according to their creative thinking skills. In the study document analysis and semi-structured interview methods were used for this purpose. A total of 2,065 questions were examined from the examinations prepared by 61 teachers working in 20 different schools in Ortahisar, which is in the central district of Trabzon. The exam questions were examined in terms of the question types commonly used in education, their class levels and their status for creative thinking skills according to their subjects. The SPSS program was used in the analysis of the questions and the findings were presented using frequency and percentage tables and graphs. Interviews with teachers were analyzed by descriptive analysis method. The themes were created from the data obtained from the interview answers. The themes were reflected on the tables according to the frequency of the teachers’ answers. For better understanding of the findings, examples of the teachers’ exam questions and direct quotations from teachers’ answers are given under the tables and graphics. As a result of the research, it was determined that the exam questions prepared by the social studies teachers were inadequate to develop and measure the creative thinking skills of the students.


Introduction
Since humanity has existed, education has been the basis of progress in the development of every society. Throughout the ages people have been trained for various purposes by various powers in the direction of specific goals. Religions, philosophical movements and branches of science have always proposed an educational system. In short, education is the most important activity realized by humanity (Ulken, 2001). People must learn not only for the continuation of its existence on the earth but out of human curiosity. Education can be defined in different ways in terms of its functions. However, the definition of contemporary education, which In the 2017 curriculum prepared to keep pace with the developing and changing era, creative thinking is emphasized as innovative thinking as in the 2005 program. The basic skills students are to gain through the program include creative thinking. The philosophy of the program emphasized the importance of creative thinking as follows (Ministry of Education, 2018: 3-4): "The ability to raise individuals who can play an active role in today's social and economic conditions is directly linked to the competitiveness of countries on the international scene. Given this situation, countries are looking for a model of education that will enable students to have responsibilities, solve problems, develop decision-making skills, and think critically and innovatively."

"In general, what is important in the development of innovative thinking is the use of techniques to produce ideas, the introduction of different ideas, and the production of ideas, the development of imagination, and the development of thinking skills."
Teachers are the most important people in bringing these skills to students. Teachers -who are inevitably part of the education system -should educate and develop both themselves and their students in a way that fulfills the requirements of the times. To ensure that students develop high-level thinking skills, teachers direct them to think and develop by directing qualified questions to them to produce different solutions to problems, to search for true truth, and to analyze old knowledge with new knowledge, taking advantage of their experience. Teachers will ask different types of questions, qualitative and supra-cognitive, will transmit cognitive skills such as creative thinking to the students (Aydemir & Ciftci, 2008;Kadir & Satriawati, 2017). The way you can achieve this is the measurement and evaluation of the curriculum. The creative thinking skills that can be developed during the course using various teaching methods can also be developed and measured by using correct measurement and evaluation methods and using the correct questions during the measurement and evaluation (Colak, 2008;Ozcan & Oluk, 2008;Selcuk, Kayili & Okut, 2004;Shaunessy, 2000;Wragg, 1998).
We can list the qualities of questions that require creative thinking skills and that develop these thinking skills with key words, question patterns, and sample questions as follows (Bloom, 1956;Fizzy, 2004;Ozden, 2014;Shaunessy, 2000;Dalton & Smith, 1986;Sensekerci & Bilgin, 2008;Tekin, 1993 If the student needs to bring an original product, • You need to make predictions and solve problems, • If it is necessary to combine things according to certain relations and rules, • Innovation, originality, invention, inventiveness and creativity are the

Samples of Question
Question 1. Could you organize a meeting on the protection of consumer rights? Speak to the people at the meeting about protection of consumer rights.
Question 2. Can you see a possible solution to prevent immigration from your village to the city?
Question 3. A tablet consisting of Assyrian nails was found in Anatolia. Which of the following judgments can be reached accordingly? A.
Agriculture is being carried out in Anatolia.

B.
Only the Assyrians used the nail script.
C. The Assyrians are the most advanced civilization.

D.
The Assyrians are in interaction with Anatolia.
The purpose of this study is, from the point of view of creative thinking in education and the importance of creative thinking skills, to determine the effect of social studies teachers' written exam questions which they applied during the first semester of 2017-2018 academic year on creative thinking skills.

Problem
The question of this research is "What is the place of creative thinking skills in the exam questions prepared by social studies teachers?" 1.2 Sub problems 1. Where are the skills of asking creative questions in social studies teachers' exams?
2. Which question types do social studies teachers prefer to measure and encourage creative thinking skills?
3. At what class level and in which branches of social studies did the teachers of social studies prepare the questions for creative thinking skills?

Method
This study was designed as descriptive research by designing within the framework of qualitative research approach. Qualitative research is generally defined as research that reaches findings without any quantification. The topics studied in these studies are described in detail, interpreted in depth and aimed to understand the viewpoints of the participants involved in the study (Ekiz, 2013;Creswell, 2016).
In this qualitatively designed research, document analysis and a semi-structured interview method prepared by the researcher were used as data collection tools.
The document review method is a research method that involves analyzing the whole of written audio-visual materials that contain traces of the information needed about the topic of research (Karasar, 2005). Document analysis can be used in qualitative research besides other methods such as interviewing and observation in order to increase the scope and validity of the research, but can also be used as a stand-alone method in cases where other methods are unable to be used directly (Cepni, 2014;Yin, 2015).

Participants
Participants of this study constitute 61 social studies teachers working in 20 different secondary schools affiliated to the Ministry of National Education in Trabzon. These teachers' questions were collected during the first semester of the academic year of 2017-2018 and a semistructured interview was conducted with 16 of these teachers. A random sample method was used when selecting the sample. Using this method; it is likely that all of the participants involved in the study have equal likelihood of being selected. In the random sampling method, the amount of sample to be reached is determined by random selection from the list (Cohen, Manion & Morrison, 2013). In the random sample method, the sample that should be reached is chosen at random from the list taken from the Trabzon National Education Directorate consisting of 113 Social Science teachers.

Data sources and data collection
A total of 2065 written exam questions were used in the study as belonging to the 5 th , 6 th , 7 th and 8 th grades applied by the Social Studies teachers in the first semester of the academic year of 2017-2018.

Data analysis
The collected questions, documented according to class levels, subjects and question types, have been analyzed in terms of being appropriate to critical thinking skills. In this evaluation, the opinions of three experts in the field of assessment and evaluation were used (Ekiz, 2013;Yin, 2015). The collected data has been analyzed in terms of frequency (f) and percentage (%) by using SPSS packet program and the results have been presented in graphics or table. In order to ensure the reliability and validity of the study, some analyzed questions have been presented as examples under the graphics (Cohen, Manion & Morrison, 2013).
Data obtained from document review was supported by semi-structured interview. The interview questions were prepared by the researcher by taking the opinions of three experts after a detailed literature review. In order to ensure the validity and reliability of the interview questions, interviews were conducted with 3 different Social Studies teachers before sampling. After this pilot scheme, the interview questions were rearranged by taking the opinions of the experts. In semi-structured interviews with teachers, researchers obtained in-depth information by asking additional questions with answers to questions along with three main questions.
For each teacher, the answers recorded on a separate paper were transferred to a computer and analyzed by reading them carefully in order to follow the order of the questions in the interview form (Yildirim & Simsek, 2013;Yin, 2015). In order to provide a more in-depth examination of descriptively analyzed questions and to reach unrecognized concepts and relationships in the answers of the participants to the interview questions, the words they used in relation to the problem of the research were identified and firstly coded and the frequency of the codes was determined done with content analysis. Themes have been created from the resulting code and these themes are presented in tables (Cohen, Manion & Morrison, 2013). In the analysis of the interviews, quotations were made without changing from the answers of the participant teachers in order to indicate the attractiveness of the opinions and to ensure the reliability of the study (Creswell, 2016).

Results
In this section, the findings are presented in a way to examine each research problem separately. In this respect, the data obtained from different data collection techniques are not presented separately and are presented in a way to explain each sub-problem. Tables and graphs were used to make the findings more clear and understandable. It has been determined that 141 (6.8%) of the social studies teachers examined in the first half of 2017-2018 academic year were prepared for creative thinking skills. In the first question of the interview with teachers to determine the place of creative thinking skills in the exam questions of the Social Studies teachers, the questions which the teachers paid attention to during the preparation of the exam questions were determined. None of the teachers gave an answer to the question of preparing creative questions. Only three teachers stated that students were careful to prepare questions that they could answer by thinking and interpreting. About: Ö1W: "According to the learning outcome of the student's power of expression and thinking of the answer in order to explore the question to ask more questions."

First sub-problem findings
The subjects that teachers pay attention to while preparing the exam questions are learnings outcome and student levels come. One of the seven teachers said they would pay attention to achievements in preparing the exam question: T10M: "I only take achievements into account when preparing the exam questions." One of the 7 teachers among the subjects said that they pay attention to students' levels in preparing the exam questions: T8M: "I prepare my exams according to the level of the students, and I am careful to prepare exam questions which will be easy for hard working people, and hardly anyone can get good grades." Another concern of the teachers was the scope of validity, 5 teachers stated that they pay attention to the scope validity when preparing the exam questions. For example: In addition to these issues, other subjects that social studies teachers have taken into consideration have been the use of visual exams in the exams and their lectures. In the course of the preparation of the exam questions, the subject pays attention to lectures.

T5M: "I don't ask anything I haven't said in class. I pay particular attention to the preparation of the exams."
The cases where social studies teachers take creative thinking skills into consideration are examined in Table 3. In a semi-structured interview to determine the ability of teachers to ask questions about creative thinking skills, it has been observed how teachers consider creative thinking skills in their exams. As a result of these examinations, 10 teachers stated that they did not consider creative thinking skills in their exams and 6 teachers considered them. The teachers who did not consider the creative thinking skills stated that they did not consider students because they would decrease their exam scores. In this regard: T2M: "Although I use open-ended questions in the exams, I use only creative thinking skills. The governance -parent and student triangle is an important factor for teachers. If the student's grade falls, the parent and the administration are uncomfortable, so we have to ignore the students' thinking skills in exams." The subject stated that student scores prevented them from asking creative questions. 5 of the teachers stated that student levels were not sufficient, 3 of the teachers did not have appropriate content for the Social Studies course thinking skills, and for 2 the success of the central exam system was an obstacle to asking creative questions.
Teachers who argue that student abilities are not sufficient for creative questions.
T5M: "It is not enough to think and write about levels of students and skills. They can't write and think." Teaches also noted the central examination system does not prepare creative questions: T11M: "The fact that the examination system is based on multiple-choice questions prevents the creative thinking skills from being taken into consideration. First, we have to change our understanding. As teachers, we can improve our exam questions." Some teachers say that they had asked creative questions but for some reason they have given up creative questions, they indicated that they are restricted due to some reasons. For example: Teachers who stated that they asked questions about creative thinking skills in the exam questions stated that they had the most open-ended questions and different types of questions. Another teacher has prepared questions about creative thinking skills through visual use and another teacher has prepared his questions according to Bloom's Taxonomy and stated that he has achieved this goal. Teachers indicating that they consider creative thinking skills by asking open-ended questions: When the 2,065 questions examined by the Social Studies teachers were examined, it was found that the most preferred question type of teachers was multiple choice questions and the least preferred questions were open-ended questions. As a result of the examination, it is understood that multiple choice questions constitute more than half of the exam questions (63%). A total of 5 questions, other than these types of questions, are categorized under other categories and these questions constitute only 0.2% of the examinations of teachers. Since this study is carried out through the measurement and evaluation methods commonly used in education, the questions in the other category have been ignored. When the question types used in exam of social studies teachers were examined according to their grade levels, it was observed that the number of multiple-choice questions increased from the 5 th grade (58.8%) to the 7 th grade (67.5%) and it was found to decrease slightly in the 8 th grade (62.3%). The questions of gap filling were mostly used in the exams of the 7 th grade (14%). Although there is no significant difference between the true/false questions according to the grade levels, the most (13.8%) is used in the 5 th grade exams. Short answer questions were mostly preferred in the 6th grade (6.9%) and least in the 5 th grade (2.3%) by the teachers. Parallel to true/false questions, matching questions were used the most in 5 th grade (12.3%) and least in 7 th grade (3.7%). Finally, open-ended questions were used the most in the 5 th grade exams (3.3%) and least in the 7 th grade exams (1.4%). In order to determine the reasons why the Social Studies teachers prefer the types of questions they use in their exams, the second question of the interview was the reasons why the teachers preferred the types of questions to be used in an ideal exam and the types of questions they used in their exams. In this direction, three titles emerged in the types of questions that the teachers found to be ideal. These are mixed, open-ended and multiple-choice questions. Ten of the teachers who were subject to the interview stated that the most ideal exam was the quizzes in which all question types classified as mixed were used in this study. While 4 of the teachers found open-ended questions to be ideal, 2 teachers stated that exams using multiple-choice questions were the ideal exams. It was determined that 7 of the teachers prepared the questions according to the exam form which they found ideal, and 11 of them prepared the exam questions which were different from the exam form which they thought ideal. One of the reasons behind these question types preferred by Social Studies teachers was the central examination system. In their exams, 6 teachers used multiple choice questions and 1 teacher who prepared mixed exams stated that they determined the type of question preferred by central exam system. One of these teachers expressed their opinions as follows: Five of the teachers stated that the ease of preparing and reading had an effect in choosing question types. In this respect, the teachers mentioned this subject:

T3W: "The ideal one is that it is mixed. I'll only ask mixed test questions on the second exam. On the first exam because it's easier to analyze the test questions."
The reason behind the types of questions that they use in their exams is found in the 5 teachers who have successfully carried out a measurement and evaluation: T1W: "The ideal is that the quiz is mixed and I use all types of questions because a good assessment is made by forcing the student to think multidimensionally."  In this section where social studies teachers created exams to encourage creative thinking skills, it is determined that the most open-ended questions (11.3%) were prepared for creative thinking skills, ignoring other categories of the question types used by teachers.

Why is it that the Urartians give the tombs a stone room and that the person who is buried together with their daily belongings?
This question, which requires the student to predict and solve the problem with the data given in the question, to establish a relationship between the data and to unify the data and create a logical reason, is a question about creative thinking skills.
It was found that the second type of questions used by the teachers for creative thinking skills on their exams were multiple-choice questions (9.9%).

Example:
With the discovery of compasses, new continents were discovered and the discovery of new continents revealed new agricultural products. Which of the following cannot be said?

a) An event can cause another event. b) An event may have different consequences. c) Each event will have positive results. d) Events with negative consequences are not considered as events.
This is a question which is prepared for the creative thinking skill because the student should analyze the given parts one by one and determine the effect of the parts on a whole and must make a new inference based on the parts.
It was found that none of the types of matching, true / false and short answer questions were questions about creative thinking. 20% of the questions under the other category were found to be questions about creative thinking skills. This method requires creative thinking skills because the student is expected to present an original product by using the data given. As can be seen, the most open-ended questions asked by teachers are questions about creative thinking skills.
3.4 Findings related to the fourth sub-problem Figure 5. The situation of questions in class levels encouraging creative thinking skills Creative thinking problems are very close together at all class levels (6-8%), but the most questions related to the creative thinking skills were found to be prepared in grade 7 (8%).

Figure 6. Distribution of questions according to topics in class levels
In this graph which examines the exam questions prepared by Social Studies teachers according to the subjects, it is seen that teachers prepare the most questions in 5 th grade about citizenship education. In 6 th and 7 th grades the most questions are about geography. It is determined that the percentage of questions about citizenship has decreased gradually from 5 th to 7 th grade. 8 th grade curriculum is the history of Atatürkism and Revolution in the Turkish Republic; all of the questions are prepared on the topics about history.  According to the subjects, 11.3% of the questions (except the 8 th grade) which are prepared for creative thinking skills are prepared in history and 7% of them are prepared for the creative thinking skills in geography. It was determined that 3.2% of the questions about citizenship issues, including the disciplines of social studies such as law, politics and economy, were prepared for creative thinking skills.

Discussion
In this section, the results obtained from the findings are discussed by making comparisons and interpretations under four headings, respectively, by considering the subproblems of the research.

Discussion on the first sub-problem
An important part of the educational program which is practical face of education system is measurement and evaluation. As such, measurement and evaluation help students to reach the goals and to ensure that the program is questioned, as well as helping students develop many high-level thinking skills such as creative thinking skills when asked the right questions (Secolsky & Denison, 2017;Tekin, 1993). Ozdemir (2006) stated that one of the most preferred ways for teachers to develop their thinking skills is in their questions. Exactly at this point, teachers have an important job. Qualitative questions prepared by teachers can provide important contributions to creative thinking skills of students (Kadir & Satriawati, 2017;Ozmen & Karamustafaoglu, 2006;Ozden, 2014).
In this study, it was determined that only 6.8% of the exam questions prepared by Social Studies teachers were directed towards creative thinking skills. According to these results, it can be said that the effects of the questions of teachers on creative thinking skills are insufficient. The reason for this result is that teachers do not pay attention to the exam questions, and teachers may neglect the creative thinking skills of students.
In parallel with this result of the research; Akbulut (1999), Akpinar and Ergin (2006), Beskisiz (2009), Kilic (2010, Kilinc (2014) Koray et al. (2005), and Sanli & Pinar (2017) analyzed the questions that teachers used during their lectures and written exams and found that the questions were generally aimed at remembering from sub-level thinking skills. Considering the It has been concluded that the subjects that the teachers take most attention in preparing the exam questions are learning outcomes, student levels and content validity. Titrek (2005),  and Tokcan & Cevik (2013) stated that the questions the teachers used in the exams were not valid and the teachers' exams did not ask any questions about some achievements while concentrating on others. The reason for the difference in the results may be that the sample, study dates and methods are different.
It has been proven in many studies that there is a positive relationship between creative thinking skills and academic achievement (Kadir & Satriawati, 2017;Ozturk, 2007;Ozyurt, 2011;Reyhan Gozcu, 2018;. Therefore, teachers need to consider improving student levels as well as supervising student levels. Parents' pressure on administration and private schools to keep students' grades high may be behind teachers' decisions (Yilmaz, 2011).
Teachers who take creative thinking skills into consideration in their exams stated that they carried out most of the questions by using all kinds of questions and with open-ended questions. Of teachers who say they ask open-ended questions, teachers T2M and T5M have short answer questions instead of open-ended questions. It was found that a teacher who said that should consider creative thinking skills in exams by using different question types is not directed to creative thinking skills of any questions. These results from the comparison of interviews and exam papers clearly show that Social Studies teachers have significant deficiencies in both measurement and evaluation and in creative thinking skills (Titrek, 2005). Palandokenliler (2008), His study found Social Studies teachers lacked creativity. In his study, Kuyubaşoglu (2009) stated that teachers wanted in-service training about creative thinking.

Discussion on the second sub-problem
Another result reaching of the study is that in Social Studies the most used question types of teachers are multiple choice questions, true/false and fill-in-the-blank questions, respectively and least used questions are open-ended questions. Similar to the results of the study, Colak (2008), Kilic (2010), Tokcan and Cevik (2013) and Yilmaz (2017) found that teachers preferred multiple choice questions in their exams.
Teachers have argued that the use of multiple-choice questions is mostly due to the central examination system and the ease of preparation and evaluation. When the distribution of the question types according to class levels is examined, although the use rates were close to each other, the use of multiple-choice questions decreased slightly from grade fifth to seventh grade level. In Turkey, it is known that the transition to secondary education is carried out during all the years of all secondary education (SBS), although it is done in the last year of secondary education (Author2, 2011). This result proves that the central examination system affects teachers' choice of questions. Askar (2009) and Yalcinkaya (2009) also stated that the importance of the central examination system and central examination system by parents in their studies directly affected the type of question that the Social Studies teachers used in their exams. Parallel to this result of the study, Gelbal and Kelecioglu (2007),  and Tokcan and Cevik (2013) stated that the teachers were easily able to prepare factors for determining the question types at the beginning. In addition, in the 5 th grade, matching, blanks filling, true false and open-ended question types were used. The use of these question types decreased as the grade level increased. The reason for this result may be that true false, fill-in-the-blank and matching, depending on the student level, are more easily answered than other types of questions.

Discussion on the third sub-problem
Another result of this research is open-ended questions are mostly prepared for encouraging creative thinking skills. Open-ended questions are the least preferred questions. 11.3% of the open-ended questions examined in the study were found to be prepared to develop creative thinking skills (Kilic, 2010).
When we examine the situation of the other question types used by the teachers in terms of creative thinking skills, 9,9% of the multiple-choice questions were found to be about creative thinking skills. It was determined that no questions regarding creative thinking skills were asked in matching, true false and short answer question types. Similar to these results, Colak (2008), examined high school history teachers' exam questions in terms of Bloom taxonomy, finding that open-ended questions were least preferred.

Discussion on the fourth sub-problem
As a result of the study, it was found that the questions about the most creative thinking skills (8%) were prepared at the 7 th grade level in the exams prepared by the Social Studies teachers. Although there is an increase in this ratio according to grade level, there is no significant difference. This result may be due to the fact that the teachers do not consider the mental development of the students in the preparation of the exam questions and the grade levels in the quality of the questions.
In the exam papers examined, most questions focused on creative thinking skills in grades 5, 6 and 7 were prepared on the topics of history. It is known that history subjects are composed of experiences and more concrete content than geography and citizenship education, there are various examples in their content and it allows the student to remove the boundaries in front of their thoughts, and it facilitates the development of thinking skills (Demircioglu, 2009;Koksal, 2002;Safran, 2006).
As a result, it has been determined that teachers have significant deficiencies in measurement and evaluation of creative thinking skills. It was revealed that teachers ignored creative thinking skills during the preparation of the exam questions and were negatively affected by external factors such as student scores and central exam systems. Although the least preferred type of question in the exam questions examined is open-ended questions, it was found that the most questions involving creative thinking skills were prepared in open-ended question type. It was determined that teachers prepared the questions about creative thinking skills in the subjects of history lesson at all grade levels and prepared them at the 7 th grade level.

Suggestions
• The importance of creative thinking skills to the Social Studies teachers should be explained by the seminars and in-service courses organized by the Ministry of National Education and teachers should be forced to prepare creative questions by examining the exam questions of the teachers more detailed.
• Seminars and in-service courses should be organized by the Ministry of National Education on the issue of academic literacy so that the Social Studies teachers will be able to ask questions and how to use the assessment and evaluation methods, and to follow current developments.
• It should be ensured that academicians and Social Studies teachers are brought together to exchange ideas on assessment and evaluation and to raise teachers' awareness.
• Teacher candidates should be trained in a more qualified way in terms of creative thinking and questioning in university faculties.

Suggestions for future research
• In this research, only the questions that the teachers used in the exams were examined. In addition to the exam questions, questions asked by the teachers during the lesson can be examined.
• In this study, 2065 questions were examined, and a semi-structured interview was conducted with 16 teachers. More detailed results can be achieved by increasing these numbers.
• In this study, questions are analyzed at the 5, 6, 7 and 8th grade levels in terms of history, geography and citizenship. A single class and a single discipline or sub-disciplines can be used to achieve more in-depth findings.
• In addition to examining the exam questions in a new study, teachers and students creative thinking skills can be determined and prepared with various scales.