INTRODUCTION

Today's world, which is changing in every moment, needs people who can adapt themselves to the new conditions. Mal Piryani et al. (2018) stated that the evaluation of teachers in academic training program can prepare the opportunity of determine of knowledge and teaching skills of them. In classroom, students need a lot of things to be successful in learning math. A mathematics teacher has to prepare their students to learn meaningful mathematical concepts in real life (Vara Nina Yulian., & Wahyudin, 2018). The teachers have to motivate and enact the student’s capacities so that students motivate to learn concepts meaningfully (Qamar Masood, & Abdul Wahid Usmani, 2015).  In this complex and changing world, education faces the increasing growth in knowledge, information and new technologies. Th teachers play key role in student’s academic achievement and all students follow to active teachers (Smith, 2019). If teachers do not prepare themselves for these conditions, they will not be able to continue their activities and if they continue their activities without preparation, they will harm both themselves and their students. Benítez et al. (2017) argued that the performance of students is understood as the observable act that depends on the performances of teacher. The evaluation of teachers estimates their capacities and performances in terms of quantity and quality of teaching. Problem-based learning in mathematics is an educational strategy that diverges from traditional teaching strategies in mathematics by turning a teacher-centered learning environment into a student-centered environment in which students are independent and active. Teachers face difficulties in teaching problem-solving methods to the students and students face difficulties in learning them. Some of these problems occur because 'finding the answer' is considered as the ultimate goal of this process.
In-service training is vital for a number of reasons; the people who are selected for a specific job usually need to properly learn the required skills and knowledge for that job position. This helps them to know about the organization's process, the content of the job and its awareness and importance; it also helps the individuals to increase their interest towards the job and the organization. Therefore, not only education is appropriate for teachers, but also it is important for the education organization; since it leads to the mutual growth of the teacher and the education organization . Officially, educators and curriculums are often evaluated, but this method has a flaw; this evaluation often involves the evaluation of the colleagues' reaction and feedback towards the education, which is usually done in the last day of training. To improve education, it is necessary to gain more information about the educational impacts and the quality of the provided training. Therefore, education and its formative assessment will certainly help the education organization to be at the center of achieving goals (Borate et al., 2014). In this study, the levels of Kirkpatrick's model are used to evaluate the extent of effectiveness of the problem-based training for mathematic teachers. One of the commonly used models to determine the place of effectiveness evaluation in employee training is Kirkpatrick's evaluation model. This model seeks to evaluate the effects of curriculum and in this regard, identifies different levels for that. A comprehensive evaluation can inform us of the effectiveness of the training results; this evaluation provides a feedback through which we can find if the trainings had been effective in achieving the desired goals or not. In the stages of needs assessment, planning, implementation and monitoring of all training courses, evaluation is one of the most important stages in educational planning, that its correct conduction, provides very useful information on how to design and implement educational programs (Bakhtiari & Kiani Manesh, 2014). In many countries, including Iran, although the aim of the problem-based learning based on Polya & Shoenfield's problem solving methods, is to prepare students for lifelong learning, this educational approach is still less widely used. Studies have shown that maintaining the acquired knowledge in students who have been trained through problem solving methods is longer than other methods and can provide conditions for the lifelong learning, and this is one of the goals of problem-based learning (Woolfolk, 2016).
A study by Waqas Raja & Wei (2014) on the variables contributing to the effectiveness of teachers training program, aimed at training computer technology and using information for the education sector in the context of Pakistan, revealed that the learners had acquired the computer skills and knowledge at the end of this training program, however, doing rigorous training need analysis, could have increase their effectiveness. It was also found that factors including availability of physical facilities in the colleges, support from the head of departments, have impact on the effectiveness of trainings. In a study entitled 'Applying the Kirkpatrick model: Evaluating Interaction for Learning Framework curriculum interventions', (Paull et al. 2016), indicated that the four-level model of Kirkpatrick, as a way of evaluating the use of interaction for learning framework has provided positive results in a time efficient way, for both students and the educators. It is recommended to evaluate the use of Kirkpatrick model continuously, since the fit with the innovations of other curriculums is not fully recognized. The aim of this study is to investigate on the four levels of Kirkpatrick's model after holding the problem-based training courses, among mathematic teachers. Therefore, these questions rise: Does problem-based learning (based on Polya & Schoenfeld's problem solving theories) in mathematics have any effect on the mathematic teachers' level of learning? Does level of learning have any effect on the mathematic teachers' level of behavior, after holding the problem-based training in mathematics? Does level of behavior have any effect on the mathematic teachers' level of organizational results, after holding the problem-based training in mathematics?
Polya's problem solving method
Polya (1957; 1968; 1978) encourages students and mathematics teachers to 'guess and try' and 'first guess, then prove'. He explains the art of guessing in this way: "Our first guess might be wrong, but we try it, and change it more or less with regard to the degree of success; finally, after several attempts and several changes, we may find a more satisfying guess by observing and using analogy"(Polya, 1968, p. 158). Polya states that: "A certain kind of guess" (Polya, 1957, p. 99) should be taken seriously, because such guesses, based on the consideration of the features of the problems, usually include "part of the reality" correct answer "), and if someone tests it in the right way, it might lead the person to the whole reality (the entirely correct answer). He believes that what is really bad is "not having no idea at all " (Same Source, p. 99). Polya introduces the 'Modern heuristic' as a tool to understand the problem-solving process as a guided mental operation by both logic and psychology (Polya, 1957, p. 129). He seeks to explain how to improve students' mathematics learning through available activities in the problem-solving process.
Schoenfeld's problem solving method
According to Schoenfeld (1985), the factors and constituents that are found in problem-solving strategies and represent the ideal solution to the problem are as follows : Analysis: The problem begins with the analysis of what the problem really wants. This stands for touching the problem: what is it considered for and what does it want (goals). An appropriate application of the approach-based strategies at this stage of problem solving is as follows: a) plotting a graph even when the problem can be answered through different types of reasoning (i.e., an algebraic one); b) Simplifying the problem (examining special cases) by a result, so that we can both solve it for specific cases and empirically observing the diagnostic patterns; c) Looking for introductory simplification . Design: In one sense, control is superior. It really is the case that penetrates into the full problem-solving process; its role ensures that you engage in activities that are beneficial. More generally, this case stands for maintaining a general view of what takes place and keeps going on constantly. Exploration: It is the center and focus of the approach-based strategy; for it is at the exploration stage that the majority of problem-solving approaches are displayed . Implementation: It needs a short interpretation, except that it (usually) should be the final stage in a real problem-solving process. On a general level you can knowingly find out the useful aspects of a problem's solution that you can apply; and it can help you in achieving a good solution.
Kirkpatrick Model

METODOLOGY AND METHODS

To study the present research situation, a method called quasi-experimental research method was used. Therefore, no intervention was used in the control group after the randomly division of the sample into the control and experimental groups, However, in the experimental group, it was tried to use the problem-based learning method for the mathematics teachers. Before the intervention, a pre-test was administered to the two groups and after intervention, a posttest was administered to the two groups. It should be noted that in a paper compiled by researchers, the two groups of control and experimental were investigated in terms of the mathematics teachers' performance. This study only investigates the status of the Kirkpatrick model in the experimental group after the use of a new problem-based intervention. In this study, the in-service training effectiveness assessment questionnaire was used, which was based on the Kirkpatrick Evaluation Model and included 33 items, through which the participants identified their ideas and thoughts on the topic by choosing one of the five options that were given for each item of the questionnaire. The items in the questionnaire were at four levels of Kirkpatrick model. The content validity of the Kirkpatrick model-based questionnaire was investigated and approved according to the opinions of the experts in the fields of mathematics education and organizational psychology. On the other hand, the validity of the questionnaire was confirmed by Cronbach's Alpha values ​​above 0.70.
Participants
The population in this study includes all mathematics teachers in Rabat Karim city in Iran, who engaged in teaching math in the first grade of secondary school during the academic year 2016-2017.100 teachers were selected through simple random sampling method (38 were assigned to the control group and 62 were assigned to the experimental group1) and were trained. The participants were working at girls' and boys' secondary schools in Rabat Karim during the academic year of 2016-2017. The selected teachers had one term experience of teaching mathematics in the ninth and seventh grades of secondary school . 48% of mathematics teachers under the study were female and 52% of them were male. 81% of the mathematics teachers under the study belonged to the under-25 age group. The teaching experience of 75% of mathematics teachers under the study, were less than 5 years. Considering the educational degree, 80% of the mathematics teachers under the study had bachelor's degree and 20% had master's degree.
Implementation of the problem-based learning method for mathematics teachers

FINDINGS

In this section, the statistical findings obtained from the data analysis in R software and the packages of ggplot 2, lavaan and semPlot from R software are shown as below. To investigate the research questions, the structural equation modeling was used. According to the questions related to each factor and according to the main questionnaire of this scale, we name the factors as it is shown in Table 1.

Table 1: Guide for naming the variables in this text

This conceptual model is represented in the path model in Figure 1. According to the values of goodness of fit indicators, this model is in a good and at least acceptable level of fitting in terms of all indicators of goodness of fit and based on the research data in this study. The only indicator which shows that this model is not at a good fitting level is the p-value of the Chi-Square test, which can be due to the large size of the sample.

Figure 1: Conceptual model of research

In Figure 2, we see the conceptual model of research with standardized factor loadings, and in Table 2, we see the goodness of fit indices.

Figure 2: The first conceptual model of research, implemented with standardized factor loadings

In Table 3, the non-standardized factor loadings are represented along with their levels of significance.

Table 2: Goodness of fit indices for the conceptual model of research

Table 3: Significance evaluation of path coefficients related to the direct effects

According to the p-value obtained from the t-test for significance of each of these paths, the effects of these paths are significant (p <0.01); and according to the sign of coefficients for these paths, it indicates the positive effect of each level of the model on its next level.

DISCUSSION

This study aimed at providing a basis for examining the efficiency and effectiveness of in-service training course with the problem-solving approach on the first-grade high school math teachers, which was not evaluated with a comprehensive and prominent model so far. In this study, in addition to the common methods of evaluating in-service courses which have been limited to surveying the participants and using questionnaires, based on the pattern of change in learning, the resulting behavior and outcomes of training the teachers were evaluated in their work environment and the areas which needed improvements were addressed in the study .One of the reasons for the importance of this study was planning to prevent wastage of organizational resources and interests by examining the quality of the provided trainings at different levels of evaluation; which means that according to the provided trainings, the evaluation levels and the conceptual research model , it turned out that if the teachers under training provide more responses,  the learning will takes place better;  and it will enhance the teacher's ability and skill, mastery, and performance, and it was found that the level of behavior has a positive impact on the teacher’s organizational results and the productivity resulting from his performance. However, due to the time constraints of training courses and the presence of influential factors in organizational definitions and the researcher's inability to make organizational decisions to create a competition and rank the teachers' activities, upgrading the conceptual model and turning it into an evaluation cycle did not become possible.

Figure 3: The Summary of Research

CONCLUSIONS

Mathematics teachers need different knowledge that provide them the necessary information about the students, and It is necessary for mathematics teachers to know how to implement problem solving method. Therefore, in this study, the attempt was taken to implement the in-service course in the form of a framework, by relying on the (Polya & Schoenfeld's) problem-solving method. The Kirkpatrick evaluation model was used to evaluate the efficiency of the in-service course for mathematics teachers. Using the standard questionnaire, the four levels of this model were examined on teachers, after the in-service course. The exact structural equation analysis showed that each level represents the positive effect of each level of the model on its next level . Since the problem-based approach opened the windows towards the new educational activities and open-mindedness for the teachers participated in the study, teachers could grasp a better understanding of a mathematical problem's outcome. This condition can be seen in the reaction level of Kirkpatrick's model. Moreover, the extent of learning and acquiring the new knowledge during the problem-solving process, by relying on the active problem-solving approaches based on Polya and Schoenfeld's methods, could raise the learning level of the new contents with a new problem-solving approach for teachers, which covers the second level of Kirkpatrick's model. Subsequently, according to the third level of Kirkpatrick's model, they changed the teachers' behavior towards the new mathematical concepts and their optimal ways of teaching. This change was evident in improving their performance. On the other hand, the last level, (i.e. the results level) in the Kirkpatrick model, was evident in this in-service training course. For instance, a reduction in costs for holding the inadequate in-service training courses which were not much effective, and increase educational quality after in-service course, were evident in the mathematics classes of these teachers, and the organizational goals of education were also met . Therefore, with regard to the better performance of the mathematics teachers and the effectiveness of each level of Kirkpatrick's levels, after the problem-based training course on some topics of the mathematics book, some suggestions for further research were presented. It is suggested that Kirkpatrick's levels be held and examined among mathematics teachers several times after an academic year to evaluate different in-service training courses for mathematics teachers with new methods and more accurate estimates of the level and the status of the mathematics teachers. It is suggested that the problem-solving method and in-service training courses be implemented and held in subjects such as geometry and algebra, based on Polya and Schoenfeld's frameworks, and then the outcomes of the training course be evaluated, according to the four levels of Kirkpatrick. Issues such as: the expertise, academic ability, and the skill of the teacher in choosing the best teaching methods, holding classes in an interesting and stimulating environment, formative assessment to ensure the effectiveness of the training courses, the active participation of the school principals in evaluating the effectiveness, Explaining learners about the goals and content before holding the training course, should be considered .Problem solving training is necessary to increase the learners' learning and academic achievements, therefore, teachers in the field of mathematics education should be more focused on using this approach. However, this should not be neglected that; teaching with a problem-solving approach is not prescribed in every situation and for every educational subject. In teaching with problem solving approach, there is a need for the experienced, flexible, and interested teachers with a high level of knowledge. Furthermore, this educational approach needs students and mathematics teachers to spend enough time and engage in problematic issues. Therefore, the requirements of this educational approach during the implementation of education, are very important.

REFERENCES

Bakhtiari, Y. & Kiyani Manesh, K. (2014). Educational Evaluation models with Emphasis on New and Practical Experiences. Tehran: Scientific Civilization Publications.

Benítez, J.E.M., Cabay, L.C.C. & Encalada, V.D.G. (2017). Formación inicial del docente de educación física y su desempeño professional, EmásF: Revista Digital de Educación Física, 8(48), 83-95.

Borate, N.S., Gopal Krishna, L. & Borate, S. (2014). A Case study for evaluation of Evaluation Training effectiveness and Development program. Proceeding of the Second International Conference on Global Business, Economics, Finance and Social Sciences (GB14 Conference) Chennai, India, 11-13 July 2014 Paper ID-C432. DOI: 10.13140/2.1.3407.4243

Bucur, M. (2015). A Study on Business Communication on Corporate Social Responsibility in Romania. Procardia Technology, 19, 996-1003.

Gotshall, L.S. (2005). Outcome engineering: status and effectiveness a cross parish nursing organizations. In partial fulfillment of the requirement for the degree of Doctor of Education, The University of West Florida.

Kirkpatrick, D.L. & Kirkpatrick, J.D. (2007). Impalement the four levels a practical guide for effective evaluation of training programs. Barrett-Koehler and the BK logo are registered trademarks of Barrett-Koehler Publishers. Inc.

Kirkpatrick, D.L. & Kirkpatrick, J.D. (2006). Evaluating Training Programs: the Four Levels; San Francisco: Barrett- Koehler Publishers. Inc.

Kong, J.Y. (2009). A comparison of the practices used by human resource development. A professional to evaluate web-based and classroom-based training programs within seven Korean companies' dissertation, The Ohio State University.   

Mal Piryani, R., Prasad Dhungana, G., Piryani, S. & Sharma Neupane, M.S. (2018). Evaluation of teachers training workshop at Kirkpatrick level 1 using retro–pre questionnaire, Advances in Medical Education and Practice, 9, 453–457. Adv Med Educ Pract. DOI: 10.2147/AMEP.S154166. eCollection 2018.

Moica, S. & Radulescu, E. (2014). Statistical Controls have a Significant Influence on Non Quality Costs. Cases Study in a Company those Manufacturing Aluminum Castings Components, Procardia Technology, 12, 489-493.

Moldovan, L. (2015). Sustainability Assessment Framework for VET Organizations. Sustainability, 7(6), 7156-7174.

Moldovan, L. (2016). Training Outcome Evaluation Model. Procardia Technology, 22, 1184 – 1190.

Paull, M., Whitsed, C. & Girardi, A. (2016). Applying the Kirkpatrick model: Evaluating Interaction for Learning Framework curriculum interventions. In Purveyors of fine learning since 1992.  proceedings of the 25th Annual Teaching Learning Forum, 28-29 January 2016. Perth: Curtin University.

Pinter, K. (2012). On Teaching Mathematical Problem-Solving and Problem Posing. Thesis in option to the scientific degree of doctor in Szeged Institution, Hungary.

Polya, G. (1957). How to solve it. Second edition. Princeton: Princeton university press.

Polya, G. (1968). Mathematics and plausible reasoning (Vol. 2): Patterns of plausible inference. Princeton: Princeton university press. (Original work published 1954).

Polya, G. (1978). Guessing and proving. The Two-Year College Mathematics Journal, 9(1), 21-27.

Qamar Masood, R. & Usmani, M.A. (2015). Evaluation of Medical Teacher’s Training Program Through Kilpatrick’s Model. KMUJ, 1(3), 76-80.

Schoenfeld, A. (1985). Mathematical problem solving. New York: Academic Press.

Smith, A. (2019). The Counternarrative of Teacher Evaluation: The Kangaroo Court, the Salem Witch Trials, and the Scarlett Letter, Educ. Sci.,9(147), pp.2-18.

Vara Nina Yulian, & Wahyudin. (2018). Analyzing categories of mathematical proficiency based on Kilpatrick opinion in junior high school. 3rd International Conference on Mathematical Sciences and Statistics, IOP Conf. Series: Journal of Physics: Conf. Series 1132 (2018) 012052.    DOI: 10.1088/1742-6596/1132/1/012052

Waqas Raja, M. & Wei, S. (2014). Evaluating the Effectiveness of Teachers Training Programs in Islamabad Model Collages, Journal of Studies in Education, 4(4), 68-79. DOI: 10.5296/jse.v4i4.6326

Datos del (los) autor (es)

¹Doctora en Psicología Educativa, Profesora investigadora y Decana de investigación en la Universidad Especializada de las Américas, Panamá. ORCID:  0000-0002-4958-4380

²Associate professor, Department of Mathematics, Science and Research Branch, Islamic Azad University, Tehran, Iran. Orcid: http://orcid.org/0000-0002-6394-2068 

³Assistant professor, Department of Mathematics, Arak Branch, Islamic Azad University, Iran. Orcid: http://orcid.org/0000-0002-2252-3844