**Said Hadjerrouit, University of Agder, Kristiansand, Norway**

*(This research was supported by a MatRIC Small Research Grant, project number 150401)*

### 1. INTRODUCTION

This project aims at evaluating the suitability of SimReal+ to support the learning of mathematics in teacher education. SimReal+ is a visualization tool that has been developed to teach a wide range of mathematical topics in various domains such as computer science, engineering and physics, but the suitability of the tool has not yet been fully evaluated for use in other educational contexts. The basic idea of SimReal+ is that visualizations are powerful mechanisms for learning mathematics, exploring, and explaining difficult topics. According to Arcavi (2003), visualization is the ability to use and reflect upon pictures, graphs, animations, images, and diagrams on paper or with digital tools with the purpose of communicating information, thinking about and advancing understandings. There is a huge interest in visualization in mathematics education, but there is little research on visualizations in mathematical education (Macnab, Phillips, & Norris, 2012; McKenzie, & Clements, 2014; Presmeg, 2014). Research aimed at exploring the potentialities of visualizations provided by digital tools for learning mathematics in a teacher education context should focus on technical and pedagogical usability issues in terms ease-of-use, motivation, variation, learning autonomy, and individualization. Moreover, SimReal+ in teacher education must be mathematically sound, and help students gain knowledge that is otherwise difficult to acquire. Furthermore, assessment issues in terms of feedback must be considered to support the learning process. Hence, the adaptation of SimReal+ to a teacher education context is a complex topic that needs to be researched and evaluated by means of criteria that capture technical, pedagogical, mathematical, assessment, and curricular issues.

### 2. THEORETICAL BACKGROUND

The theoretical background of this project draws upon two main bodies of knowledge: Didactical transposition and didactical functionalities of digital tools. The concept of didactical transposition goes back to Chevallard (1991), and it concerns the anthropological theory of didactics from its beginnings in the early eighties because of the need to model the widening gap between "academic" mathematics and school mathematics (Winsløw, 2011, p. 121). This theory can be applied to transpose the mathematics provided by SimReal+ to a teacher education environment. After the first study in 2014 (see MatRIC research report), SimReal+ has been redesigned to implement students' recommendations in terms of better user interface, types of mathematical tasks, and pedagogical issues. The result of the adaptation was a modified user interface and focused mathematical tasks following a logical sequence from lower to higher level of mathematical knowledge.

The notion of didactical functionality (Cerulli et al, 2005 cited in Artigue & Cirulli, 2008) describes three different dimensions to be taken into account when considering a digital tool for design or evaluation purposes: A set of characteristics of the digital tool, educational goals, and the modalities of use of the tool to reach such goals. Hadjerrouit (2015) added two other dimensions for the purpose of evaluation: Assessment issues and teacher education context. Identifying and distinguishing these dimensions helps to define appropriate criteria to evaluate the didactical functionalities of SimReal+. These can be divided into five key criteria: Technical usability, pedagogical usability, assessment, mathematical content, and teacher education issues (Bokhove, & Drijvers, 2010; Hadjerrouit, & Bronner, 2014; Hadjerrouit, 2015; Nokelainen, 2006). These criteria are not independent of each other. The technical and mathematical features of SimReal+ can only become functionally meaningful when understood in relation to the pedagogical use of the tool to teach a specific mathematical topic and the assessment of students' learning.

The first criterion refers to technical usability, which measures the extent to which SimReal+ is easy to use, user friendly, and intuitive, and whether the tool has improved display and management facilities such as recording students' activities (Nielsen, 1993). The second criterion extends technical usability beyond technical features and addresses pedagogical issues (Nokelainen, 2006). It describes properties of tools that act as facilitators of teaching and learning such as learner control, student autonomy, collaboration, variation, motivation, differentiation, and individualisation. The third criterion focuses on assessment of learning. It lays the ground for student profiles and tool's feedback to students' actions, and statistical analysis of students' activities. The fourth criterion is the mathematical content provided by the tool in terms of mathematical accuracy and representation of mathematical properties and operations, e.g., formulas, functions, graphs, and geometrical figures, and congruence between the tool's mathematics and ideal mathematics with paper-pencil techniques (Artigue, et al, 2009; Bokhove, & Drijvers, 2010). Finally, SimReal+ has to be evaluated to see whether it works in teacher education and under what conditions, and whether it meets the requirement of adapted education and enables the teacher to concretize the mathematics subject curriculum in teacher education.

### 3. METHODS

Twenty-two students were involved in the project in the autumn semester 2015. The students had different knowledge background both in mathematics and digital literacy. None of the students had prior experience with SimReal+. The work used a survey questionnaire with open-ended questions to collect empirical data. Teaching activities over a period of two weeks focused on basic, elementary and advanced mathematics. Basic mathematics included mostly games, such as dice, tower of Hanoi, and similar tasks. Elementary mathematics consisted of arithmetic and algebraic exercises. Advanced mathematics included measurement exercises, trigonometry, conic section, use of parameters, differentiation, and Fourier analysis. The teaching activities included simulations and visualizations of basic, elementary, and advanced mathematics using SimReal+, and additional online teaching material. To evaluate students' perceptions of SimReal+, a survey questionnaire based on a five-point Likert scale from 1 to 5 was used, where 1 was coded as the highest and 5 as the lowest (1="Strongly Agree"; 2 = "Agree"; 3 = "Neither Agree or Disagree"; 4 = "Disagree"; 5= "Strongly Disagree"). The average result (Mean) and Standard Deviation were calculated. The survey included 70 statements that were distributed as follows: Technical usability (12), pedagogical usability (22), mathematical issues (14), assessment (12), and teacher education (10). The students were asked to comment each of the statements in their own words. In addition, the students were required to address 10 open-ended questions to express in their own words what they think about specific issues of SimReal+.

### 4. RESULTS

The sample size (*n=22*) of this project is rather small to adequately support the generalization of the results. However, since refinements of the project have been made through two successive cycles of experimentations, where the shortcomings of each cycle were identified, re-implemented, and re-evaluated, it can be asserted that the results have more validity and reliability. Therefore, despite the small sample size, it is possible to make some reasonable interpretations of the results and draw some recommendations for improving the didactical functionalities of SimReal+ in teacher education. Given these considerations, the results are described for the five broad categories that emerged from the evaluation criteria, and supplementary issues that were addressed by open-ended questions.

Firstly, the results show that SimReal+ is technically well designed in terms of accessibility and management facilities, but despite the recommendations from the first study, the tool is still not sufficiently user-friendly, and it has not improved very much in terms of ease-of-use, user-friendliness, and recording of students' work. Hence, in terms of technical usability, the user interface needs to be simplified to make SimReal+ more intuitive and easy to use in teacher education. A well designed user interface is a requirement for any digital tool in mathematics education. It helps to reduce the cognitive load and effort resulting from the interaction with the tool in order to free more resources for the learning process. The following students' comments describe well the importance of a user-friendly interface:

*I think SimReal+ does not have a visually very attractive menu and structure (…).*

*It is difficult to navigate through the menu. Too much text (…). I assume the design part of the program has not been the main focus, but rather the content.*

*The excessive number of buttons can make the program hard to use the first time.*

Secondly, from a mathematical point of view, SimReal+ has improved following the recommendations from the first study in 2014 in terms of mathematical quality, since more students agreed that the tool has a high degree of mathematical content in terms of accuracy and representation of mathematical properties and operations. Likewise, the issue of practical applications and exercises obtained a good result in terms of design quality, and that the advanced exercises were not difficult to understand for most students. Furthermore, the students believed that SimReal+ covers a wide range of mathematical content with varied levels of difficulty, including advanced and basic mathematics. Some of the students' comments describe well the characteristics and features of the mathematical content provided by SimReal+:

*SimReal+ presents the mathematics in a thorough and principally correct manner. *

*I think SimReal+ has a high quality of mathematical content, but more can be done, for example to solve an equation of second degree, (…), or to do more logical games, (...), or exercises from number theory. *

*Many tasks lack explanation or are not well-formulated, but once one understands what should be done, the exercises show a high degree of quality that promotes knowledge acquisition. *

*SimReal+ is absolutely essential when it combines video lessons, simulations, live streaming of lessons, and exercises. It can help students to understand and see the connections between different themes of mathematics.*

Thirdly, from a pedagogical usability point of view, SimReal+ seems to provide motivating activities, variation in teaching, and many ways of representing mathematical tasks. More specifically, students think that SimReal+ provides various mathematical activities, multiple representation of mathematical content, and that SimReal+ can be used as a lecture and textbook supplement. Most students also agreed that SimReal+ is fully appropriate to use as an alternative to achieve variation in teaching mathematics, while enabling students to work at their own pace, which is a motivational factor in keeping students engaged in mathematics. Nevertheless, some students still think that SimReal+ has not improved much in terms of individualization and choice of level of difficulty, perhaps because some mathematical tasks are not differentiated to suit students' different knowledge levels. Furthermore, the tool does not easily allow students to customize the tool. Most participants also still think that SimReal+ does not fully allow students to work independently. Finally, the vast majority of the students did not find deliberate collaborative tools or communication forms. Some representative comments on these issues are:

* It is a great tool that can contribute to teaching and amplify the learning outcome and the comprehension of notions, for example in the case of graphs of functions and derivatives. *

*Variation is the best part of SimReal+. It offers a lot of opportunities to work with visualisations and animations.*

*I think that SimReal+ needs even more content to really be applicable as a differentiable learning tool.*

*Simulations are highly stimulating and pose a problem in a much more exciting way than verbal description would do.*

*There exist different types of exercises that have diverse level of difficulty (….). But, when it comes to each category specifically there is not really a variety of scaled-difficulty exercises.*

Furthermore, SimReal+ does not provide assessment capabilities. The tool neither has a review mode showing what the student has done wrong or right, nor allows for the use of several question types. Furthermore, SimReal+ neither provides a diagnosis of students' problem solving nor appropriate feedback that is adapted to the students' knowledge level as these comments reveal:

*As far as I can see, the program does not take into account the profile of the user. There are no demands to have completed a level before you can move on to more advanced subjects. This requires the student to be capable of assessing his/her own level and learning path.*

*Feedback in each attempt must be ameliorated. Where one fails to provide a correct answer, SimReal+ could "gauge" the nature of it and provide the proper feedback/help. *

*SimReal+ does not take the profile of the student into account, and serve up appropriate questions. You have to look by yourself which questions are appropriate for your level of knowledge.*

Additional analysis of the project results show that SimReal+ provides opportunities for teaching and learning mathematics in teacher education. Most students think that SimReal+ could be an appropriate tool for secondary school teachers, but to lesser degree for middle and primary school teachers. SimReal+ also enables teachers to concretize the mathematics subject curriculum. However, when asked whether they will continue using SimReal+ for teaching mathematics, most students answered negatively, but globally better in terms of average scores than those achieved in the study that was carried out in 2014.

Finally, students gave the following comments to specific aspects of SimReal+ such as flipped classroom approaches, programming, and designing animations. Most students like the idea of flipped classroom approaches, because they think that out-of-class activities and use of videos *"will give better prepared students and better discussions in classroom"*. Most students are also interested in programming mathematical animations and visualisations directly into a Web site. They think that using different templates can help them to concentrate on the mathematical part of the task, and *"investigate empirically properties of mathematical structures that lie behind the visual representations", "help to explore mathematics realities in ways pen and paper can't"*, and *"force to think through how the concepts really are"*. Most students like the idea of using templates, as this comment clearly indicates: *"Would be a great idea to have easy access in such a thing"*.

### 5. CONCLUSIONS AND RECOMMENDATIONS

SimReal+ has been developed, in the first instance, to support the visualization of mathematics taught at the university level. The following recommendations are therefore not criticisms of this aspect of SimReal+, but rather suggestions for development if the tool is to be used effectively within teacher education. Basically, SimReal+ in its present form can be used to teach and learn mathematics but not on a regular basis, because it is still not fully appropriate for use in teacher education, unless the technical usability is improved to make the user interface more intuitive and easy to use. Likewise, there is a need for more pedagogical functionalities in terms of differentiation, individualisation, assessment, feedback, and design of mathematical animations. The most frequent recommendations expressed by the students to make SimReal+ more appropriate for use in teacher education are:

- Better user interface for different type of users
- Assessment modalities
- More help and feedback
- Possibility to design mathematical animations
- Differentiation and individualization

### REFERENCES

Arcavi, A. (2003). The Role of Visual Representations in the Learning of Mathematics. *Educational Studies in Mathematics, 52*(3), pp. 215-241.

Artigue, Cerulli, M. (2008). Connecting Theoretical Frameworks: The Telma Perspective. *PME 32 PME-NA XXX Joint meeting of the International Group and the North American Chapter of Psychology of Mathematics,* 2008, Morelia, Mexico.

Artigue, M., et al (2009). Connecting and Integrating Theoretical Frames: The TELMA Contribution. *International Journal of Computers for Mathematical Learning 14*, pp. 217-240.

Bokhove, K., & Drijvers, P. (2010). Digital Tools for Algebra Education: Criteria and Evaluation.* International Journal of Mathematics Learning, 15*, pp. 45-62.

Chevallard, Y. (1985). *La transposition didactique - Du savoir savant au savoir enseigné*. La Pensée Sauvage, Grenoble (126 p.). Deuxième édition augmentée 1991.

Drijvers, et al. (2010). Integrating Technology into Mathematics Education: Theoretical Perspectives. In C. Hoyles, & J.-B. Lagrange (Eds.). *Mathematics and Technology-Rethinking the Terrain*. Berlin: Springer.

Hadjerrouit, S. (2010). A Conceptual Framework for Using and Evaluating Web-Based Learning Resources in School Education. *Journal of Information Technology Education 9*, pp. 54-79.

Hadjerrouit, S. (2015). Evaluating the Interactive Learning Tool SimReal+ for Visualizing and Simulating Mathematical Concepts. *Proceedings of the 12th International Conference on Cognition and Exploratory Learning in Digital Age (CELDA 2015)*, pp. 101-108.

Hadjerrouit, S., & Bronner, A. (2014). An Instrument for Assessing the Educational Value of Aplusix (a+x) for Learning School Algebra. In M. Searson & M. Ochoa (Eds.). *Proceedings of Society for Information Technology & Teacher Education International Conference 2014*, pp. 2241-2248. Chesapeake, VA: AACE.

McKenzie, K., & Clements, A. (2014). Fifty Years of Thinking about Visualization and Visualizing in Mathematics Education: A Historical Overview. In M.N. Fried, & T. Dreyfus (Eds.). *Mathematics & Mathematics Education: Searching for Common Ground, Advances in Mathematics Education*, pp. 177-192. Berlin: Springer.

Macnab, J.S., Phillips, L.M., & Norris, S.P. (2012). Visualizations and Visualization in Mathematics Education. In S.P. Norris (Ed.). *Reading for Evidence and Interpreting Visualizations in Mathematics and Science Education*, pp. 103–122. Rotterdam: Sense Publishers.

Nielsen, J. (1993). *Usability Engineering*. Boston, MA: Academic Press.

Nokelainen, P. (2006). An Empirical Assessment of Pedagogical Usability Criteria for Digital Learning Material with Elementary School Students. *Educational Technology & Society, 9*(2), pp. 178-197.

Presmeg, N. (2014). Visualization and Learning in Mathematics Education. In S. Lerman (Ed.). *Encyclopedia of Mathematics Education*, pp. 636-640. Berlin: Springer.

Winsløw, C. (2011). Anthropological Theory of Didactic Phenomena: Some Examples and Principles of its Use in the Study of Mathematics Education. In M. Bosch, J. Gascón, A. Ruiz Olarría, M. Artaud, A. Bronner, Y. Chevallard, G. Cirade, C. Ladage, & M. Larguier (Eds.). *Un panorama de TAD. CRM Documents 10*, (pp. 117-138). Bellaterra (Barcelona): Centre de Recerca Matemàtica.