Federal University of Maranhão - UFMA, Imperatriz, Maranhão, Brazil
University of Trás-os-Montes and Alto Douro - UTAD, Vila Real, Portugal
University of Trás-os-Montes and Alto Douro - UTAD, Vila Real, Portugal
Study of the adoption of non-traditional tools as support for Higher Education curricula in Electrical Engineering and Computers. We highlight the use of blended-learning, interactive and remote virtual laboratories, computer simulation, and methodologies, such as Active Learning and Problem Based Learning and their applications in the curricular units of the course. The study is a literature review with the systematization and presentation of the findings through a conceptual map. We concluded that the initiatives that have resorted to new technologies in engineering degrees, as well as reports of similar experiments on this topic, are reduced, not formalized in curricula, and ad hoc.
KEYWORDS: Conceptual Mapping; Engineering Education; Engineering & Computer Tools.
Several solutions have been developed with new technologies in order to contribute towards an improvement of the teaching-learning process in higher education and, according to Kusano et al. (2013, p.30) there are many benefits to using technology as an educational tool integrated into the classroom.
Specifically in engineering degrees (Benitti, 2012; Corter et al., 2011; Cortizo et al., 2010; Fabregas et al., 2011; Garrison et Kanuka, 2004; Méndez et González, 2010; Tiernan, 2010), these solutions have proved to enhance students’ performance and they will be the main topic of discussion in this study.
Several solutions were tested in these curricula. For instance, in the automation and robotics curricular units, by resorting to blended-learning (Jara et al., 2011), in the teaching of JAVA programming language (Alonso et al., 2009), in curricular units on introductory studies to programming by using active learning following the recommendations suggested by the Bologna Declaration (Fernandez et al., 2011), in mathematics tests (Macedo-Rouet et al., 2009), all of them presenting efficient results in terms of the students level of learning and performance, when compared to face-to-face teaching.
In the control engineering courses, literature has reported advancements in the use of remote laboratories, although there are still some pedagogical shortcomings that need to be overcome (Chevalier et al., 2017; Dormido et al., 2008; Gomes et Bogosyan, 2009; Stefanovic et al., 2011; Jara et al., 2009; Lazar et Carari, 2008).
Several methodologies, from other fields of expertise, have been implemented in engineering’s higher education teaching, namely, “Case Study”, “Grounded Theory”, “Action Research”, “Problem-Based Learning” and “Narrative Analysis” (Case et Light, 2011). In this list “the Systematic Literature Review” methodology is also included (Borrego et al., 2014). Some of the texts cited in this paper use some of these methodologies in engineering.
This paper seeks to identify the main tools used in the teaching of engineering, specifically in electrical and computer engineering, and the curricular units in which they are being implemented. It will also seek to identify the best current practices. The main internal motivation for this research is to contribute towards the improvement of the results obtained in engineering degrees offered in universities. The methodology used is based on a literature review together with the systematization and presentation of the findings through a conceptual mapping. This study may contribute towards the improvement of the teaching processes used in engineering.
Benitti’s (2012) study has made a systematic literature review of the scientific productions published between 2000 and 2009. The studies were selected from the use of the following keywords: “robotics”, “school” and their variations. The databases used were: IEEE Xplore; ACM Digital Library; ScienceDirect; SpringerLink; ERIC; and Wilson Education. After a categorized selection of 197 articles, Benitti filtered, selected and analysed the ten studies regarding the teaching processes that have resorted to experiments with robots. As a result, (Benitti, 2012, p.988), the following engineering courses were assessed: Computer programming, geospatial concepts, and engineering/robotics from the studies by Barker et Ansorge (2007), Nugent et al. (2010), and Nugent et al. (2016) and the course Systems and Computation (Sullivan, 2008). In these studies, involving non-traditional methods, Benitti (2012, p. 986) has identified the main skills developed in students and has cited them according to each author’s respective works: thinking skills; science processing skills (Sullivan’s, 2008) evaluation of solution, hypothesis generation, hypothesis testing and control of variables, problem-solving approaches, and teamwork (Nugent et al., 2010).
Tiernan (2010) has conducted an experiment with 91 students from industrial automation, for a period of four years, seeking through the incorporation of hardware linked programming to increase their interests in research. By observing and recording the students’ interest in a graphical-based computer language for programming control and data acquisitions, the LabVIEW software. During this study, the software was used to describe and simulate physical phenomena by using a mathematical model. The software allows for the creation of applications in a short amount of time without the need to have a deep knowledge of programming languages. The results have shown that the students’ experiences with LabVIEW with associated hardware have increased their interest and enthusiasm for the subject of industrial automation.
Garrison et Kanuka’s (2004) article opens this section because it offers a classical discussion on the transformative potential of BL in the context of the challenges of higher education. It presents the potential of BL as a support for learning and it discusses the need to rethink and restructure the learning experience through its transformative potential. Twelve years after it was written, its premise “The academic benefit, evidence, and competitive advantages are clear; only the will and commitment remains. Blended learning can begin the necessary process of redefining higher education institutions as being learning-centered and facilitating a higher learning experience” (Garrison et Kanuka, 2004, p.104) confirms that BL is consistent with the values of the traditional Higher Education institutions and it has the proven potential to significantly improve the effectiveness and efficiency of the learning experiences.
Cortizo et al.’s (2010) study presents an application of Mechanical Couplings (MC) developed for Mechanical Engineering students, by using the BL approach. This tool allows students to view simulations of coupling assembly, to access databases on the technical characteristics of the different types of couplings, to calculate and choose the correct coupling for a specific application of power drive between machine shafts and to perform self-evaluation examinations. Cortizo et al. (2010) have performed a detailed experimental analysis in order to quantify the existing learning differences between the traditional mode and through the use of BL. The results showcase that the use of the MC application, together with BL, has increased the level of knowledge of the students in the experimental group, who have achieved the highest average mark in the test. The results of the experiment have shown that the solution has reduced the level of difficulty when compared to the traditional mode (Cortizo et al., 2010, p.1018) and it has increased the level of knowledge of all the students (Cortizo et al., 2010, p.1018), thus it can be evaluated as a useful tool in the pedagogical teaching of mechanical engineering.
Méndez et González (2010) highlight the ample use of BL in the teaching of courses on process control in Electrical Engineering curricula. The study is carried out within this scenario and it includes a reactive element and a fuzzy logic based controller. This controller was designed to regulate each student’s workload, according to his/her activity and performance. The non-traditional methodology combines the traditional face-to-face classes with the on-line resources: Modular Object Oriented Development Learning Environment (Moodle), a Content Management System (CMS) and the ControlWeb simulator (Méndez et González, 2010, p.857). The pedagogical results of the use of this methodology, based on the students’ feedback, attests to its efficiency in terms of learning degree, satisfaction, motivation and performance in the course.
In engineering, undergraduate curricula, practical activities demand laboratory experiments. Laboratory experiments are fundamental towards the acquisition of the necessary skills in the specific courses, and these experiments may reinforce and deepen the conceptual understanding of the content.
An important review of the literature on virtual and remote labs, which includes publications from 1993 to 2015, was the object of a study by Heradio et al. (2016). An analysis of the various platforms and tools used for virtual and remote labs is presented in a state of the art in Potkonjak et al. (2016). Therefore, next studies by Corter et al. (2011), Fabregas et al. (2011) and Jara et al. (2011) will be presented.
When assessing whether these critical experiments can be effectively performed remotely or by computational simulation, in an experimental study carried out throughout many years, Corter et al. (2011) assessed the learning results through processes performed in three types of laboratories. The students performed hands-on operations, remote-based operations and operations based on practice simulators. The results suggest that the work with real data, instead of simulated data, may lead to higher levels of motivation. They also suggest that learning with computer mediated technologies can be improved through a careful design and coordination of individual and group activities (Corter et al., 2011, p. 2054). In summary, Corter et al.’s (2011) experiment has concluded that the new technologies applied in remote and simulated laboratories may effectively contribute towards the enrichment of the understanding of the teaching of conceptual engineering.
Fabregas et al.’s (2011) study describes, through a teacher’s orientation, how to transform a local laboratory into a remote interactive laboratory (RL) to be used in the teaching of courses in automatic control systems. A university in Spain has used two software tools: Simulink for the control area and Easy Java Simulations (EJS), an authoring tool that allows for the creation of interactive applications in Java without demanding special programming skills from students. The study highlights its intention of providing a pedagogical approach to teachers in order to facilitate the creation of remote laboratories. The reported results were pedagogically positive. Mainly, no statistical measures were used to specifically evaluate the performance and variance of the variables in the results.
Jara et al. (2011) have confirmed the improvements in the learning of robotics and automatics when the teaching in the classroom is supported by laboratories with adequate experiments. In order to minimize the high costs of equipment there are several low-cost and flexible solutions developed aimed to achieve a better cost-benefit relation and to allow for an effective teaching. Virtual and remote laboratories are part of this group of solutions. The study presents an experimental teaching based on a BL method by using a virtual and remote robotic laboratory, the RobUAlab, and another empirical assessment of its efficiency without BL. The RobUAlab offers a virtual environment that allows the student to perform experiments with a simulated robot arm, with the capacity to interact in its simulated workspace and, with tele-operational functions to execute the programmed task in a real robot, identical to the simulated one (Jara et al., 2011, p.2453). The students experiment in a set of exercises in the face-to-face classes and, later, they will access the experimentation environment to finish them remotely in the RobUAlab. The results from the evaluation of the proposed educational methodology attest to its efficiency in terms of the students learning and performance outcomes.
We begin this section by highlighting the aforementioned studies that were conducted throughout the undergraduate curricula of electrical and computer engineering: the study by Corter et al. (2011), on the different types of remote and simulated laboratories; and by Méndez & González (2010), which highlights the use of BL on the topics of process control. In the Computer Engineering curriculum, Sullivan (2008) performed an experiment with 26 students by using observational and experimental methods to measure the thinking skills and the science processing skills in solving problems in the robotics course. The pre/post test results revealed an increased understanding of the systems.
A recent study by He et al. (2015) has researched the impact of Problem-Based Learning (PBL) on the electrical engineering students’ conceptual understanding and compared it with the traditional face-to-face classes. The experiment was performed on 55 students. The participants completed pre/post tests on the topics covered in the study and a learning assessment research. The results suggest that the participants with a PBL learning got double the gain than that of a traditional learning.
Fernandez et al.’s (2011) experiment is highlighted in a pilot case-study in an introductory programming course in the undergraduate degree on Computer Engineering, based on an active learning strategy. The initiative tests the effect of the application of the principles of the Bologna Declaration in adopting teaching methods with student-centred methodologies. It is understood that the use of student-centred instructional strategies, such as active learning, is an excellent initiative with positive results. Nevertheless, the negative effects of these kinds of actions will be discussed in the following section.
In the Electrical and Computer Engineering curricula, the combination of methodologies and hybrid tools have been discussed and have presented favourable results, mainly, from the students’ motivational and performance point of view.
The use of the term “hybrid” has been typically associated to the combination of the online and face-to-face instruction components (Garrison et Kanuka, 2004). In Ward (2004) et Lindsay’s (2004) rhetoric, in studies with homonymous titles, they refer to this term as “the best of both worlds”. On the other hand, in a study performed on the discipline “Society and Technology”, based on time, Verkroost et al. (2008) note that there aren not clear and comparative proportions between online time and face-to-face time that allow for the definition of the BL, and its implementation might undergo several variations. Therefore, Verkroost et al. (2008) discuss the need for an effective balance in the use of BL in learning.
By performing an experiment in the introduction to programming courses in the Computer Engineering curriculum, Fernandez et al. (2011) discuss the difficulties that both the professors and the students have in adapting to the teaching methods set in the Bologna Declaration. The results have shown good outcomes for the students. However, it highlights that the teaching system in the engineering curricula is not fully ready to support the new instructional process and it may still undergo several adaptations.
In an experimental study by Bowen et al. (2014), performed with 605 students from the same course in six public universities, random trials were used. The results have shown that a hybrid instruction has not presented a statistically significant advantage when compared to a traditional instruction. Actually, students in the hybrid format have reported that they have spent 0.3 hours more per week than students in the traditional format. This difference means that the students in the hybrid format would spend about 18 percent more of their time studying than the students in the traditional format. The next section presents a conceptual mapping of the courses that used support tools in the traditional teaching of engineering curricula.
A hierarchy diagram was chosen to visually represent, in statements, the significant links between the discovered concepts. This is a conceptual map widely used in knowledge management studies and it can be understood as a two-dimensional diagram, whose main function is to display concepts hierarchically organized and the relations between them. The connection lines represent the relationship between concepts (Moon et al., 2011). The computer software used to create the conceptual map was Cmap Tools Knowledge Modelling Kit.
This is a methodological tool which uses assimilation theory to determine what students know or what they discovered during the literature review (Cañas et al., 2000, p. 1-2).
A recent study by Hagemans et al. (2013) describes the experience of students who visualized the conceptual map and overcame students who had only conducted a descriptive analysis. Results show that using conceptual maps helps to improve learning and to discover connections between concepts (Alonso et al., 2009; Arnab et al., 2015; Dias et al., 2015; Rawson et al., 2015; Wake et Dysthe, 2007).
Based on the findings discussed throughout this paper, a conceptual mapping of the non-traditional teaching processes applied to the teaching of engineering was created (Figure 1). Based on the mapping it is possible to identify the courses in which there were reports on the experiments and on the main methodologies used. The systematic vision offered by Figure 1 makes the analysis process of the scenario of new teaching processes in engineering more dynamic.
The eight main categories were mapped because they present positive results through the use of non-traditional teaching processes. It should be noted that the six main non-traditional teaching processes used were associated to the courses. The limited amount of courses found in the literature when compared to the total volume of courses in a curriculum program should also be noted.
Figure 1. Mapping of the non-traditional teaching processes applied in engineering
Source: The authors’ own
By analysing these reports on experiments applied in engineering, none of them mentioned that they were formally institutionalized processes. Therefore, it can be concluded that the outcomes from the presented experiments result from individual initiatives by the teachers of the courses, and not from teaching patterns discussed and institutionalized in engineering curricula pedagogical projects.
The conceptual mapping of the courses and teaching processes, has allowed us to present the findings in a systematic manner. Since the analysed literature has shown a tendency for individual initiatives reported in Electrical and Computer Engineering, we have chosen to nominally cite each unit reported in the map. Nevertheless, it is evident to the reader that the systematic vision of the mapping leads to a diagnosis in which there would be a possibility of grouping courses by affinity and/or similarity of contents. For example, it should be noted that the two first courses shown in Figure 1, Computer Programming and JAVA Programming, are both in the computer programming area, and there are two others that could be grouped since they belong to one great area: automation and control. In a way, these findings allow us to understand that the effective number of courses that are resorting to new technologies or new processes, based on what has been published internationally, has been a limiting factor in engineering.
The challenges to engineering curricula for the consolidation of these actions are still big, since it is clear that there are difficulties that transcend the classroom environment: the teachers need proper training and they need to “buy the idea” of changing from teaching-centred methods to student-centred methodologies; the courses and institutions need to institutionalize these methodologies in their pedagogical projects and bear the burden of their adaptation and implementation; it is also clear that most engineering curricula have laboratory limitations due to their high implementation and maintenance costs. Therefore, initiatives in the use of virtual or remote laboratories tend to emerge as a means of meeting the students’ needs with their limited resources and, therefore, they are not created with the sole purpose of improving teaching processes.
Adapting engineering curricula to the teaching methods set in the Bologna Declaration, places the responsibility of this adaptation on both the courses and the Higher Education Institutions. The adoption of student-centred active-learning methodologies is still challenging for the traditional teachers and the process of creating a new culture is always lengthy and it needs monitoring. It is up to the engineering curricula (and courses) to be prepared to support the new instructional context and the various adaptations that arise from it.
Regarding the limitations of this paper, we could not detail the multiple associations and combinations used in the hybrid teaching methodologies described here.
The literature review considered international articles published in the Web of Science – Current Contents Connect database. Although it is a reference database and internationally recognized, it would be interesting to expand with the use of other databases such as Scopus and Google Scholar
Nevertheless, this analysis could uncover functional synergies that may lead to better practices to be implemented in engineering and which can become potential topics for future research.
Alonso, F. et al. (2009), “A moderate constructivist e-learning instructional model evaluated on computer specialists”, Computers & Education, Vol.53 No.1, pp.57–65. https://doi.org/10.1016/j.compedu.2009.01.002
Arnab, S. et al. (2015), “Mapping learning and game mechanics for serious games analysis” British Journal of Educational Technology, Vol. 46, No.2, pp. 391–411. http://dx.doi.org/10.1111/bjet.12113
Barker, B. S.; Ansorge, J. (2007), “Robotics as means to increase achievement scores in an informal learning environment”, Journal of Research on Technology in Education, Vol. 39 No. 3, pp. 229–43. http://dx.doi.org/10.1080/15391523.2007.10782481
Benitti, F. B. V. (2012). “Exploring the educational potential of robotics in schools: A systematic review”, Computers & Education, Vol. 58, No.3, pp. 978-88. http://doi.org/10.1016/j.compedu.2011.10.006
Borrego, M. et al. (2014), “Systematic Literature Reviews in Engineering Education and Other Developing Interdisciplinary Fields”, Journal of Engineering Education, Vol.103 No.1, pp.45–76. http://doi.wiley.com/10.1002/jee.20038
Bowen, W.G. et al. (2014), “Interactive Learning Online at Public Universities: Evidence from a Six-Campus Randomized Trial”, Journal of Policy Analysis and Management, Vol. 33, No. 1, pp. 94–111. http://doi.wiley.com/10.1002/pam.21728
Cañas, A. J. et al. (2000), “Herramientas para construir y compartir modelos de conocimiento basados en mapas conceptuales”, Revista de Informática Educativa, Vol.13 No.2, pp.145–158.
Case, J. M.; Light, G. (2011), “Emerging Research Methodologies in Engineering Education Research”, Journal of Engineering Education, Vol. 100, No. 1, pp. 186-210. http://doi.org/10.1002/j.2168-9830.2011.tb00008.x
Chevalier, A. et al. (2017), “A Three-Year Feedback Study of a Remote Laboratory Used in Control Engineering Studies”, IEEE Transactions on Education, Vol. 60, No. 2, pp. 127-33. http://doi.org/10.1109/TE.2016.2605080
Corter, J. E. et al. (2011), “Process and learning outcomes from remotely-operated, simulated, and hands-on student laboratories”, Computers & Education, Vol. 57, No.3, pp. 2054-67. http://doi.org/10.1016/j.compedu.2011.04.009
Cortizo, J. L. et al. (2010), “Blended learning applied to the study of Mechanical Couplings in engineering”, Computers & Education, Vol. 54, No.4, pp. 1006-19. http://doi.org/10.1016/j.compedu.2009.10.006
Dias, S. B. et al. (2015), “Fuzzy cognitive mapping of LMS users’ Quality of Interaction within higher education blended-learning environment”, Expert Systems with Applications, Vol.42, No.21, pp. 7399–423. http://doi.org/10.1016/j.eswa.2015.05.048
Dormido, R. et al. (2008), “Development of a web-based control laboratory for Automation Technicians: the three-Tank system”, IEEE Transactions on Education, Vol.51, No.1, pp. 35–44 http://doi.org/10.1109/TE.2007.893356
Fabregas, E. et al. (2011), “Developing a remote laboratory for engineering education”, Computers & Education, Vol.57, No.2, pp. 1686-697. http://doi.org/10.1016/j.compedu.2011.02.015
Fernandez, C. et al. (2011), “A Student-Centered Introductory Programming Course: The Cost of Applying Bologna Principles to Computer Engineering Education”, International Journal of Engineering Education, Vol.27, No.1, pp.14–23.
Garrison, D. R.; Kanuka, H. (2004) “Blended learning: Uncovering its transformative potential in higher education”, The Internet and Higher Education, Vol.7, No.2, pp. 95–105. http://doi.org/10.1016/j.iheduc.2004.02.001
Gomes, L.; Bogosyan, S. (2009), “Current trends in remote laboratories”, IEEE Transactions on Industrial Electronics, Vol.56, No.12, pp. 4744–56. http://doi.org/10.1109/TIE.2009.2033293
Hagemans, M. G. et al. (2013), “The effects of a concept map-based support tool on simulation-based inquiry learning”, Journal of Educational Psychology, Vol.105, No. 1, pp.1-24. http://dx.doi.org/10.1037/a0029433
He, W. et al. (2015), “Implementing flexible hybrid instruction in an electrical engineering course: The best of three worlds?”, Computers & Education, Vol. 81, No.1, pp.59–68. http://doi.org/10.1016/j.compedu.2014.09.005
Heradio, R. et al. (2016), “Virtual and remote labs in education: A bibliometric analysis”, Computers & Education, Vol. 98, Supplement C, pp. 14–38. http://doi.org/https://doi.org/10.1016/j.compedu.2016.03.010
Jara, C. A. et al. (2011), “Hands-on experiences of undergraduate students in Automatics and Robotics using a virtual and remote laboratory”, Computers & Education, Vol. 57 No. 4, pp. 2451-461. http://doi.org/10.1016/j.compedu.2011.07.003
Jara, C. et al. (2009), “Real-time collaboration of virtual laboratories through the Internet”, Computers & Education, Vol.52, No.1, pp. 126-40. http://doi.org/10.1016/j.compedu.2008.07.007
Kusano, K. et al. (2013), “The effects of ICT environment on teachers’ attitudes and technology integration in Japan and the US”, Journal of Information Technology Education: Innovations in Practice, Vol.12, No.1, pp. 29–43.
Lazar, C.; Carari, S. (2008), “A remote-control engineering laboratory”, IEEE Transaction on Industrial Electronics, Vol.55, No.6, pp. 2368–375. http://doi.org/10.1109/TIE.2008.920650
Lindsay, E. B. (2004), “The best of two worlds: Teaching a hybrid course”, Academic Exchange Quarterly, Vol.8, No.4, pp. 16-20. Macedo-Rouet, M. et al. (2009), “Students’ performance and satisfaction with Web vs. paper-based practice quizzes and lecture notes”, Computers & Education, Vol. 53 No. 2, pp. 375-84. http://doi.org/10.1016/j.compedu.2009.02.013
Méndez, J. A.; González, E. J. (2010), “A reactive blended learning proposal for an introductory control engineering course”, Computers & Education, Vol.54, No.4, pp. 856–65. http://doi.org/10.1016/j.compedu.2009.09.015.
Moon, B. et al. (2011), Applied concept mapping: Capturing, analyzing, and organizing knowledge, CRC Press Taylor & Francis Group, Boca Ratom, Flórida.
Nugent, G. et al. (2010), “Impact of Robotics and Geospatial Technology Interventions on Youth STEM Learning and Attitudes”, Journal of Research on Technology in Education, Vol. 42 No. 4, pp. 391–408. http://doi.org/10.1080/15391523.2010.10782557
Nugent, G. et al. (2016), “Robotics camps, clubs, and competitions: Results from a US robotics project”, Robotics and Autonomous Systems, Vol.75, Part B, pp. 686–91. http://doi.org/10.1016/j.robot.2015.07.011
Potkonjak, V. et al. (2016), “Virtual laboratories for education in science, technology, and engineering: A review”, Computers & Education, Vol.95, Supplement C, pp.309-27. http://doi.org/10.1016/j.compedu.2016.02.002
Rawson, K. A. et al. (2015), “The power of examples: illustrative examples enhance conceptual learning of declarative concepts”, Educational Psychology Review, Vol.27, No. 3, pp.483–504. http://doi.org/10.1007/s10648-014-9273-3
Stefanovic, M. et al. (2011), “A LabVIEW‐based remote laboratory experiments for control engineering education”, Computer Applications in Engineering Education, Vol. 19, No. 3, pp.538–549. http://doi.org/10.1002/cae.20334
Sullivan, F. R. (2008), “Robotics and science literacy: thinking skills, science process skills and systems understanding”, Journal of Research in Science Teaching, Vol.45, No. 3, pp. 373–94. http://dx.doi.org/10.1002/tea.20238
Tiernan, P. (2010), “Enhancing the learning experience of undergraduate technology students with LabVIEWTM software”, Computers & Education, Vol. 55, No.4, pp.1579-88. https://doi.org/10.1016/j.compedu.2010.07.001
Verkroost, M. J. et al. (2008), “Finding a balance in dimensions of blended learning”, International Journal on eLearning, Vol.7, No.3, pp.499-522.
Wake, J. D.; Dysthe, O. (2007), “New and changing teacher roles in higher education in a digital age”, Educational Technology & Society, Vol.10, No.1, pp.40–51.
Ward, B. (2004), “The Best of Both Worlds: A Hybrid Statistics Course”, Journal of Statistics Education, Vol. 12 No. 3, pp. 1-10. ww2.amstat.org/publications/jse/v12n3/ward.html
Received: 23 Nov 2017
Approved: 10 May 2018
How to cite: Pereira, C.A; Oliveira, P.; Reis, M.J.C.S. (2018), “Non-traditional process in higher education engineering: a conceptual mapping”, Brazilian Journal of Operations & Production Management, Vol. 15, No. 1, pp. 12-20, available from: https://bjopm.emnuvens.com.br/bjopm/article/view/412 (access year month day).