Performance measurement in product development process (PDP): literature review and gaps for further research

Goal: this paper performs a literature review to identify and analyze the main themes related to PDP performance measurement and presents the gaps for further exploration. 
Design / Methodology / Approach: this paper searches the main bases of academic data in order to find scientific papers related to PDP performance measurement and classify them on five main themes: design performance; human resources; knowledge resources; financial performance, and literature review. 
Results: the results show that more than fifty percent of the papers are associated with knowledge resource issues and the principal gaps to be explored are associated with design performance, human resources, financial performance and literature review. 
Limitations of the investigation: the main limitation is related to the bases of academic data. Additional papers available on different bases can lead to different results.    
Practical implications: this paper intends to make a contribution to academia and practitioners as it shows gaps on PDP performance measurement research that can be better explored. 
Originality / Value: to the best of the authors’ knowledge, there is no study that evaluates a literature review for PDP performance measurement. This work contributes to the literature by presenting the directions for further PDP performance measurement research.


INTRODUCTION
An intense competitive environment requires from the organizations to create superior customer value and meet customer needs in a superior way. Thus, a fast and efficient development of new products has become a top priority in many organizations and, consequently, the product development process has become a dominant concern (Lee and Wong, 2011). Gurumurthy and Kodali (2012) show that competitiveness imposes tremendous pressure on organizations to bring new products to market faster and faster, which requires organizations to improve their current Product Development Process (PDP).
According to Browning and Ramasesh (2007), "product development can be a major competitive lever for a firm". "A new product plays an important role in the growth of any organization. It is a strategic step that shapes the several key decisions of an organization, having implications on both internal and external subsystems. It furthers the efforts of any organization to achieve sustainability and thereby increase profitability" (Sinha, 2015). Moreover, the pressures for "faster, better, cheaper" products has taken the interest in engineering, project management, operations management, organizational science, marketing and others disciplines and it is generating an extensive body of literature in the product development area -particularly in PDP (Browning and Ramasesh, 2007;Castro et al., 2019).
The PDP is a complex and singular activity. This complexity and singularity makes the PDP improvement a perennial task with a wide range of methodologies that provide theoretical support, recommended procedures / techniques and useful tools in the various phases of the project (Ensslin et al., 2011).
In this context, the performance measurement of the PDP becomes a significant issue. According to Tyagi et al. (2015), "an efficient PDP is simply an enabler of better products with improved quality at cheaper cost, but countless obstacles prevent PDP from being under control and well managed." Many frameworks have been developed and tested for the purpose of analyze the PDP and improve its efficiency. Based on these frameworks, PDP performances indicators such as iteration, time, efficiency and others, have been examined and analyzed from different panorama and point of view (Fu et al., 2013). Oliveira and Kaminski (2012) observe that most of organizations use an internal product development methodology; however, it is still a difficulty to achieve weighty and sustained improvements.
Taking into consideration the large, complex and challenging characteristics of PDP performance measurement, a considerable quantity of published papers is available; however, no research paper providing a literature review of PDP performance measurement has been found, to the best of these authors' knowledge. Thus, the authors understand that a survey, which synthesizes all this extent of work, is both appropriate and effective. Thus, this paper presents a methodological review for better understanding the contemporary state of research regarding the subject "performance measurement into PDP" and the following question is answered: "What are the main themes addressed by academic research related to the PDP performance measurement and what are the issues with the potential to be better exploited?" To answer this question, the following research methods were unfolded: 1) analyzing the main scientific bases, looking for papers related to PDP performance measurement; 2) performing the screening of paper related to PDP performance measurement, analyzing and classifying the results; and 3) establishing conclusions on the main issues addressed and the gaps yet to be further exploited by academia.

Definitions and concepts related to PDP
Taking into consideration the analyzed papers, various definitions were found for PDP and there is a consensus among them, even though they are not exactly equal. In this paper, the considered definition for PDP is in agreement with Whitney (1990) apud Fu et al. (2013) -"this is a specific process of converting customer needs into a technical and commercial solution". A similar definition is presented by Rozenfeld et al. (2006) -"PDP is a business process, a set of activities that aims to achieve a product or service that meets the needs of an internal or external client of the company" and Quintella and Rocha (2007) state -"PDP is a strategic activity linked to consumer's needs, identifying their wants and needs and responding with innovative and well-designed products". Lee and Wong (2011) present a more generic definition, considering PDP as "a system encompassing the dynamic interaction between internal and external factors." This broad definition of PDP makes it a challenging and complicated process to model (Paula and Ribeiro, 2007). According to Browning et al. (2006), the interdependencies of actions, their outcomes and their assigned professionals are among the main characteristics that make PDP extremely complex to model. PDP can be considered an extensive, complex and unstructured network. In fact, human resource, hardware resource, software resource, knowledge resource, among others are connected in PDP models in addition to the design activities (Fu et al., 2013).
According to Smith and Morrow (1999) apud Fu et al. (2013), the importance to model PDP is to guide the management for future product development processes. Each organization needs specific characteristics for their PDP, but some of these characteristics can be shared or common (Fu et al., 2013). According to Quintella and Rocha (2007), as PDP requires organizations to integrate and align their resources in a structured way, each organization needs specific PDP tools and methods to fulfill these requirements.
Based on this scenario, PDP involves a series of phases, tasks and activities that complement each other, ranging from simultaneous to sequential, and presenting their specificities according to the product to be developed (Ensslin et al., 2011). Driva et al. (2000 apud Aydin et al. (2007) state that product development demands a proper combination of intersectional activities, restricted costs, short time to market, high quality, and increased flexibility to be effective and successful.

Performance measurement for PDP
According to Silva (2001), PDP needs to be efficient and effective to create a competitive advantage to an organization. Based on it, emerges the vital relevance of the PDP performance measurement -systematically evaluating the efficiency and effectiveness of PDP. Moreover, the importance of measuring PDP performance lies in documenting the value of development efforts, evaluating research programs, and properly allocating resources (Hauser and Zettelmeyer, 1997 apud Lettice et al., 2006). However, to achieve the goals mentioned the main difficulties occur due to the multi-dimensionality of product development, the convergence on micro-level metrics (isolated project level), long time scales, volatility, unstructured and intangible characteristics. Likewise, the importance of measuring and managing PDP-related knowledge assets, mainly in a knowledge driven economy such as the current one, is of high importance (Lettice et al., 2006).
Taking into account the multidimensional characteristics of PDP, Kleinschmidt et al. (2010) present two fundamental aspects for the performance assessment of a new product development process -the non-financial and financial aspect. The non-financial aspect is related to the strategy of the organization, in which launch efficiency and windows-of-opportunity are considered. The financial aspect contemplates the perceived financial performance. Montoya-Weiss and Calantone (1994), apud McNally et al. (2011), also present financial (profit, sales, payback period, costs and others) and non-financial aspects (market-based, market share and technical) for performance assessment.
According to Lettice et al. (2006), the financial aspect based on traditional financial measures is the one with the best capability to be measured by the organizations. McNally et al. (2011) demonstrate process impact on delivering product profitability directly through time, quality and expense trade-offs, and indirectly, through market speed and development investment timing.
According to Fu et al. (2013) the complex PDP network involves product (design activities and others) and resources (human resources, knowledge resources and others) and all these elements together and the relationship between them are ignored in most of the traditional studies. Each of these elements is detailed in its research as follows below.
• Design -it refers to activities that have specific requirements, such as design performance, cost, and others, usually with an independent input, output and technical requirements; • Resource -it refers to the resource used in the product development process, such as human resource, tool resource, and knowledge resource; • Human Resource -it mainly includes designers, lab staff, etc., with characteristics of work ability, work reliability, work load, and others; • Tool Resource -it mainly includes test equipment, measuring equipment, manufacturing equipment (hardware tool resource) and CAD / CAM software (e.g. Pro/Engineering, UG, Solidworks), finite element analysis software (e.g. Ansys, Abaqus), calculation software (e.g. Fluent, Matlab), and others (software tool resource); • Knowledge Resource -it refers to the explicit and implicit knowledge used in PDP, such as industrial standard, national standard, calculation program, empirical formula, and so on.
The importance of the elements mentioned is reinforced by other studies. According to Aydin et al. (2007), a positive performance of innovation and new product development capability, both grouped in the design element, influence positively the organization performance, enabling competitive advantage. Humphreys et al. (2005) show the importance to add customers and suppliers to the value system of an organization and proposes a process redesign to achieve this integration which directly affects the organization resources.
The relevance of sustainability inclusion in the design and resource elements is also important to mention. According to Genç and Benedetto (2015), the unite of an environmental professional in a PDP team during the concept development phase, commercialization stage, and product launch influences the product performance positively. Allais et al. (2015) propose the inclusion of territorial resources in the design process for the purpose of adding value for both the organization and society.

Scientific research classification
Based on the classical criteria, this scientific research can be classified as: deductive from the perspective of methods, bibliographic regarding the technical procedures, qualitative concerning the approach of the issue, applied regarding the nature of this research and descriptive in regards of its objective.
According to Andrade (1999), the deductive was firstly introduced by René Descartes, for which the only way to find a certainty is through the reason -the absolute principle of human knowledge. Lakatos and Marconi (2003) and Silva and Menezes (2005) consider that deductive method intends to explain the content of assumptions and facts through reason.
Regarding the technical procedures, Gil (2002) and Lakatos and Marconi (2003) considers that bibliographic researches cover the complete bibliography published in relation to the studied subject. So, it is aligned with this paper research method -examine materials developed and published mainly in books and scientific papers.
Based on Gil (2002), Silva and Menezes (2005) and Godoy (1995) a qualitative study has a sequence of activities comprehending the data base interpretation, data reduction, categorization, interpretation and final conclusion. It not make use of statistical methods nor employs statistical tools in database analysis.
Considering the nature of this research, Silva and Menezes (2005) refer applied research as the one that generates knowledge for practical application and aims to solve practical problems.
Lastly, Gil (2002) considers descriptive research as the one that describe certain population or phenomenon char-acteristics or establish relationships between variables. The most significant features of the descriptive research are the use of standardized techniques for data collection, such as questionnaires or systematic observation.

Research method
A literature review had been performed in order to achieve the purpose of this paper, which is identifying papers related to PDP performance measurement and classify them in order to find gaps and opportunities for further studies and research.
The following scientific bases were investigated via internet: Emerald Insight, Periodical Capes (Periódicos Capes, in Portuguese), SciELO, Scopus, Web of Science and Wiley. The terms used for the search were generated by application of the "relevance tree" method. According to Saunders et al. (2009), this method helps to define which key words are directly relevant to the research question and objectives.
The applied terms were "product development process" combined with the words "performance measurement", "performance assessment", "measurement model", "assessment model", "maturity", "key performance indicators", and "critical success factors". The "critical success factors" was taken into consideration because the authors believe that papers related to this term could deal with PDP performance measurement issues.
Seven combinations of these key words were considered -product development process AND performance measurement, product development process AND performance assessment, product development process AND measurement model, product development process AND assessment model, product development process AND maturity, product development process AND key performance indicators, and product development process AND critical success factors. Using the "title, abstract, keywords" search in scientific bases, "journal" papers (conference papers, books and chapters of books were excluded) were collected and stored for the above mentioned search terms with maximum published age of ten (10) years. This last decade has presented a plethora of studies on PDP performance measurement that debate definitions, scopes and study causal relationships with antecedents and implications.
The beginning search on scientific bases resulted in one hundred and six (106) papers. After a detailed analysis and eliminating the duplicated papers, the study sample consisted in thirty-seven (37) papers.
The themes addressed in these papers were identified and classified according to the Figure 1. environmental specialists into product development teams, and others related to the PDP human resources; Knowledge resources (2.2): this item refers to the explicit and implicit knowledge used in PDP. It includes proposed frameworks, models, mechanisms, tools, knowledge management into PDP, and applied methodologies, such as Concurrent Engineering (CE), Lean Product Development (LPD), Agile Product Development (APD), and others related to PDP; Financial Performance (3): this item refers to papers whose themes discuss the performance of PDP under a financial perspective. This paper considers the financial subject as a specific category because financial prosperity should be the final target of every organization. Among the most commonly used metrics for this category are the product profitability, return on investment (ROI), and sales; Literature Review (4): this item refers to papers that investigate the present literature on various characteristics and dimensions related to PDP performance.   The following is a description of each category above. This classification was elaborated by the authors based on Fu et al. (2013).

RESULTS AND DISCUSSION
Design Performance (1): this item refers to papers whose themes discuss activities related to PDP and products such as the rapid prototyping method, Computer-Aided Design (CAD), Computer-Aided Engineering (CAE), robotics, design innovation, sustainable products development, marketing performance and others, and their impact and effects on PDP; Resource (2): this item refers to papers whose themes are related to PDP resources. Due to the involvement of many areas on PDP, many types of resources are used in different fields. Specifically, on this paper, which is focused on the performance measurement of PDP, the resources are classified in two essential groups: human resources and knowledge resources.
Human resources (2.1): this item refers to papers whose themes are related to work ability, work reliability, work load, designers, product development teams, integration of Present the importance of taking into consideration the territorial resources in a product development process (sustainable perspective).
Inclusion of territorial resources in the product development process

3
Bare and Cox Present a method for applying principles of mass customization to the empirical sub-processes within a product development process.
Applying principles of mass customization to improve the empirical product development process Journal of Intelligent Manufacturing 4 Berginc et al.
Analyze the cooperation of the furniture industry and designers in product development processes based on a SWOT analysis.
Analysis of cooperation between the furniture industry and designers in product development process Examine whether a needs assessment can identify the gap between what "is" occurring and what "should" be occurring in an organization's PDP.

DRVNA Industrija
Using needs assessment as a learning tool in the product development process  Dayan and Colak (2008) Explore the antecedents and consequences of the level of procedural justice climate in new product development teams.
The role of procedural justice in the new product development process Examine experimentally the ability of product design groups to achieve specific or general cost reduction goals under simulated sequential or concurrent new product development.
The effect of cost goal specificity and new product development process on cost reduction performance Accounting, Organizations and Society Address the problem of selecting a suitable product development methodology to improve a current PDP.
An application of analytic hierarchy process for the selection of a methodology to improve the product development process Journal of Modeling in Management 19 Josifovski and Minovski (2015) Identify potential solution to a performance measurement and management method within the new PDP to improve transparency, efficiency, and the management of this process.
Defining a performance measurement system as an improvement to the new product development process Develop a framework comprised of knowledge processes in order to provide support for the issue of integration as a key factor in organizations and in innovation projects.
Integrating knowledge and knowledge processes: a critical incident study of product development projects

Journal of Production and Innovation Management 22
Laurenti et al.
Present the findings of a case study carried out to assess the application of the methods FMEA and of an automotive parts manufacturer.

Assessment of the methods FMEA and DRBFM applied in the new PDP of an auto parts manufacturer
Gestão da Produção 23 Lee and Wong (2011) Examine the extent to which the technological change and competitive intensity impact the new product development practices on their launches.
Identifying the moderating influences of external environments on the new product development process Reconcile conflicting results regarding the speed to market, product quality, and their joint impact on product profitability.
New product development processes and new product profitability: exploring the mediating role of speed to market and product quality Journal of Production and Innovation Management 29 Oduola et al.
Present a comparative study on product development process in the Nigeria Coat of Arms, using computer numeric control (CNC) and rapid prototyping methods.
A comparative study of product development process using computer numerical control and rapid prototyping methods Present a comprehensive set of process-related key performance indicators for product development, based on a three-step systematic literature review, followed by the systematization of indicators and a critical analysis.
Process-related key performance indicators for measuring the sustainability performance of ecodesign implementation into product development Journal of Cleaner Production 32 Tyagi et al. (2015) Exploit lean thinking concepts in order to manage, improve and develop the product faster while improving or at least maintaining the level of performance and quality.
Value stream mapping to reduce the leadtime of a product development process International Journal of Production Economics 33 Sinha (2015) Examine new product development process through the new products  Figure 2, according to the categories presented in Figure 1, is presented on Figure 4. As showed, 62% of papers (23 papers) have their themes associated with "knowledge resources" (item 2.2 in Figure 1), 27% of papers (10 papers) have their themes asso-ciated with "design performance" (item 1 in Figure 1), 5.5% of papers (2 papers) have their themes associated with "human resources" (item 2.1 in Figure 1), 5.5% of papers (2 papers) have their themes associated with "financial performance" (item 3 in Figure 1) and 0% of papers (no paper) have their themes associated with "literature review" (item 4 in Figure 1).
In regards to "design performance" (item 1 in Figure  1), it is important to mention Berginc et al. (2011), who demonstrate the importance of having an efficient process to develop innovative products and, consequently, maintain the competitiveness of the organization at levels consistent with the market.
The "human resources" (item 2.1 in Figure 1) takes also a relevant aspect as the team work influences decisively the PDP performance. Dayan and Colak (2008) found that product development work team characteristics, such as procedural justice climate, had significant impacts on new product creativity and speed to market, which are important variables to the PDP performance. Such impacts are caused by other factors as the integration of new specialists, as environmental specialists, into a product development team that is usually composed by marketing, manufacturing, R&D personnel, and other specialists (Genç and Benedetto, 2015). Thus, there is still a large field for research regarding this item in order to complement the two papers mentioned and which are the only ones in this item.
The item "financial performance" (item 3 in Figure 1) has a special importance, as it is the survival factor for organizations. McNally et al. (2011) explored its relationship with product quality and speed to market. This is an item with great possibility of research, as there are only two papers.
As to the item "literature review" (item 5 in Figure 1), no paper has been found. It is important to mention that literature review papers have an important role revealing an contemporary scenario and identifying the gaps of the latest published researches. So, this paper intends to make a contribution in this item showing and analyzing the main points addressed by academic research related to the PDP performance measurement and the issues with the potential to be better explored.
Finally, the item "knowledge resources" (item 2.1 in Figure 1) is the one with the highest number of published papers -more than 50% of the studies. It can be explained due to the current "economy knowledge", in which organizations that generate the necessity for regular innovation are inserted (Colombo et al., 2015), and also due to the condition that each organization needs specific characteristics for its PDP (Fu et al., 2013). There-fore, research in this field remains important as there are challenges to be better explored in this item.

CONCLUSION
As mentioned, this paper is determined in the analysis of the principal bases of scientific data for the purpose of identify the main topics of research associated with the PDP performance measurement. The search in scientific basis make possible the finding of thirty-seven (37) papers that were analyzed and classified into structured categories, based on Fu et al. (2013).
Based on the found results, it was possible to answer the proposed question. The main theme researched on the PDP performance measurement is related to "knowledge resources" (item 2.1 in Figure 1), and the second most researched theme on the PDP performance measurement is "design performance" (item 1 in the Figure  1), with a difference of thirteen (13) papers. Both items presented a significant number of scientific articles higher than other themes that had one or, at most, two published papers.
The conclusion is that there are many fields to be explored in scientific research that can provide positive results to the PDP performance measurement. The gaps to be explored by researchers are noticed mainly in the items "design performance" (item 1 in Figure 1), "human resources" (item 2.1 in Figure 1), "financial performance" (item 3 in Figure 1), and "literature review" (item 4 in Figure 1), due to lower quantity of papers when compared to the item "knowledge resources" (item 2.2 in Figure 1). Therefore, it is important to mention that the contribution on any subject related to PDP performance measurement is expected due to the intricate characteristic, involvement of many areas, various forms of measurement and application specific to each organization.
It is expected that these results and remarks be valuable to both the academia and practitioners, stimulating further research on the PDP performance measurement area.