According to Ulrich and Eppinger (2008), new product design and development (NPD) is defined generally as “the set of activities beginning with the perception of a market opportunity and ending in the production, sale and delivery of a product.” Market opportunity may come from market pull or from technology push. The “technology push” describes a situation where an emerging technology or a new combination of existing technologies provide the driving force for an innovative product and problem solution in the marketplace (Herstatt and Lettl, 2000). Conversely, the term “market pull” implies that the product or process innovation has its origins in latent, unsatisfied customer needs in the marketplace. The identification of these needs occurs first and is then followed by the required development activities (Chidamber and Kon, 1993). There are two parallel paths involved in the NPD process: one involves the generation of ideas (idea generation), development of concepts (concept development), and detailed engineering calculations (detailed engineering); the other involves market research and marketing analysis (market pull) or internal R&D (technology push). Engineering oriented researchers are interested in the first path which is common in both technology push and market pull strategy; it consists of idea generation, concept development and detailed engineering. Typically, the steps of an NPD project are shown in Figure 1.

NPD plays an important role among the activities of a company. Companies typically see new product development as the first stage in generating and commercializing new products within the overall strategic process of product life cycle management used to maintain or grow their market share. Conventionally, NPD is executed inside the company within the NPD team which consists of engineers and experts with various majors: mechanical engineering, electrical engineering, industrial engineering, finance, marketing, etc. The external resources are limited to close partners or vendors.

In the mid 2000s, the Internet explosion and Web 2.0 era opened opportunities for a boundless collaboration in NPD. Plenty of companies executed their NPD projects not only in the office but also on the open Internet environment. By making open calls to the “crowd” on the Internet to attract their attention to NPD problems, some companies can grasp solutions and suggestions for their NPD projects in short time at low cost (Brabham, 2008; Schenk and Guittard, 2009; Fahling et al., 2011). As a case of inviting the crowd to an NPD project, FIAT Brazil-an automobile company made an open call for designs and suggestions for their new model named Fiat Mio CC. The Fiat Mio CC project website at www.fiatmio.cc draws together 14,000 visitors from 140 countries, most of whom can be considered as potential customers (Brondoni, 2010).

The action of giving an open call to the “crowd” on the Internet and getting feedbacks in the form of solutions or suggestions is termed “crowdsourcing” by Howe (2006) in an article in Wired Magazine. The term “crowdsourcing” is the combination of “crowd” and “outsourcing”. Howe (2009) defined “crowdsourcing is the act of taking a job traditionally performed by a designated agent (usually an employee) and outsourcing it to an undefined, generally large group of people in the form of an open call.”

Crowdsourcing is based on a simple concept: anyone has the potential to plug in valuable information (Greengard, 2011).

Innocentive.com (Howe, 2006; Piller, 2010) is a website that exploits crowdsourcing for general problem solving. In this case, companies who have unsolved engineering and scientific problems can post their problem on InnoCentive website and receive solutions submitted by individuals from the crowd. Once a solution is selected to be used, the company will patent it and give the reward to the solution provider.

There are plenty of companies applying the crowdsourcing concept for product development. In Figure 2, Kim et al. (2010) introduce a classification result of cases of applying the crowdsourcing concept for NPD.

There are several reasons that can explain why individuals are willing to join a crowdsourcing project.

Most important reasons might be the following:

1.3.1 Financial rewards

In most crowdsourcing projects, individuals from the crowd are encouraged to join the project to gain financial rewards. For example, in Mechanical Turk (Howe, 2006; Horton and Chilton, 2010)？a website belonging to Amazon, individuals can be rewarded as they complete a certain task (termed human intelligence task [HIT]) required by the company (on behalf of their clients), or in the case of Innocentive.com, the winning solution provider is the only person who will be rewarded. The amount of financial rewards may vary from tens of cents for each task in the case of Mechanical Turk or hundreds of thousand US dollars in the case of big projects posted on Innocentive.com. Doing some jobs in free time with no string attached and having chances to earn extra money is an important motivation for the participation of the crowd in a crowdsourcing project.

1.3.2 Reputation and recognition

Besides crowdsourcing projects that offer financial rewards, there are many crowdsourcing projects which do not offer any kind of payment for the participants. But still, there are plenty of individuals joining those projects for free. In the case of Yahoo! Answers (Wightman, 2010), a website where users ask and answer one another’s questions, a user who provides good answers will gain points, and the higher the points he has, the higher the reliability his answer will be. The points here are considered as the user’s reputation.

1.3.3 Opportunities

By joining a crowdsourcing project, especially in community websites, participants can gain many kinds of opportunities. The crowdsourcing project itself is a place where participants can show off their best and get attention from potential customers, partners, or recruiters.

In the case of Flickr (Kleemann et al., 2008), individuals are allowed to upload their photos on the website. Some people upload photos for fun while others use the website as the place for their free online galleries to show their best artworks to the world, which includes people who potentially want to purchase the works, hire the author or make some kinds of co-operation.

1.3.4 Joy and fun

Some persons join a certain crowdsourcing project for nothing but joy and fun. Facebook-the largest social network service so far can be referred as an example. Most of the Facebook users use the website to share their status, thoughts, stuffs, etc., and connect to their friends, colleagues, family to update information.

1.3.5 Contribution willingness

Wikipedia (Kleemann et al., 2008) is a website that allows anyone to submit, edit, and discuss on articles about any kind of topics. The quality of the contents posted on Wikipedia is surprisingly high and the motivation of the individuals to continuously update articles is their willingness to contribute.

1.3.6 “Prosumer” trend

“Prosumer” is the combination of “pro” and “consumer.” Consumers nowadays are no longer satisfied with the pre-configured products and services offered by manufacturers. They want to join the “co-creation” process to put their own ideas on the future products and services.

Threadless (Howe, 2009), Fiat Mio CC, Dell Idea Storm (Kleemann et al., 2008) are representatives of crowdsourcing websites that fit this trend. Voices of customers are heard and good ideas are grasped to make products and services that people are willing to buy.

The major motivations for crowd participation in a crowdsourcing project are listed in Table 1.

It is the Web 2.0 which allows the crowd to participate in a crowdsourcing project. Web 2.0 has interactive forms that allow individuals from the crowd easily to participate in the project by submitting content such as: ideas, suggestions, files, etc., and leaving comments, votes, etc. Web 2.0 is the environment for users to generate content (termed “user generated content”) (Kleemann et al., 2008). Without these features of Web 2.0 (or later web generation), crowdsourcing projects cannot be executed on the Internet.

As the name suggests, in a crowdsourcing model, crowd is the core factor. Therefore, the crowd participation pattern (for a crowdsourced NPD project) must be analyzed for an effective implementation of crowdsourcing. Generally, most of the previous research mentioned reward as a factor leading to participation of the crowd in a crowdsourcing project (Ebner et al., 2009; Albors et al., 2008; Horton and Chilton, 2010). In a crowdsourced NPD process, other factors such as: NPD phase characteristics (“process” factor), product type and complexity (“product” factor), as well as crowd type and size (“crowd” factor) might also have influences on the participation of the crowd.

For the process factor, each phase in an NPD process has its own characteristics that are different from the others.’ Participation of the crowd on different phases is different.

In the work of Kim et al. (2010), the number of cases for different phases is different. In this work, among the total of 33 cases surveyed, 13 of them (i.e., 39.4%) laid on the phases of Concept/Design; while only 6 of them (i.e., 18.2%) laid on the phase of Testing/Customer’s Rate (Figure 2).

There have been many cases of applying the crowdsourcing concept for new product development but the surveyed cases are relatively noticeable ones in terms of attracting crowd participation. Hence, the distribution of cases through phases might imply the dependence of crowd participation on phase characteristics. This paper aims to investigate that kind of “crowd participation pattern” over NPD phases.

Ward et al. (1995) suggested that sets of design alternatives work better for Toyota designers to reach the final answers than pursuing one alternative iteratively. Open innovation can bring chances to obtain design alternatives. Dunphy et al. (1996) argued that there are 7 macro and micro level discriminators making technological innovation happen: 1) the technological prerequisites; 2) the country's socio-cultural tendencies; 3) the country's material, human, and institutional infrastructures; 4) the structure of the particular industry; 5) the size and nature of the individual firm; 6) the attitudes of the management of the firm; and 7) the establishment of standards for the widespread diffusion of the innovation. These 7 discriminators apply for open innovation as well.

Chesbrough (2006) pointed out that, in today’s distributed knowledge environment, a company can hardly afford to rely entirely on their own ideas to advance their business. Open innovation enables companies to create and profit from their own ideas as well as others’ ideas. Open innovation outlines a new environment for R&D, and demonstrates that this new environment replaces the logic of an earlier era, where innovation is closed from outside ideas and technologies.

West and Gallagher (2006) suggested that user involvement can help to obtain innovation by allowing users to contribute to the development of products using pooled product development strategy.

Open innovation can help to improve new product development by suggesting new ideas and design alternatives. Open innovation is sometimes referred to contain crowdsourcing as one of its forms (Ebner et al., 2009; Bartl et al., 2010). Some authors mentioned crowdsourcing as “community-based innovation” (Elmquist et al., 2009). Schenk and Guittard, (2009), on the other hand, argued that open innovation and crowdsourcing have differences in their targets (“innovation” for open innovation), and interacting objects (“firms” for open innovation).

Dekkers (2011) discussed Living Labs as innovation networks, the position of Living Labs as a part of the innovation process, the move from closed innovation to open innovation and the necessity to manage innovation within the setting of networks to collaboratively deliver new products and services to potential users. He implied that open innovation aims at increasing the throughput of innovation processes and at reducing uncertainty through interaction with users during new product (and service) development.

Crowdsourcing for new product development has gained popularity recently. Kleemann et al. (2008) reviewed phenomena related to crowdsourcing, factors contributing to the increasing prevalence of crowdsourcing and suggested 9 types of crowdsourcing by the form they apply. Among these types, ‘Participation of consumers in product development and configuration’ and ‘Product design’ can be directly applied to NPD.

Hinchcliffe (2007) claimed that “leveraging crowdsourcing effectively” is a critical factor for “product development 2.0” in terms of competitive advantage. In other work, Hinchcliffe (2009) suggested five functional business areas that are suitable for applying crowdsourcing. These are problem solving, design, collaborative work, testing and support. Furthermore, Bertoni and Chirumalla, (2011) suggested that crowdsourcing, through Web 2.0, can be an intuitive way to leverage bottom-up tools for the benefit of product development where customers and the crowd play in the innovation process. Snow et al. (2011) also agreed that crowdsourcing can be used as a problem solving model for collaboration issues in organizing continuous product development and commercialization. One of the reasons that crowdsourcing can be a source of product innovation is that there might be someone outside the company who knows the solution to the problems faced inside the company (Panchal and Fathianathan, 2008). Poetz and Schreier (2012) claimed that crowdsourcing initiatives among users can actually outperform professional in-house activities for the generation of product ideas, at least under certain conditions. Taha et al. (2011) studied user’s involvement in NPD and identified factors governing user involvement and designer practices in NPD.

Crowdsourcing for new product development can benefit firms by using consumers’ expertise, and the innovation provided by consumers can also be used for marketing purposes (Kleemann et al., 2008; Whitla, 2009).

Cooper and Edgett (2008) commented that crowdsourcing results are weak (in some cases), and recommend to implement crowdsourcing in certain types of firms.

Still, there is a lack of practical guideline for firms to decide what kind of task should be outsourced to the crowd in order to gain high rate of participation.

To get good results from crowdsourcing implementation for NPD projects, crowd participation is an important issue. The high crowd participation rate might lead to high chance of bringing innovation (Redlich et al., 2008).

The participation of the crowd in a certain crowdsourced NPD project depends on many factors, such as: reward factor, process factor, product factor, and crowd factor. So far, most of the research considered reward as the major factor that leads to the participation of the crowd to a certain crowdsourcing project. Reward factor is mentioned and considered as the main motivation for crowd participation by many authors (Albors et al., 2008; Brabham, 2008; Malone et al., 2009; Ebner et al., 2009; Fahling et al., 2011).

Mason and Watts, (2009) claimed that increased payments increases the quantity of work performed, but not its quality, and particular design of the compensation scheme can have a significant effect on quality in a crowdsourcing project. Horton and Chilton (2010) presented a model of workers supplying labor to paid crowdsourcing projects. In this work, a method of estimating a worker’s reservation wage (the lowest wage a worker is willing to accept for a task) has been introduced. Leimeister et al. (2009) described how activation enabling functionalities can be systematically implemented in an IT-based ideas competition and proposed an approach to designing components for participation support by utilizing incentives and motives of users. Borst (2010) explored the effects of motivations and rewards on participation as well as performance in voluntary online activities.

However, the effects of other factors (rather than reward) on crowd participation have not been investigated extensively. Only a few noticeable papers mentioned the effect of task complexity which includes the characteristics of NPD phases on the continuance of the crowd’s attention to the problem and the crowd’s sustained participation (Sun et al., 2012).

Although crowdsourcing gained popularity due to improved Internet access and web 2.0, it is little understood: 1) What are the different factors that affect the crowd participation in an NPD project?; 2) What is the crowd participation pattern for various product design and development phases for a new product development project?; and 3) How can crowdsourcing contribute with respect to innovation in new product development?

This paper focuses on the second point and the authors try to answer the question through literature and experiment. The third point is important with respect to crowdsourcing and innovation in NPD. It is a question that calls for another independent research and is not included in the scope of this paper. Specifically, this paper focuses on quantitatively figuring out the variation of participation of the crowd on various phases of a certain NPD process through experimental study.

In this paper we will consider the relationship: CP = f1(process|fixed {crowd, product, and reward}) where, CP stands for crowd participation.

We performed an experiment on a “crowd” of engineering related individuals in an online engineering community. This community was founded in 2006 and is currently running as an online forum. Most of its members are engineers, experts, students, lecturers, professors and engineering managers living all around the world and its contents can be accessed worldwide for free.

Table 2 shows some major statistical data about this community website as of March 29, 2011.

In each phase of idea generation, product concept design, detailed engineering calculation, physical prototyping, and design evaluation, we investigated how the crowd responds to the open call. In each phase, we observed how many percent among the announced individuals would download and look deeply at the problem description sheet and how many percent among them would try to solve and submit the solution. By analyzing the experimental results, conclusions are drawn about the participation of the crowd to an NPD problem as well as the “participation pattern” that illustrates different responses of the crowd to open calls for different NPD phases.

The experiment, which lasted for 75 days, was performed in November and December 2010 in the form of a product design and development contest named “NPD 11.” The data from the contest is collected as experiment data and those data will be used for analysis.

The contest was conducted in 5 phases as shown in Figure 3.

All individuals from the website are allowed to participate in this contest without any restriction. For judging the submissions of participants, the authors establish a committee of experts consists of 7 members who are well known and respected among the community. They are currently working as designers, CAD operators, and engineering managers. For each phase, the organizer of the contest (named “the organizer” from now on) uploads the announcement as well as the attachment of problem description sheet on the website and send invitation to all members to join the contest. Participants are allowed to raise any kind of questions about the contest. Participants submit their solutions via email to the organizer. Once the submission is received from a participant, it will be checked and forwarded to the committee of experts for judging. Judging results then will be sent back to the organizer to announce to participants as well as to identify and reward the winners.

Figure 4 shows the workflow of one phase of NPD11.

Details about each phase as follows:

Phase 1: Product idea generation

In this phase, the participants are allowed to freely propose any product idea that they think to be novel, useful, unique, etc. The proposal should include: name, description, working environment, social contribution, technical and economic feasibility, ease of implementation, environment friendliness, competitiveness, and further development in the future of the product. Participants are given exactly 10 days for completing and submitting their solutions.

Phase 2: Concept design

In this phase, the organizer gives participants information about one product chosen among the proposed ones. The information includes: name, description, working environment, general technical and economic requirements. Participants are asked to submit a concept design for that product. The submission should include: 3D images or sketches from different viewing angles, 2D projections, cross-sectioned drawings, explanation of its working mechanism, advantages and optimization of the design and creative product architecture. Participants are given exactly 10 days for completing and submitting their solutions.

Phase 3: Detailed engineering calculation

In this phase, the organizer gives participants a designed concept and its working conditions as well as suggestions of materials range, loading factors, etc. Participants then conduct detailed engineering calculations (mechanical engineering, electrical engineering, materials engineering, etc.) to figure out the specifications of the product (working parameters and product dimensions). The submission should include: 3D view, 2D projections, and cross-sectioned drawings with dimensions, explanation of calculation methods and detail calculation steps, detail drawings for manufacturing of product components (with tolerance) and the choice of materials. Participants are given exactly 10 days for completing and submitting their solutions.

Phase 4: Physical prototyping

In this phase, participants are given one product with detailed drawings with dimensions and materials as the result of phase 3. Participants can use existing materials (carton, plastic, rubber, wood, steel, aluminum, etc.) to build physical prototypes of that product. The submission should include: real photos of the prototype with different viewing angles, working videos (if available) and bill of materials (BOM). Participants are given exactly 10 days for completing and submitting their solutions.

Phase 5: Design evaluation

In this phase, participants are given a designed product with 3D images, 2D projections, and cross-sectioned drawings with detail dimensions and photos of physical prototype. Participants are asked to submit their comments and feedback and also suggestions about the design product, its design advantages and disadvantages so that the design team can make corrections, refinements and improvements for final ramp-up. The submitted solution should cover those kinds of information. Participants are given exactly 10 days for completing and submitting their solutions.

The reason why we give 10 days to users in all phases is that we want to make the “same condition” of the crowd, product, reward and also the time to fairly compare the crowd participation for different phases. As we observed from the experiment, for the individuals who submit, they only need at most 4 days to finish the task, even for the hardest tasks (the solutions are submitted within 4 days from the day when the open call is made). That’s why we believe that “10 days for all phases” works fine for the tasks in this experiment. Through a quick survey, those who submitted as well as those who did not submit agreed that the period of 10 days is sufficient for finishing the tasks. The reason why some individuals did not work on the problem and submit solutions might be that they compared the reward they might get and the efforts they need to make.

Crowd actions in the NPD11 contest are shown in Figure 5 below.

For each phase of the NPD11 contest, the authors record data about number of views of contest webpage (V), number of downloads of problem description sheet (D), and number of submitted solutions (S). Values of V, D, and S can be achieved from the web and email systems. Results are shown in Table 3.

The data in Table 3 above can be represented in Figure 6.

For a fair comparison of crowd participation among phases, the authors will use ratios instead of original recorded data. The ratios of D/V, S/D, and S/V are calculated. Calculated results are shown in Table 4 below.

In this section, we conducted a sample size calculation to prove that the sample in the experiment is large enough to give statistically significant output data.

4.2.1 Statistics formula for minimum sample size

The minimum sample size needed for statistically significant output data can be calculated as follows:

where SS is the minimum sample size (for infinite population), Z is Z value, which can be retrieved from Z value charts (e.g., 1.96 for 95% and 2.33 for 99% of confidence level), p is the percentage picking a choice expressed as decimal (for example, p = 0.5), and c is confidence interval expressed as decimal (e.g., 0.05 = ±5)

In this paper’s case which is provided with sample sizes, p-values, and certain confidence levels, we will consider the confidence interval of the collected data.

After applying the above formula for confidence level of 90% (usually used by the researchers), i.e., Z = 1.96, we have the following values of c and possible ranges of the S/V ratios for 5 phases as listed in Table 5 below.

Considering the weak coverage of the ranges of the S/V ratios in Table 5, we can say that with the confidence level of 90%, the data of S/V ratios are statistically significant and can be compared with ensured accuracy.

The D/V ratio lets us know how many among a certain number of individuals who are informed about the open call might be interested in the problem and willing to work further to solve the problem. The D/V ratio reflects the level of interest of the crowd in the announced problem due to the degree of complexity and the interest of the problem itself.

The experimental data show that the D/V ratio decreases when moving from phase 1 to phase 4 and increases again while moving to phase 5. This means that phases 2, 3 and 4, which require more detailed and skilled works, are less “attractive” in the eyes of the individuals from the crowd.

The S/D ratio lets us know how many among the individuals who are interested in the problem can solve the problem and submit solutions. The S/D ratio reflects the ability of the crowd to respond to the open call, and once again, this ability depends on the degree of complexity of the problem itself. The S/D ratio has the same pattern of variations over the phases like the D/V ratio.

The S/V ratio reflects the total effectiveness of the crowdsourcing process in terms of participation. By looking at the S/V ratio, companies can assess how many submissions they can receive for a certain population of the crowd that they can reach through the Internet environment. The S/V ratio shows a pattern similar to the S/D and D/V ratios over phases. This result fits well with the qualitative argument made by Sun et al. (2012) about the relationship between sustained participation of the crowd and task complexity in virtual communities.

Besides rewards, another factor that drives the participation behavior of the crowd in a certain crowdsourced NPD process is the characteristic of the phase itself. The phases with lower levels of complexity that require less efforts, skills, knowledge, etc., would attract more participants than the phases with higher levels of complexity that require more efforts, skills, knowledge, etc. From Table 4, we have Figures 8 and 9 to show how those ratios vary over phases. By looking at the charts in Figures 8 and 9, we can see that the D/V, S/D, and S/V ratios have the same pattern. They all decrease when moving from phase 1 to phase 4 and increases again while moving to phase 5.

On the website of Dell Idea Storm, a case that is mentioned by Kleemann et al. (2008) within the 52 newest submissions in October 2011, there are 32 submissions for idea generation, 8 submissions that suggest product concepts, 12 submissions for evaluating Dell’s designs; and there is neither submission of detailed engineering design nor physical prototyping. This can be explained by the results from this paper which indicate that phase 1 (idea generation) would get the most submissions, phases 2 (concept development) and 5 (design evaluation) would get less submissions than phase 1 and phases 3 (engineering design) and 4 (physical prototyping) would get the lowest quantity of submissions.

To answer the question of how crowd participation depends on phase characteristics quantitatively and to make the picture of crowd participation clearer, we conducted an experimental study on crowd participation pattern over phases of a crowdsourced NPD project in a quantitative manner. Our findings indicate that the average submission rate for phase 1 (idea generation, S/V = 0.91%) is about 10 times higher than that for phase 4 (physical prototyping, S/V = 0.09%).

According to the above discussions, the paper has suggestions for companies who want to execute crowdsourced NPD processes as follows:

Among phases, according to experimental data, phase 1 which is related to idea generation is the one that attracts the most solutions in terms of quantity. For a certain crowd, implementing phase 1 as a crowdsourcing project is more promising than other phases in terms of quantity of submitted solutions. Phases 2 and 5 can also gain a relatively high quantity of submitted solutions compared with phases 3 and 4 whose quantities are very low.

Although phases 3 and 4 have a low gain of submitted solutions, they can still be utilized because companies can expect high quality submissions from well filtered solution providers. For a company who wants to apply crowdsourcing for phases 3 or 4, the company should divide the problem into mini-tasks so that less efforts and skills from the crowd are required. This will help to encourage the participation of the crowd to the project. The company can also combine the implementation of crowdsourcing with the use of its own product development team to gain better participation and performance.

By conducting an experiment on a real crowd of engineering related individuals, this paper investigated how the crowd responds to open calls for various product design and development phases. It is found that the level of complexity of a certain NPD phase (i.e., level of efforts, skills, knowledge it requires from the crowd) is an important factor that affects the number of individuals who participate in the phase.

This paper discovered a participation pattern of crowd over NPD phases. Experimental results give a quantitative view of how the crowd participates in various phases of an NPD process that utilizes crowdsourcing. This provides experimental data for further research in this field.

This paper provides practical guidelines for companies who want to conduct crowdsourcing for their NPD projects. When companies want to get a high rate of crowd participation in their NPD projects, they should apply crowdsourcing to phases 1, 2 or 5. If they want to apply crowdsourcing to phases 3 and 4, they should devide the problems into mini-tasks so that more participation can be obtained. By understanding the crowd participation pattern, companies can choose the suitable NPD phases to apply crowdsourcing. In other words, they can build a better environment for individuals to participate in crowdsourcing projects and benefit from their participation.

There are also limitations to the results of this paper. First, in this paper, we consider only the crowd participation (i.e., quantity wise). The high participation rate might lead to a high chance of bringing innovation. Companies who want to grasp high quality submissions with crowdsourcing should consider seriously about developing filter mechanisms to filter qualified users as well as submitted contents. By this way, companies can benefit from implementing crowdsourcing for new product development. Second, in this paper, we assume that there is no correlation among crowd, product, process and reward factors. For the general case of possibly having correlations among some of these factors, another experiment is needed to investigate the general pattern.

Future work for this topic might include: 1) the relationship between crowd participation and other factors simultaneously, 2) the mathematical model that allows companies to predict crowd participation for a certain combination of {crowd, reward, product, process}, and 3) the quality of submissions from the crowd and influencing factors.