7 Study 2 - ranking by experts
From EvolutionaryProductDevelopment
7.1 Pilot-study
This research was carried out in two parts. In the first part, the pilot-study, with four subjects a test was undertaken in order to find out if the framework of the research would function well, if the instructions were clear and if the statements (see paragraph 6) were well formulated and understandable. The subjects were (assistant) professors of the Industrial Design Engineering courses at the Universities of Delft and Twente. All subjects were familiar with the theory of product phases.
In both the pilot-study and the main study statements were printed on stickers. They were collected based on product characteristics. These collections were offered to the subjects in a changing, random order. The statements within the collections were randomised as well. The subjects were asked to arrange the statements in historical order, and to attach the stickers in the order they expect them to take place (or order of importance) in the product life cycle. It was possible to let statements take place simultaneously or on several different occasions in time. In the pilot-study, the subjects were asked about their meaning with regards to the design, the execution and the comprehensibility of the test. Based on their comments some formulations were adapted and two extra statements were added.
7.2 Main study
For the main study, subjects were selected from a population of experienced industrial designers, design managers and marketing managers because it was expected that they have enough knowledge of product life cycles and are therefore able to work with the statements about the product characteristics. The aim was to select a minimum of seventy subjects to be able to draw statistically reliable conclusions. Due to the fact that it would take a lot of time to approach these persons one by one, make appointments with them and then visit them, the approach was modified in an attempt to combine the experiment with the events whereby members of the target group would meet. In this way, four sessions could be organised that delivered sixty four subjects. Another seven subjects, all experienced industrial design engineers were then approached separately, bringing the total number of subjects to seventy one.
7.3 Working method
The working method in the main study was as follows. Each subject received, in random order, ten folders with indications of the contents: newness, functionality, product development, styling, number of competitors, pricing, production, promotion, service and ethics. Each folder contained between two and ten sheets with stickers with the statements as described in section 6, where each sheet had a statement printed on it fourteen times. This was done in order to make it clear that it is possible and allowed to affix a sticker more than once. Overall each subject received 49 sheets with each 14 stickers, bringing the total amount of stickers to 686. Next to that, they received a piece of paper measuring 50 by 73 centimetres with indications of the time the product is on the market and the market penetration of the products. They could affix the stickers on this piece of paper (the 'field of play').
7.4 Composition of the group of experts
As described in section 4, 'Research questions', it was the intention to have experts judge and arrange the statements, and since the statements were about both product development and marketing, an attempt was made to find experts from both groups. The intention was to have a division of 50/50. This effort has not been successful, since the selected group had more than twice the number of product designers than marketers. Nevertheless, it was expected that the knowledge of marketing within the group was adequate as more than half of the subjects said that they were managers too. Managers in product development are supposed to have at least some knowledge of marketing. The level of education confirms this: 95% of the subjects studied at a university or a college of advanced technology, as the syllabus is heavily biased towards marketing. It was also the intention to have experienced subjects participate in this study. This effort has been very successful: 56% had more than five years experience, 77% more than two years.
7.5 Interpretation of the results
Based on the position of the stickers, a decision was made as to which product phase was related to the statement printed on it. In section 6, 'Product characteristics', figure 6.1 indicates which statement belongs to which product phase. Another thing that was indicated is that there are some statements that typify one product phase, others two or three or even a maximum of four product phases. The following criteria were used to decide if a statement can significantly be linked to a product phase. If a statement typifies only one product phase and 17% (rounded off upwards) of the stickers were affixed in the column of this phase, this can be called 'coincidence'. (If the stickers would have been affixed randomly this would give the same result.) If a statement typifies one product phase, and 45% of the stickers have been affixed in the column of this product phase, then the conclusion will be that the statement is significantly linked to the product phase. If the number of stickers linked to the product phase is larger (or equals) 36% and is smaller than 45% the link will be called a strong indication, and if the number is between 28% and 35% it will be called an indication, etcetera. In figure 7.1 an overview is given of the criteria that were used to decide whether a link is significant, has a strong indication, an indication, a weak indication, has coincidence or is denied for statements belonging to one, two, three or four product phases.
Figure 7.1 Overview of the criteria used to decide if a statement typifies a product phase in a significant number of the cases.
7.6 Justification of the criteria
The criteria of figure 7.1 were defined in the following way. A statement can fit to a maximum of four product phases. Therefore, the situation where this is the case was looked at first. A demand that 95% of the stickers are affixed to these four product phases seems necessary, because if 'coincidence' would be the case, then 67% of the stickers would already be linked to these phases. Finally, this demand proved to be too high, since none of the statements could actually meet this demand. For each of the lower product phases, the chance of coincidence is reduced by 16,667%. It therefore seems realistic to minimize the criterion of significance with the same steps. In this way the percentages of 45, 62 and 78 were determined. Next, the difference between the situation of 'coincidence' and 'significant' was divided into three equal steps that were named 'strong indication', 'indication' and 'weak indication'.
Based on the criteria formulated above, the presumed relationship between a statement and the associated product phases were significant in 17 cases, and in 13 cases there was a strong indication, on three occasions there was an indication, on ten occasions there was a weak indication, on five occasions no evidence of a relationship was found ('coincidence') and only on one occasion the result found that there was no relationship at all.
7.7 Conclusions regarding the product phases
Figure 7.2 indicates per product phase to what measure the statement describe the product phases correctly.
| Product characteristics | Performance | Optimisation | Itemisation | Segmentation | Individualisation | Awareness |
|---|---|---|---|---|---|---|
| 1 The product is new to the market. | +++ | +++ | ||||
| 2 The product is known (but not well known) within the target group. | +++ | +++ | ||||
| 3 The product is well known within the target group. | +++ | +++ | +++ | |||
| 4 The market penetration of the product is high. | +++ | +++ | +++ | |||
| 5 The performance of the product is poor. | +++ | +++ | ||||
| 6 The product originates from a 'technology push'. | +++ | +++ | ||||
| 7 Comparatively the product has many parts. | +++ | +++ | ||||
| 8 The performance of the product is acceptable. | +++ | |||||
| 9 The performance and reliability of the product are good. | ++ | ++ | ++ | ++ | ||
| 10 The product is easy to handle and meets the ergonomic demands. | ~ | ~ | ~ | ~ | ||
| 11 The product is safe. | - | - | - | - | ||
| 12 The product offers much choice - there is a large assortment. | +++ | +++ | +++ | |||
| 13 The user is interested in adaptations of the product to extend the product life cycle (instead of discarding the product to buy a newer one). | ~ | |||||
| 14 The competitor can distinguish himself with 'positive aging': the product becomes more attractive to use. | ++ | |||||
| 15 Product development is mainly aimed at improving of the performance of the product. | ++ | ++ | ||||
| 16 Product development is aimed at products that have a better performance, are easier to handle, and have improved reliability, ergonomics and safety. | ~ | ~ | ||||
| 17 Product development is aimed at extra features and accessories. | ++ | ++ | ++ | |||
| 18 Product development is aimed at different products for different market channels or target groups. | + | + | + | + | ||
| 19 Product development is aimed at the possibility for the user to influence the result by choosing from extra features or to have the product assembled to meet his demands. | ~ | ~ | ||||
| 20 Styling is not very important. | ++ | ++ | ||||
| 21 There is not much unity in the styling of the parts of the product. | +++ | +++ | ||||
| 22 The styling of the parts of the product (integration of form) is good. | + | + | + | + | ||
| 23 The styling of the product is expressive. | o | o | ||||
| 24 The competitor can distinguish himself with a well cared for design that is also simple and sober. | o | |||||
| 25 There is only one or there are very few competitors. | +++ | +++ | ||||
| 26 There are several competitors (but not many). | ++ | ++ | ||||
| 27 There are a lot of competitors, and the market is highly competitive. | ++ | ++ | ++ | |||
| 28 The price of the product (per unity) is relative high and people find the product expensive. | +++ | +++ | ||||
| 29 The pricing of products is competitive. | ~ | ~ | ||||
| 30 There is a lot of competition, prices are under pressure and are going down. | ++ | ++ | ||||
| 31 Prices have reached their lowest possible level. | +++ | +++ | +++ | |||
| 32 Prices vary because the products are customised. | o | o | ||||
| 33 The product is designed for production with standard machines, such as lathes, and milling-, trimming-, bending- and welding machines. | ++ | ++ | ||||
| 34 Assembly of the product is mainly done by hand. | +++ | +++ | ||||
| 35 The number of parts of the product decreases, and automation becomes more important. | ++ | ++ | ||||
| 36 Assembly of the product is highly automated. | +++ | +++ | +++ | |||
| 37 Production is highly automated. | ++ | ++ | ++ | |||
| 38 Promotion is mainly based on free publicity and trade fairs. | ++ | ++ | ||||
| 39 Promotion is done on a small scale: trade fairs, the internet, brochures with retailers, etc. | +++ | +++ | ||||
| 40 Promotion is done through advertising in magazines and papers and/or on radio and TV. | ~ | ~ | ||||
| 41 Promotion is done through direct marketing. | ~ | ~ | ||||
| 42 Promotion activities are intensive: a lot of advertising in many different media. | ~ | ~ | ~ | |||
| 43 Interactive media are used to attune the product to the wishes of the individual user. | o | o | ||||
| 44 The user communicates directly with the competitor to make his individual wishes known. | o | o | ||||
| 45 The competitor communicates about the ethics of his company. | ~ | |||||
| 46 There is no well organised service organisation. (This does not mean that the service is poor.) | +++ | +++ | ||||
| 47 There is a well organised service organisation supporting the product. | ++ | ++ | ++ | ++ | ||
| 48 The ethics of the competitor (manufacturer) is not very important for the decision of the user. | ~ | ~ | ~ | ~ | ||
| 49 The competitor (manufacturer) can distinguish himself from the competition by its social behaviour, for example with regard to the environment or child labour. | + | + |
Figure 7.2 Significance of the statements.
+++ = Significant; ++ = Strong indication; + = Indication; ~ = Weak indication; o = Coincidence; - = Denied
7.7.1 Performance
With the exception of the statement on the social behaviour of the competitor (see statement 48, figure 7.2) it can be concluded that the product characteristics describe the product performance phase very well.
7.7.2 Optimisation
With the exception of the statements on product development (see statement 16, figure 7.2), pricing (statement 29) and the social behaviour of the competitor (statement 48) it can be concluded that the product characteristics describe the product optimisation phase quite well.
7.7.3 Itemisation
The product characteristics only reasonably describe the product itemisation phase and statements regarding the performance of the product (10 and 11), product development (16), pricing (29) and promotion (40 and 41) give a different picture. Also, the statement about the social behaviour is not correct, but that is the case for all of the product phases.
7.7.4 Segmentation
The product characteristics describe the product segmentation phase in the range of 'reasonable' to 'well'. The statements regarding the promotion (40, 41 and 42), the safety (11), and the styling of the product (23) give different results. Once again, the statement about social behaviour is not correct.
7.7.5 Individualisation
The product characteristics describe the product individualisation phase with mixed success. The statements concerning the performance, product development and styling are precarious. The statements regarding the newness, the market, the production and the service have proved to be correct. The statements regarding promotion and social behaviour of the competitor are once again, inadequate.
7.7.6 Awareness
The product characteristics do not adequately describe the product awareness phase. In particular, the characteristics that typify the product phase, such as a sober design (statement 24) and the importance of ethics (statements 13 and 45) are hardly recognised by the subjects. The statements that match with this phase are the ones that also match with the individualisation phase: newness, production, service and (only partially) the market.
7.8 Conclusion
Based on the classification by the experts, of the forty nine statements it can be concluded that they describe the product characteristics, and with that the product phases with mixed results. The first two product phases are described in the range of 'well' to 'very well'. From the statement about the product performance phase, a value of 93% is confirmed by the experts. For optimisation, this percentage reaches 85. The next four phases are not described so well. Itemisation has the lowest score, as only 56% of the statements are confirmed by the experts. For segmentation, the percentage is 67, for individualisation it is 62% and for awareness it is 57%. It should be noted that the experts only deny a statement once, and that any other statements that are not confirmed are not denied either.
| Product phase | Number of statements (1) | Number confirmed (2) | Percentage |
|---|---|---|---|
| Performance | 15 | 14 | 93% |
| Optimisation | 20 | 17 | 85% |
| Itemisation | 16 | 9 | 56% |
| Segmentation | 21 | 14 | 67% |
| Individualisation | 21 | 13 | 62% |
| Awareness | 23 | 13 | 57% |
Figure 7.3 The percentage of the statements that were confirmed by the experts per product phase.
(1) The number of statements that are concerned with this product phase.
(2) The number of statements that were considered by the experts to relate to this product phase.
| Product characteristic | Number | Number of statements (1) | Number confirmed (2) | Percentage |
|---|---|---|---|---|
| Product | ||||
| Newness | 1-4 | 10 | 10 | 100% |
| Functionality | 5-14 | 24 | 15 | 63% |
| Product development | 15-19 | 13 | 9 | 69% |
| Styling | 20-24 | 11 | 8 | 73% |
| Market | ||||
| Number of competitors | 25-27 | 7 | 7 | 100% |
| Pricing | 28-32 | 11 | 7 | 64% |
| Production | 33-37 | 12 | 12 | 100% |
| Promotion | 38-45 | 16 | 4 | 25% |
| Service | 46-47 | 6 | 6 | 100% |
| Ethics | 48-49 | 6 | 2 | 33% |
Figure 7.4 The percentage of the statements that were confirmed by the experts per product characteristic.
(1) The number of statements that are concerned with this product characteristic.
(2) The number of statements that were considered by the experts to be concerned with this product characteristic.
In general, it can be concluded that the product characteristics pertaining to the product, such as: newness, functionality, product development and form giving, describe the product phases very well. The same can be said about the product characteristics pertaining to the market (the number of competitors and price), production and service. According to the experts, the two remaining product characteristics, 'promotion' and 'ethics' do not have adequate enough descriptions in order to base a conclusion.
