Because Design of experiments (DOEs) are so powerful, yet often costly in terms of time and resources, the experimenter should be extra careful in their design and development:
⚠️ 𝗠𝗶𝘀𝘁𝗮𝗸𝗲 #𝟭: 𝗡𝗼𝘁 𝘂𝗻𝗱𝗲𝗿𝘀𝘁𝗮𝗻𝗱𝗶𝗻𝗴 𝘁𝗵𝗲 𝘀𝗰𝗶𝗲𝗻𝗰𝗲.
“𝘛𝘩𝘦𝘳𝘦 𝘪𝘴 𝘯𝘰 𝘴𝘶𝘣𝘴𝘵𝘪𝘵𝘶𝘵𝘦 𝘧𝘰𝘳 𝘬𝘯𝘰𝘸𝘭𝘦𝘥𝘨𝘦.” - W. Edwards Deming.
All the statistics in the world are no substitute for a basic understanding of the physics, chemistry, biology, manufacturing, technology, etc. that is the subject of the experiment. Research, interview SMEs, and learn as much as you can about the subject 𝘣𝘦𝘧𝘰𝘳𝘦 you plan the DOE.
⚠️ 𝗠𝗶𝘀𝘁𝗮𝗸𝗲 #𝟮: 𝗥𝗲-𝗶𝗻𝘃𝗲𝗻𝘁𝗶𝗻𝗴 𝘁𝗵𝗲 𝘄𝗵𝗲𝗲𝗹.
“𝘐𝘵 𝘪𝘴 𝘣𝘦𝘵𝘵𝘦𝘳 𝘵𝘰 𝘭𝘦𝘢𝘳𝘯 𝘧𝘳𝘰𝘮 𝘵𝘩𝘦 𝘮𝘪𝘴𝘵𝘢𝘬𝘦𝘴 𝘰𝘧 𝘰𝘵𝘩𝘦𝘳𝘴 𝘵𝘩𝘢𝘯 𝘵𝘰 𝘸𝘢𝘪𝘵 𝘶𝘯𝘵𝘪𝘭 𝘺𝘰𝘶 𝘮𝘢𝘬𝘦 𝘵𝘩𝘦𝘮 𝘺𝘰𝘶𝘳𝘴𝘦𝘭𝘧.”
Very likely someone has run a similar experiment in the past: what did they find? What factors were most important? What errors did they make? Treat these as potentially valuable contributions to your own DOE.
⚠️ 𝗠𝗶𝘀𝘁𝗮𝗸𝗲 #𝟯: 𝗢𝗺𝗶𝘁𝘁𝗶𝗻𝗴 𝘁𝗵𝗲 𝗖𝗮𝘂𝘀𝗲-𝗮𝗻𝗱-𝗘𝗳𝗳𝗲𝗰𝘁 𝗱𝗶𝗮𝗴𝗿𝗮𝗺.
Root cause analysis brainstorming tools such as Cause-and-Effect diagrams, Y-to-X, Fault-Tree Analysis, etc. will help you consider a larger universe of possible inputs and assist in separating “the vital few from the trivial many”.
⚠️ 𝗠𝗶𝘀𝘁𝗮𝗸𝗲 #𝟰: 𝗖𝗵𝗼𝗼𝘀𝗶𝗻𝗴 𝘁𝗵𝗲 𝘄𝗿𝗼𝗻𝗴 𝗶𝗻𝗽𝘂𝘁 𝗹𝗲𝘃𝗲𝗹𝘀.
Be bold! Too often the results will not show significance – not because an input is not a factor - but because the 𝘳𝘢𝘯𝘨𝘦 of input levels used was 𝘵𝘰𝘰 𝘴𝘮𝘢𝘭𝘭.
⚠️ 𝗠𝗶𝘀𝘁𝗮𝗸𝗲 #𝟱: 𝗦𝗸𝗶𝗽𝗽𝗶𝗻𝗴 𝗰𝗲𝗻𝘁𝗲𝗿 𝗽𝗼𝗶𝗻𝘁 𝗿𝘂𝗻𝘀.
The most underrated of all runs, center points are crucial in detecting process drift, curvature, and giving us the extra degrees of freedom to calculate model residual errors. A must!
⚠️ 𝗠𝗶𝘀𝘁𝗮𝗸𝗲 #𝟲: 𝗡𝗼𝘁 𝗸𝗻𝗼𝘄𝗶𝗻𝗴 𝘁𝗵𝗲 𝗘𝗳𝗳𝗲𝗰𝘁 𝘀𝗶𝘇𝗲 𝘆𝗼𝘂’𝗿𝗲 𝘁𝗿𝘆𝗶𝗻𝗴 𝘁𝗼 𝗱𝗲𝘁𝗲𝗰𝘁.
The most common – and often disastrous – mistake. Without the Effect size you won’t know if the Sample Size is sufficient. And without that, the study will most likely be underpowered and/or fail to detect significant factors.
…All this and we haven’t run the experiment yet! Stay tuned for more...