1 00:00:02,210 --> 00:00:06,540 We will give a high level overview, of the concepts we have covered in this course related 2 00:00:06,540 --> 00:00:13,200 to Design of Experiments for Improvement. We will also mention some topics we did not cover. 3 00:00:13,290 --> 00:00:18,320 The area of experimental design, is fairly broad, and the concepts we omitted, can quickly 4 00:00:18,320 --> 00:00:23,700 become very mathematical. But, if you choose to study these topics on your own afterwards, 5 00:00:23,700 --> 00:00:28,340 you will see that they build on the ideas we did cover in this course. 6 00:00:28,340 --> 00:00:34,350 You now have a solid foundation to base your self-learning on. All further, advanced experimental 7 00:00:34,350 --> 00:00:37,379 tools build on these concepts. 8 00:00:37,379 --> 00:00:42,809 We started the course by looking at some basic terminology: factors, outcomes, variables, 9 00:00:42,809 --> 00:00:47,850 low and high levels, and so forth. Very quickly we learned how to interpret - visually - the 10 00:00:47,850 --> 00:00:53,999 results from an experiment. And that was crucial; a visual interpretation is so important, and 11 00:00:53,999 --> 00:00:57,649 not having to run and rely on software. 12 00:00:57,649 --> 00:01:01,460 This is a theme we've seen throughout the course. We've resorted to visual tools the 13 00:01:01,460 --> 00:01:07,640 entire way. Cube plots, to visualize the results; Pareto plots to identify, or screen out factors, 14 00:01:07,640 --> 00:01:12,390 that appear important and those that are not. And finally, in the response surface module 15 00:01:12,390 --> 00:01:16,439 we looked at contour plot, to visualize the surface we are moving on. 16 00:01:16,439 --> 00:01:21,680 We also learned along the way, at several point how NOT to run an experiment. Changing 17 00:01:21,680 --> 00:01:26,430 one factor at a time, is something we have known for at least the last 8 decades as being 18 00:01:26,430 --> 00:01:31,200 inefficient, especially if we want to learn about and exploit interactions to reach that 19 00:01:31,200 --> 00:01:33,009 optimum. 20 00:01:33,009 --> 00:01:37,590 We learned how to set up our standard order table, to assist us. There are 2 to the k 21 00:01:37,590 --> 00:01:42,920 experiments in a full factorial. And once that full factorial was run, we saw how to 22 00:01:42,920 --> 00:01:47,820 manually create a simple prediction model. Remember that "high minus low", "high minus 23 00:01:47,820 --> 00:01:51,960 low" idea? No software was required. 24 00:01:51,960 --> 00:01:57,030 So by the end of the second module we saw that experiments often had more than one outcome. 25 00:01:57,030 --> 00:02:02,259 We might want to decrease pollution as much as possible, but also do so cheaply, and obey 26 00:02:02,259 --> 00:02:05,179 safety, or regulatory constraints. 27 00:02:05,179 --> 00:02:09,390 In systems where this is the case, we must either reformulate our objective to include 28 00:02:09,390 --> 00:02:16,060 multiple outcomes - maybe by using a weighted sum, for example - or by visualizing the overlapping, 29 00:02:16,060 --> 00:02:22,600 competing criteria on two contour plots. This visual approach is, again, very effective. 30 00:02:22,600 --> 00:02:26,840 We can see the trade offs in our system, and communicate with our colleagues effectively 31 00:02:26,840 --> 00:02:33,050 that don't understand the terminology of response surfaces and optimization. Furthermore, if 32 00:02:33,050 --> 00:02:38,160 things change in our system, we can quickly see how to compensate for them. 33 00:02:38,160 --> 00:02:42,330 Now in the third module of the course, we started using software, to speed up our hand 34 00:02:42,330 --> 00:02:48,460 calculations. We used a high quality, freely available tool to do that. The R software 35 00:02:48,460 --> 00:02:53,240 has many packages available to extend its functionality. But there are though other 36 00:02:53,240 --> 00:02:58,890 software tools, and some that are specifically designed for experimental analysis, feel free 37 00:02:58,890 --> 00:03:03,150 to download their trial versions and test them out for your own needs. 38 00:03:03,150 --> 00:03:08,290 We liked R, because of its traceability in the code. We can always go back, and reproduce 39 00:03:08,290 --> 00:03:14,200 our results. See where we've made mistakes and even share that code with our colleagues. 40 00:03:14,200 --> 00:03:18,060 You might be wondering about formal statistical tools that you might use to make your work 41 00:03:18,060 --> 00:03:24,450 more analytically: such as p-values, confidence intervals, analysis of variance, and so on. 42 00:03:24,450 --> 00:03:29,540 These are absolutely available, and have been there all along in the R output. As you've 43 00:03:29,540 --> 00:03:34,190 seen, we've been far more reliant on visual tools in this course, and less so on detailed 44 00:03:34,190 --> 00:03:36,870 statistical knowledge. 45 00:03:36,870 --> 00:03:41,290 In the fourth module of the course we started to look at fractional factorials. We use these 46 00:03:41,290 --> 00:03:46,290 when we have a large number of factors, and want to practically reduce the number of experiments 47 00:03:46,290 --> 00:03:52,370 to some lower value. We know that there's no free lunch, and that aliasing will occur. 48 00:03:52,370 --> 00:03:56,650 But we have this trade off table to help guide us in that choice. 49 00:03:56,650 --> 00:04:01,120 We learned about blocking for nuisance factors, and we also covered the idea of covariates 50 00:04:01,120 --> 00:04:06,370 in that fourth module. I had also mentioned the concept of definitive screening designs, 51 00:04:06,370 --> 00:04:11,070 which are emerging as a more effective design than fractional factorials. 52 00:04:11,070 --> 00:04:16,190 Perhaps this is a good time, to mention the book by Peter Goos and Bradley Jones. That 53 00:04:16,190 --> 00:04:21,479 book starts where this course ends. It's a great book, written in conversational style, 54 00:04:21,479 --> 00:04:26,159 that would help you peer into the minds of statisticians as they actually plan complex 55 00:04:26,159 --> 00:04:30,789 experiments. They cover topics, that many of you have asked about: response surface 56 00:04:30,789 --> 00:04:36,340 methods with categorical factors, screening designs, mixture designs, blocking and covariates, 57 00:04:36,340 --> 00:04:40,819 as well as the very practical requirements of a split plot design. 58 00:04:40,819 --> 00:04:46,219 Those are important topics, in practical experimentation. But, they go beyond the level we have intended 59 00:04:46,219 --> 00:04:51,280 for this course. Those topics build on the concepts we have covered though. 60 00:04:51,280 --> 00:04:56,229 Then we started the last module of this course: experiments to move outside our region where 61 00:04:56,229 --> 00:05:02,099 we started, and seek out an optimum. We initially looked at the single factor case. Mainly because 62 00:05:02,099 --> 00:05:07,169 we can easily visualize that, and illustrate the important concepts of noise, model prediction 63 00:05:07,169 --> 00:05:12,310 error, lack of fit, and building and rebuilding the model as we go. 64 00:05:12,310 --> 00:05:17,300 We applied those concepts to the idea of optimizing in two dimensions, and we saw a sequence of 65 00:05:17,300 --> 00:05:22,249 videos on the details on how to go about that in the fifth module. Even though those last 66 00:05:22,249 --> 00:05:27,449 videos were long, they covered some digressions, on the practical aspects of dealing with constraints 67 00:05:27,449 --> 00:05:30,229 and making mistakes. 68 00:05:30,229 --> 00:05:34,699 Now the response surface idea, expands in a natural way, to the case of three or more 69 00:05:34,699 --> 00:05:40,259 variables. We can also bring in the idea of fractional factorials. This will reduce the 70 00:05:40,259 --> 00:05:46,580 number of experiments required. The only thing to be aware of is aliasing. Because, remember, 71 00:05:46,580 --> 00:05:52,499 as you approach the optimum, it is those interactions and quadratic nonlinear terms, that will start 72 00:05:52,499 --> 00:05:56,990 to dominate. You need sufficient resolution at the optimum, and a fractional factorial 73 00:05:56,990 --> 00:05:59,639 may not provide that for you. 74 00:05:59,639 --> 00:06:04,249 Now this course does not end here. On the website there are some practice problems to 75 00:06:04,249 --> 00:06:10,159 try out. I've also posted a list of resources that relate to the area of designed experiments. 76 00:06:10,159 --> 00:06:14,389 If you come across any others, please share them with your fellow students, and make a 77 00:06:14,389 --> 00:06:20,639 short post in the forums, or email me. We'll keep that list up to date. 78 00:06:20,639 --> 00:06:24,439 As you might start to realize now, the topic of Designed Experiments spans into many other 79 00:06:24,439 --> 00:06:30,650 application areas. Also keep posting in the forums about how you've used Designed Experiments. 80 00:06:30,650 --> 00:06:34,740 This is a topic that applies to many application areas, which is one of the reasons why we 81 00:06:34,740 --> 00:06:37,120 chose to teach this course. 82 00:06:37,120 --> 00:06:41,939 So this is the end. I will thank specific people in the credits that follow, but by 83 00:06:41,939 --> 00:06:47,360 far the biggest thanks goes to many of you, on the forums, both the current and prior 84 00:06:47,360 --> 00:06:49,620 students in this course. 85 00:06:49,620 --> 00:06:54,219 Your participation and questions have lead me to learn so many interesting ways of using 86 00:06:54,219 --> 00:07:01,240 and applying experiments. I have made improvements to my own life, and career because of it. 87 00:07:01,240 --> 00:07:06,219 Thank you also for your feedback. We keep collecting your suggestions, and we use them 88 00:07:06,219 --> 00:07:09,099 for future iterations of this course. 89 00:07:09,099 --> 00:07:13,949 We do ask that you take a few minutes and fill out our final survey. There are two simple 90 00:07:13,949 --> 00:07:18,620 questions we would like you to answer, and there are a few other optional questions if 91 00:07:18,620 --> 00:07:19,909 you have time. 92 00:07:19,909 --> 00:08:08,180 So, thank you again for your time and effort. Remember to keep disturbing, and observing.