1 00:00:00,316 --> 00:00:04,416 I left you with a number of questions to think about in the prior videos. 2 00:00:04,416 --> 00:00:06,676 How do we climb up the surface? 3 00:00:06,676 --> 00:00:08,586 Which direction should we go up? 4 00:00:08,886 --> 00:00:10,816 And how big should our steps be? 5 00:00:11,476 --> 00:00:14,756 How are we going to handle the nonlinear surface as we climb? 6 00:00:15,156 --> 00:00:17,026 And, when do we stop? 7 00:00:17,356 --> 00:00:20,406 Because remember, we don't have a map of what the surface looks like. 8 00:00:20,846 --> 00:00:23,426 So we need a way to know that we've reached the peak. 9 00:00:23,606 --> 00:00:29,486 Before we get start answering those questions, please take a look again at the course logo. 10 00:00:29,816 --> 00:00:32,246 It's going to show you what we've been working towards. 11 00:00:32,306 --> 00:00:36,306 The answers to all of those questions can be provided 12 00:00:36,306 --> 00:00:38,816 with a single, comprehensive case study. 13 00:00:39,666 --> 00:00:42,076 This case study runs for the rest of the module, 14 00:00:42,076 --> 00:00:45,386 and is split over several videos, starting with this one. 15 00:00:45,386 --> 00:00:50,016 You have all the tools now to climb the mountain, to optimize your system. 16 00:00:50,016 --> 00:00:51,186 Let's get going! 17 00:00:52,136 --> 00:00:56,366 In this case study, we are considering the manufacture of a mass produced product. 18 00:00:56,476 --> 00:00:59,346 For example, it could be plastic parts. 19 00:00:59,576 --> 00:01:05,126 Maybe it's frozen food such as pizza, petroleum, cars, metals, electronics. 20 00:01:05,566 --> 00:01:09,446 In fact, this example can apply to most anything that you can imagine. 21 00:01:10,016 --> 00:01:13,426 When goods are manufactured, a company can vary its production rate, 22 00:01:13,806 --> 00:01:16,586 measured as the number of parts per unit time. 23 00:01:17,226 --> 00:01:22,896 We call this throughput, and it will be factor T. Another variable the company can change is 24 00:01:22,896 --> 00:01:23,906 the selling price. 25 00:01:24,566 --> 00:01:27,556 This is our price measured in "dollars per unit". 26 00:01:28,586 --> 00:01:32,636 The outcome variable here is profit, measured as "dollars per hour". 27 00:01:33,746 --> 00:01:37,016 Now, I mentioned in a prior video, that if you're stuck thinking 28 00:01:37,016 --> 00:01:40,356 of a good outcome variable, the profit often works well. 29 00:01:41,466 --> 00:01:46,106 In this example, if we go to higher and higher throughput rates, we create more product. 30 00:01:46,666 --> 00:01:49,256 But our expenses to make that product also go up, 31 00:01:49,716 --> 00:01:52,556 we need more labour, more electricity, more material. 32 00:01:53,236 --> 00:01:56,126 And at higher throughput, we might also make more scrap. 33 00:01:56,376 --> 00:01:58,976 So, we could suffer some losses over there. 34 00:01:59,666 --> 00:02:03,116 At higher throughput, we also have more product now to sell. 35 00:02:03,116 --> 00:02:05,476 So my profit could go up, or it could go down. 36 00:02:06,086 --> 00:02:07,226 I'm really not sure. 37 00:02:08,246 --> 00:02:13,066 There is an optimally profitable point, not too fast, and not too slow. 38 00:02:13,906 --> 00:02:17,926 Another bonus is that the profit value is relatively easy to calculate 39 00:02:18,376 --> 00:02:20,996 if you have all the costs and incomes available. 40 00:02:21,566 --> 00:02:25,266 And it is relatively precise, there's a low amount of noise. 41 00:02:26,906 --> 00:02:27,816 So let's get going. 42 00:02:28,326 --> 00:02:32,246 Your company has been making parts for some time, at a production throughput 43 00:02:32,416 --> 00:02:37,356 of 325 per hour, and selling them for $0.75 per part. 44 00:02:38,086 --> 00:02:41,596 At these conditions, they make $407 per hour. 45 00:02:42,616 --> 00:02:45,356 Now let's run a full factorial around this baseline. 46 00:02:46,356 --> 00:02:49,836 We need to choose high levels and low levels for the factors. 47 00:02:50,766 --> 00:02:58,746 I'm going to pick these, 320 and 330 parts per hour for factor T, and $0.50 and $1.00 48 00:02:58,746 --> 00:03:01,716 for the price, P. How did I choose these? 49 00:03:02,226 --> 00:03:04,536 What are suitable low and high level values? 50 00:03:05,426 --> 00:03:07,986 I will give this advice, based on my experience. 51 00:03:08,646 --> 00:03:12,046 And this won't always work, but it helps answer questions 52 00:03:12,046 --> 00:03:14,116 that I see experimenters struggling with. 53 00:03:15,296 --> 00:03:18,066 The range from low to high should be large enough 54 00:03:18,256 --> 00:03:20,176 that you notice a difference in the outcome. 55 00:03:21,066 --> 00:03:25,166 If you choose values too close together, you may not notice a difference, 56 00:03:25,216 --> 00:03:26,986 and you're just really picking up noise. 57 00:03:27,416 --> 00:03:31,076 If they are too far apart, you might cover such a wide range 58 00:03:31,156 --> 00:03:34,176 that you expose the system to serious nonlinearities. 59 00:03:34,806 --> 00:03:37,746 Let me give you an example of an experiment that uses water. 60 00:03:38,476 --> 00:03:45,106 If your current baseline is 25°C, then your low level might be 15 and your high level 35°C. 61 00:03:45,196 --> 00:03:52,516 If you went too far though, say from -35°C at the low and +85°C at the high. 62 00:03:53,236 --> 00:03:58,016 Now you cover a range where water has frozen down here, and is close to boiling up there. 63 00:03:58,516 --> 00:04:00,526 It's very nonlinear over this range. 64 00:04:00,906 --> 00:04:04,486 Secondly, consider the typical ranges of the variable. 65 00:04:05,256 --> 00:04:10,256 Everything we work with is finite, and there are extreme lower and upper bounds determined 66 00:04:10,256 --> 00:04:14,316 by safety constraints, physical limitations, and just practicality. 67 00:04:15,246 --> 00:04:17,876 Let's call these the extreme lower and upper bounds. 68 00:04:18,226 --> 00:04:24,006 For example in the previous video, we looked at the height of product on a grocery store shelf. 69 00:04:24,756 --> 00:04:28,006 There were limits of zero centimeters and two centimeters. 70 00:04:29,016 --> 00:04:34,216 In this example, we cannot sell the product for less than $0.25, else we'll make a loss. 71 00:04:34,536 --> 00:04:39,036 And we have an upper limit of about $3 in mind, based on market conditions. 72 00:04:40,116 --> 00:04:43,106 Never run your first experiments at these extreme points. 73 00:04:43,646 --> 00:04:48,146 Because remember, it will likely violate the previous rule of nonlinear behaviour. 74 00:04:48,456 --> 00:04:53,666 And secondly, the whole point of this optimization is that you move outside the box. 75 00:04:54,246 --> 00:04:58,016 If you are already at the extremes of the box, you cannot move outside of it. 76 00:04:59,106 --> 00:05:05,526 And lastly, if you're really stuck, pick a starting factorial range that is about 25% 77 00:05:05,526 --> 00:05:08,096 of the extreme range you calculated in the prior point. 78 00:05:09,466 --> 00:05:14,416 In this example, our equipment cannot really go much lower than 300 parts per hour, 79 00:05:14,956 --> 00:05:18,806 and cannot exceed 350 parts per hour due to safety issues. 80 00:05:19,646 --> 00:05:22,336 These are the physical limitations in the equipment's design. 81 00:05:23,406 --> 00:05:28,666 That's an extreme range of 50 units, so 25% of that is 12.5, 82 00:05:29,006 --> 00:05:30,766 and I'm going to round that down to 10. 83 00:05:31,796 --> 00:05:40,276 So my lower limit for the starting factorial is 325 - 5, and my upper limit is 325 + 5. 84 00:05:40,716 --> 00:05:42,166 That's how I came to that range. 85 00:05:43,416 --> 00:05:49,196 For the price, I'm going to use my business knowledge of the process and try $0.50 and $1.00 86 00:05:49,196 --> 00:05:50,966 as the low and high levels, respectively. 87 00:05:51,746 --> 00:05:58,036 That corresponds to a $0.50 range, and when I compare it to the extreme range of $2.50, 88 00:05:58,466 --> 00:06:01,656 it's about 20%, so it matches that guide. 89 00:06:02,796 --> 00:06:05,236 You usually will have a good idea of what a low 90 00:06:05,236 --> 00:06:07,816 and high value should be based on your experience. 91 00:06:08,536 --> 00:06:11,536 So go ahead, use a large dose of intuition, 92 00:06:11,926 --> 00:06:14,876 and just try running a few experiments if you're not sure. 93 00:06:15,786 --> 00:06:19,846 Here's another expression from George Box that really is suitable for this situation. 94 00:06:20,496 --> 00:06:24,596 He said: "the best time to run an experiment is after an experiment". 95 00:06:25,946 --> 00:06:30,466 So let's go use that formula from class 5B that shows how to convert 96 00:06:30,466 --> 00:06:32,796 between real-world units and coded units. 97 00:06:33,686 --> 00:06:37,026 In this formula, the center point is just another name for the baseline. 98 00:06:38,066 --> 00:06:44,276 So the coded value for a low level of 320 is equal to (320 - 99 00:06:44,276 --> 00:06:49,706 325) divided by half of 10, which equals -1. 100 00:06:50,956 --> 00:06:55,336 You can prove to yourself that the coded value for the high level is equal to +1. 101 00:06:55,956 --> 00:06:59,916 And you can also try proving to yourself what the coded values for price are. 102 00:07:01,016 --> 00:07:04,246 The coded value for the baseline is trivial to calculate. 103 00:07:04,596 --> 00:07:06,436 It is at the (0, 0) point. 104 00:07:07,376 --> 00:07:11,076 So we go ahead, and run off 4 experiments here in random order, 105 00:07:11,076 --> 00:07:14,196 but we will show them in the table in standard order. 106 00:07:15,286 --> 00:07:17,136 And here are the profit values as well. 107 00:07:18,116 --> 00:07:23,766 Take a minute now, and draw a cube plot of the system, showing these five values, 108 00:07:24,186 --> 00:07:27,916 and try to draw an approximate set of contours on the plot. 109 00:07:28,496 --> 00:07:32,186 Here's the solution for you. 110 00:07:33,006 --> 00:07:36,576 We can add contours using techniques we've seen in the prior videos, 111 00:07:36,946 --> 00:07:39,066 where we connect points that have equal value. 112 00:07:39,956 --> 00:07:44,796 This time though, we have a fifth baseline point to include in our contours. 113 00:07:45,956 --> 00:07:48,926 Now we can also use computer software to draw the contours. 114 00:07:49,506 --> 00:07:53,146 To do that we need the linear model first, so let's start R 115 00:07:53,696 --> 00:07:57,686 and add these five data points and build a single linear model. 116 00:07:58,276 --> 00:08:03,126 Here's the code, we have 5 entries in the vector this time 117 00:08:03,236 --> 00:08:05,116 because of that additional baseline point. 118 00:08:05,646 --> 00:08:07,926 But otherwise, it is the same as you've seen before. 119 00:08:09,016 --> 00:08:14,096 Click the button over here to run the code and it will generate the model. 120 00:08:15,526 --> 00:08:18,446 Now here is some additional code to draw the contour plot. 121 00:08:25,526 --> 00:08:28,706 The results show you what you might have expected intuitively, 122 00:08:29,326 --> 00:08:31,686 higher production rates lead to higher profits. 123 00:08:32,406 --> 00:08:34,446 Higher prices also lead to higher profit. 124 00:08:36,636 --> 00:08:38,626 But which direction should we move in next? 125 00:08:38,976 --> 00:08:40,486 Do we increase prices more? 126 00:08:40,486 --> 00:08:42,566 Or do we increase the throughput more? 127 00:08:42,786 --> 00:08:44,966 Or do we increase both, roughly equally? 128 00:08:46,266 --> 00:08:50,156 The concept of response surface optimization is that to reach the peak 129 00:08:50,156 --> 00:08:55,606 of the mountain efficiently, we should take the fastest way up, and the fastest way 130 00:08:55,606 --> 00:08:57,786 up is the steepest path of ascent. 131 00:08:58,426 --> 00:09:02,026 Now we don't actually know what the shape of the mountain is, 132 00:09:02,416 --> 00:09:04,476 but we do have these five data points here, 133 00:09:05,176 --> 00:09:08,026 using that idea of the ski pole from a prior video. 134 00:09:09,076 --> 00:09:13,816 These five points can be used to fit a local model, and that local model, 135 00:09:14,076 --> 00:09:15,566 even though it's going to be wrong, 136 00:09:16,036 --> 00:09:18,816 will be useful in telling us how to climb up the mountain. 137 00:09:20,206 --> 00:09:26,146 Let me show you by revealing the true profit surface shown here in the dashed grey lines. 138 00:09:27,466 --> 00:09:31,176 Superimposed in blue is the local model you've just built 139 00:09:31,176 --> 00:09:36,406 in R. Now this plot looks a little bit different than the contour plot from R, 140 00:09:36,816 --> 00:09:39,926 because I've used real world units on the axes. 141 00:09:40,506 --> 00:09:42,166 Otherwise it is pretty much the same. 142 00:09:43,176 --> 00:09:47,426 As you can see, this is a pretty realistic model of the underlying surface. 143 00:09:47,766 --> 00:09:50,966 It deviates a little bit down here in the bottom left and the top right, 144 00:09:51,496 --> 00:09:54,396 but overall it's telling us the right direction to go. 145 00:09:55,136 --> 00:09:57,206 Now in practice, we don't really know 146 00:09:57,206 --> 00:10:01,016 where these true contours in the dashed grey really are. 147 00:10:02,026 --> 00:10:04,786 But let's go use the model and test its prediction ability, 148 00:10:05,516 --> 00:10:11,956 and right at the center point, where the coded values are 0 and 0 for x_T and x_P respectively, 149 00:10:12,636 --> 00:10:16,266 we can use the model and predict a value of $390. 150 00:10:17,556 --> 00:10:20,856 Notice that the actual value at the center is $407. 151 00:10:21,256 --> 00:10:23,136 That's a $17 difference. 152 00:10:24,036 --> 00:10:28,636 It gives an idea of what statisticians call goodness of fit of the model. 153 00:10:29,436 --> 00:10:33,736 In other words, it tells us how well that model approximates the true surface. 154 00:10:34,356 --> 00:10:36,176 I'm going to ask you to remember that number. 155 00:10:37,106 --> 00:10:42,346 We can, and if we have time and money, we should repeat several experiments at the baseline 156 00:10:42,506 --> 00:10:46,266 to get an idea of the reproducibility, or noise, in the system. 157 00:10:47,776 --> 00:10:51,156 Now, pause the video, and use the model to predict the values 158 00:10:51,156 --> 00:10:53,226 of the four corner points in the factorial. 159 00:10:57,366 --> 00:11:02,996 You should see that there's prediction error of about $4 to $5. 160 00:11:03,146 --> 00:11:07,676 We will come back to all these points again, but for now let's go climb that mountain. 161 00:11:08,476 --> 00:11:11,176 The prediction model tells us which direction to go up. 162 00:11:11,746 --> 00:11:16,186 And the model shows a one unit increase in the coded value of selling price, 163 00:11:16,186 --> 00:11:20,026 x_P, raises profit by $134 per hour. 164 00:11:20,026 --> 00:11:24,296 For every one unit increase in the coded value of throughput, 165 00:11:24,726 --> 00:11:27,766 we should expect a $55 increase in profit. 166 00:11:28,606 --> 00:11:31,786 That's what that coefficient in front of x_T means. 167 00:11:33,626 --> 00:11:37,846 Now, what does it mean to say to increase by "one coded unit of throughput"? 168 00:11:38,766 --> 00:11:40,186 What does that mean in the real world? 169 00:11:40,816 --> 00:11:43,176 We have to communicate our results with our colleagues, 170 00:11:43,176 --> 00:11:45,136 and they don't understand coded units. 171 00:11:45,926 --> 00:11:49,716 Here's the connection between coded units and real-world units from before. 172 00:11:50,206 --> 00:11:54,976 And below it, I've written how you can determine a change in coded units, 173 00:11:55,246 --> 00:11:57,676 with a corresponding change in real-world units. 174 00:11:58,426 --> 00:12:00,056 There are those delta symbols again. 175 00:12:01,366 --> 00:12:05,636 So to answer the question then, one coded unit change in throughput, 176 00:12:06,056 --> 00:12:09,206 corresponds to a 5 part per hour increase. 177 00:12:10,476 --> 00:12:12,686 That implies increasing production 178 00:12:12,686 --> 00:12:17,706 by 5 parts per hour will lead to an increase in profit by $55. 179 00:12:17,836 --> 00:12:23,556 Similarly, a one coded unit change in sales price corresponds 180 00:12:23,556 --> 00:12:25,886 to a $0.25 increase in the real world. 181 00:12:27,516 --> 00:12:31,766 Now we can take the steepest path up the mountain, starting at the (0, 0) baseline. 182 00:12:32,676 --> 00:12:36,886 One can prove mathematically, the steepest path is perpendicular, 183 00:12:37,166 --> 00:12:38,986 or 90 degrees, with the contour line. 184 00:12:40,356 --> 00:12:45,996 A way of saying that mathematically, is every 55 steps we increase in throughput, 185 00:12:46,626 --> 00:12:50,506 we should also increase by 134 steps in price. 186 00:12:50,846 --> 00:12:55,116 So to keep that ratio in proportion, we can write the following mathematical equation. 187 00:12:55,766 --> 00:13:03,986 To use this equation, we pick either the change in P here in the numerator, 188 00:13:04,106 --> 00:13:06,486 and solve for the denominator change in T. 189 00:13:06,486 --> 00:13:10,886 Or we can pick the denominator change and solve for the numerator. 190 00:13:11,696 --> 00:13:12,866 Either approach works. 191 00:13:13,566 --> 00:13:18,816 I'm going to pick a change in T, and going to increase it by one coded unit from the baseline. 192 00:13:19,576 --> 00:13:26,236 So then, the change in coded units for P is 134 divided by 55 times 1. 193 00:13:27,336 --> 00:13:29,686 Let's convert these changes to real world units. 194 00:13:30,496 --> 00:13:33,976 For selling price, that's an increase of $0.61 per part. 195 00:13:34,896 --> 00:13:40,416 For throughput, that is an increase of plus 5 parts per hour, using our previous formula. 196 00:13:41,076 --> 00:13:44,476 So that's my fifth experiment, over here, 197 00:13:45,296 --> 00:13:50,246 with a throughput of 330 parts per hour, and a price of $1.36. 198 00:13:51,206 --> 00:13:55,146 Notice how this is a greater change in price, than the change is in throughput, 199 00:13:55,746 --> 00:13:58,436 and that's exactly correct in what we saw in the model. 200 00:13:59,676 --> 00:14:04,106 What are the corresponding x_T and x_P values for these in coded units? 201 00:14:05,126 --> 00:14:12,236 Using the formula from before, that corresponds to x_T equals 1 and x_P equals 2.44. 202 00:14:13,206 --> 00:14:15,836 It's always good practice, before we run the experiment, 203 00:14:16,016 --> 00:14:18,016 to predict what the outcome is going to be. 204 00:14:19,096 --> 00:14:23,866 Using those coded values, we can see that it's 390 from the baseline, 205 00:14:24,606 --> 00:14:32,336 and another $327 increase due to the price, plus $55 increase due to the throughput change. 206 00:14:33,366 --> 00:14:37,986 Finally, there's a small interaction that reduces profit by $8.50 207 00:14:38,156 --> 00:14:41,716 so the total prediction here is $764. 208 00:14:42,226 --> 00:14:48,246 When I actually run the experiment though, I record a value of $669. 209 00:14:48,586 --> 00:14:50,496 That's a $95 difference. 210 00:14:51,896 --> 00:14:53,126 That's quite substantial. 211 00:14:53,416 --> 00:15:00,466 In fact, it's much larger than a single coded step in factor T, and just under the effect size 212 00:15:00,646 --> 00:15:05,026 of a coded step in factor P. That's a large deviation, 213 00:15:05,576 --> 00:15:08,236 and it indicates my model has broken down over here. 214 00:15:09,846 --> 00:15:14,936 If I keep going along this path of steepest ascent, I will end up in this general direction. 215 00:15:15,546 --> 00:15:20,626 But it really isn't worth exploring, because that direction we are climbing along is based 216 00:15:20,626 --> 00:15:23,436 on a model that we've shown needs some refitting. 217 00:15:24,556 --> 00:15:27,156 It is no different to the prior popcorn example. 218 00:15:28,346 --> 00:15:32,196 Recall in an earlier video, I mention that response surface methods are 219 00:15:32,196 --> 00:15:35,656 about sequential experimentation as we seek out the optimum. 220 00:15:36,156 --> 00:15:39,396 And the way we do that is to start a new factorial 221 00:15:39,766 --> 00:15:41,896 that better approximates the new region. 222 00:15:42,706 --> 00:15:48,556 So somewhere over here, we have to start a new factorial to refit the local surface. 223 00:15:50,046 --> 00:15:54,246 As with the first factorial, we can choose the range in real world units, 224 00:15:54,286 --> 00:15:56,986 that define the minus one and plus one positions. 225 00:15:57,746 --> 00:16:01,116 Once we pick the range, we have the center point defined as well. 226 00:16:02,456 --> 00:16:05,736 Now, there are several locations I could pick for this new factorial. 227 00:16:06,626 --> 00:16:08,916 In the course textbook, I show you one option, 228 00:16:09,276 --> 00:16:12,706 which is actually slightly further along the direction of steepest descent. 229 00:16:13,506 --> 00:16:17,476 That's a valid approach, and especially so if experiments are cheap. 230 00:16:18,526 --> 00:16:22,816 If we assume experiments are expensive, we might want to reuse as many 231 00:16:22,816 --> 00:16:25,166 of our prior factorial points as possible. 232 00:16:25,626 --> 00:16:26,706 And that's what I'm going to show. 233 00:16:26,706 --> 00:16:31,746 Place experiments 6, 7 and 8 over here. 234 00:16:31,816 --> 00:16:35,056 That decision means that my range for factor P is now fixed. 235 00:16:35,616 --> 00:16:40,396 I only have to select the range for factor T. Now, here's another tip. 236 00:16:40,916 --> 00:16:46,186 As you sense you're approaching the optimum, it is generally wise to reduce your step size. 237 00:16:46,576 --> 00:16:51,316 For two reasons, you don't want to overshoot that optimum, and secondly, 238 00:16:51,736 --> 00:16:55,536 remember the definition of an optimum is that it is at the peak of a mountain. 239 00:16:56,256 --> 00:16:58,606 Well, the peak of a mountain is definitely nonlinear. 240 00:16:59,396 --> 00:17:02,486 We want to retain a fairly linear model, if possible, 241 00:17:02,486 --> 00:17:04,836 as we climb that direction of steepest ascent. 242 00:17:05,306 --> 00:17:09,146 We're going to have to deal with nonlinearity at some point, 243 00:17:09,146 --> 00:17:10,686 and you'll see that in a coming video. 244 00:17:10,786 --> 00:17:16,876 For now, however, let's go select a smaller range here, for the throughput of eight units. 245 00:17:18,686 --> 00:17:24,356 So our low level is going to be 330 and our high level at 338 parts per hour. 246 00:17:24,356 --> 00:17:30,076 The range for factor P, the price, is already defined with a low of $1 and a high 247 00:17:30,076 --> 00:17:33,596 of $1.36, so that's a $0.36 range. 248 00:17:34,206 --> 00:17:37,016 The baseline is going to be right here, in the center. 249 00:17:38,426 --> 00:17:40,946 Now, I'm not going to show you the next steps in this video. 250 00:17:41,466 --> 00:17:44,216 I'm going to ask you to try the calculations yourself. 251 00:17:45,476 --> 00:17:49,226 Here are the outcome values from those new experiments. 252 00:17:49,586 --> 00:17:53,636 Use these data, and try to follow these steps. 253 00:17:54,696 --> 00:17:56,376 Start by visualizing the system. 254 00:17:56,736 --> 00:18:00,006 Then build a model in R, using these 5 new data points. 255 00:18:00,646 --> 00:18:03,766 Sketch some contours, either by hand, or with the software. 256 00:18:04,976 --> 00:18:07,776 Find the direction to climb up the path of steepest ascent. 257 00:18:08,546 --> 00:18:15,296 In the next video, I'm going to use a step of delta x_P of 1.5 units. 258 00:18:15,916 --> 00:18:16,756 So try that. 259 00:18:17,136 --> 00:18:19,186 What is the corresponding delta x_T? 260 00:18:20,276 --> 00:18:23,226 But if you want to use a different delta x_P, go ahead. 261 00:18:23,586 --> 00:18:26,016 Give that a try. 262 00:18:26,246 --> 00:18:30,736 What do these delta x_T and x_P values correspond to in real world units? 263 00:18:31,236 --> 00:18:32,386 Calculate what those are. 264 00:18:33,316 --> 00:18:37,036 Also predict what your next experiment's outcome is going to be. 265 00:18:37,816 --> 00:18:40,796 And finally, when you're ready to go run the experiments, 266 00:18:41,286 --> 00:18:45,816 go use this tool on the course website and run those experiments for yourself. 267 00:18:47,186 --> 00:18:50,996 If you're adventurous, go ahead and try climbing the rest of the mountain 268 00:18:50,996 --> 00:18:55,466 on your own.These experiments cost you no money and won't lead 269 00:18:55,466 --> 00:18:58,106 to any negative consequence if you get it wrong. 270 00:18:58,646 --> 00:19:01,206 This is the perfect opportunity to give that a try. 271 00:19:01,976 --> 00:19:02,906 Challenge yourself. 272 00:19:03,406 --> 00:19:09,296 Go see if the ideas from this class can be used to reach the peak and keep track of the number 273 00:19:09,296 --> 00:19:10,966 of experiments it takes you to get there. 274 00:19:12,106 --> 00:19:17,126 Also, try to answer this important question: "how do you really know that you're at the top?"