We will give a high level overview, of the concepts we have covered in this course related
to Design of Experiments for Improvement. We will also mention some topics we did not cover.
The area of experimental design, is fairly broad, and the concepts we omitted, can quickly
become very mathematical. But, if you choose to study these topics on your own afterwards,
you will see that they build on the ideas we did cover in this course.
You now have a solid foundation to base your self-learning on. All further, advanced experimental
tools build on these concepts.
We started the course by looking at some basic terminology: factors, outcomes, variables,
low and high levels, and so forth. Very quickly we learned how to interpret - visually - the
results from an experiment. And that was crucial; a visual interpretation is so important, and
not having to run and rely on software.
This is a theme we've seen throughout the course. We've resorted to visual tools the
entire way. Cube plots, to visualize the results; Pareto plots to identify, or screen out factors,
that appear important and those that are not. And finally, in the response surface module
we looked at contour plot, to visualize the surface we are moving on.
We also learned along the way, at several point how NOT to run an experiment. Changing
one factor at a time, is something we have known for at least the last 8 decades as being
inefficient, especially if we want to learn about and exploit interactions to reach that
optimum.
We learned how to set up our standard order table, to assist us. There are 2 to the k
experiments in a full factorial. And once that full factorial was run, we saw how to
manually create a simple prediction model. Remember that "high minus low", "high minus
low" idea? No software was required.
So by the end of the second module we saw that experiments often had more than one outcome.
We might want to decrease pollution as much as possible, but also do so cheaply, and obey
safety, or regulatory constraints.
In systems where this is the case, we must either reformulate our objective to include
multiple outcomes - maybe by using a weighted sum, for example - or by visualizing the overlapping,
competing criteria on two contour plots. This visual approach is, again, very effective.
We can see the trade offs in our system, and communicate with our colleagues effectively
that don't understand the terminology of response surfaces and optimization. Furthermore, if
things change in our system, we can quickly see how to compensate for them.
Now in the third module of the course, we started using software, to speed up our hand
calculations. We used a high quality, freely available tool to do that. The R software
has many packages available to extend its functionality. But there are though other
software tools, and some that are specifically designed for experimental analysis, feel free
to download their trial versions and test them out for your own needs.
We liked R, because of its traceability in the code. We can always go back, and reproduce
our results. See where we've made mistakes and even share that code with our colleagues.
You might be wondering about formal statistical tools that you might use to make your work
more analytically: such as p-values, confidence intervals, analysis of variance, and so on.
These are absolutely available, and have been there all along in the R output. As you've
seen, we've been far more reliant on visual tools in this course, and less so on detailed
statistical knowledge.
In the fourth module of the course we started to look at fractional factorials. We use these
when we have a large number of factors, and want to practically reduce the number of experiments
to some lower value. We know that there's no free lunch, and that aliasing will occur.
But we have this trade off table to help guide us in that choice.
We learned about blocking for nuisance factors, and we also covered the idea of covariates
in that fourth module. I had also mentioned the concept of definitive screening designs,
which are emerging as a more effective design than fractional factorials.
Perhaps this is a good time, to mention the book by Peter Goos and Bradley Jones. That
book starts where this course ends. It's a great book, written in conversational style,
that would help you peer into the minds of statisticians as they actually plan complex
experiments. They cover topics, that many of you have asked about: response surface
methods with categorical factors, screening designs, mixture designs, blocking and covariates,
as well as the very practical requirements of a split plot design.
Those are important topics, in practical experimentation. But, they go beyond the level we have intended
for this course. Those topics build on the concepts we have covered though.
Then we started the last module of this course: experiments to move outside our region where
we started, and seek out an optimum. We initially looked at the single factor case. Mainly because
we can easily visualize that, and illustrate the important concepts of noise, model prediction
error, lack of fit, and building and rebuilding the model as we go.
We applied those concepts to the idea of optimizing in two dimensions, and we saw a sequence of
videos on the details on how to go about that in the fifth module. Even though those last
videos were long, they covered some digressions, on the practical aspects of dealing with constraints
and making mistakes.
Now the response surface idea, expands in a natural way, to the case of three or more
variables. We can also bring in the idea of fractional factorials. This will reduce the
number of experiments required. The only thing to be aware of is aliasing. Because, remember,
as you approach the optimum, it is those interactions and quadratic nonlinear terms, that will start
to dominate. You need sufficient resolution at the optimum, and a fractional factorial
may not provide that for you.
Now this course does not end here. On the website there are some practice problems to
try out. I've also posted a list of resources that relate to the area of designed experiments.
If you come across any others, please share them with your fellow students, and make a
short post in the forums, or email me. We'll keep that list up to date.
As you might start to realize now, the topic of Designed Experiments spans into many other
application areas. Also keep posting in the forums about how you've used Designed Experiments.
This is a topic that applies to many application areas, which is one of the reasons why we
chose to teach this course.
So this is the end. I will thank specific people in the credits that follow, but by
far the biggest thanks goes to many of you, on the forums, both the current and prior
students in this course.
Your participation and questions have lead me to learn so many interesting ways of using
and applying experiments. I have made improvements to my own life, and career because of it.
Thank you also for your feedback. We keep collecting your suggestions, and we use them
for future iterations of this course.
We do ask that you take a few minutes and fill out our final survey. There are two simple
questions we would like you to answer, and there are a few other optional questions if
you have time.
So, thank you again for your time and effort. Remember to keep disturbing, and observing.