0:00

In the previous video, I mentioned that broadcasting

Â is another technique that you can use to make your Python code run faster.

Â In this video, let's delve into how broadcasting in Python actually works.

Â Let's suppose today broadcasting with an example.

Â In this matrix, I've shown the number of calories from carbohydrates,

Â proteins, and fats in 100 grams of four different foods.

Â So for example, a 100 grams of apples turns out,

Â has 56 calories from carbs, and much less from proteins and fats.

Â Whereas, in contrast, a 100 grams of beef has 104 calories from protein and

Â 135 calories from fat.

Â Now, let's say your goal is to calculate the percentage of calories

Â from carbs, proteins and fats for each of the four foods.

Â So, for example, if you look at this column and

Â add up the numbers in that column you get that 100 grams of apple

Â has 56 plus 1.2 plus 1.8 so that's 59 calories.

Â And so as a percentage the percentage of

Â calories from carbohydrates in an apple would

Â be 56 over 59, that's about 94.9%.

Â So most of the calories in an apple come from carbs, whereas in contrast,

Â most of the calories of beef come from protein and fat and so on.

Â So the calculation you want is really to sum up each of the four columns

Â of this matrix to get the total number of calories in 100 grams of apples,

Â beef, eggs, and potatoes.

Â And then to divide throughout the matrix,

Â 1:47

so as to get the percentage of calories from carbs, proteins and

Â fats for each of the four foods.

Â So the question is, can you do this without an explicit for-loop?

Â Let's take a look at how you could do that.

Â 2:04

What I'm going to do is show you how you can set,

Â say this matrix equal to three by four matrix A.

Â And then with one line of Python code we're going to sum down the columns.

Â So we're going to get four numbers corresponding to the total number

Â of calories in these four different types of foods,

Â 100 grams of these four different types of foods.

Â And I'm going to use a second line of Python code to divide each of

Â the four columns by their corresponding sum.

Â If that verbal description wasn't very clearly,

Â hopefully it will be clearer in a second when we look in the Python code.

Â So here we are in the Jupiter notebook.

Â I've already written this first piece of code to prepopulate

Â the matrix A with the numbers we had just now, so we'll hit shift enter and

Â just run that, so there's the matrix A.

Â And now here are the two lines of Python code.

Â First, we're going to compute tau equals a, that sum.

Â And x is equals 0 means to sum vertically.

Â We'll say more about that in a little bit.

Â And then print cal.

Â So we'll sum vertically.

Â Now 59 is the total number of calories in the apple, 239 was

Â the total number of calories in the beef and the eggs and potato and so on.

Â And then with a compute percentage

Â equals A/cal.reshape 1,4.

Â Actually we want percentages, so multiply by 100 here.

Â 3:40

Let's run that.

Â And so that command we've taken the matrix A and

Â divided it by this one by four matrix.

Â And this gives us the matrix of percentages.

Â So as we worked out kind of by hand just now in the apple there

Â was a first column 94.9% of the calories are from carbs.

Â Let's go back to the slides.

Â So just to repeat the two lines of code we had,

Â this is what have written out in the Jupiter notebook.

Â To add a bit of detail this parameter,

Â (axis = 0), means that you want Python to sum vertically.

Â So if this is axis 0 this means to sum vertically,

Â where as the horizontal axis is axis 1.

Â So be able to write axis 1 or sum horizontally instead of sum vertically.

Â And then this command here,

Â this is an example of Python broadcasting where you take a matrix A.

Â So this is a three by four matrix and you divide it by a one by four matrix.

Â And technically, after this first line of codes cal, the variable cal,

Â is already a one by four matrix.

Â So technically you don't need to call reshape here again, so

Â that's actually a little bit redundant.

Â But when I'm writing Python codes if I'm not entirely sure what matrix,

Â whether the dimensions of a matrix I often would just call a reshape command just to

Â make sure that it's the right column vector or the row vector or

Â whatever you want it to be.

Â The reshape command is a constant time.

Â It's a order one operation that's very cheap to call.

Â So don't be shy about using the reshape command to make sure that your matrices

Â are the size you need it to be.

Â 5:21

Now, let's explain in greater detail how this type of operation works, right?

Â We had a three by four matrix and we divided it by a one by four matrix.

Â So, how can you divide a three by four matrix by a one by four matrix?

Â Or by one by four vector?

Â 5:40

Let's go through a few more examples of broadcasting.

Â If you take a 4 by 1 vector and add it to a number, what

Â Python will do is take this number and auto-expand

Â it into a four by one vector as well, as follows.

Â And so the vector [1, 2, 3,

Â 4] plus the number 100 ends up with that vector on the right.

Â You're adding a 100 to every element, and in fact we use this form of

Â broadcasting where that constant was the parameter b in an earlier video.

Â And this type of broadcasting works with both column vectors and row vectors,

Â and in fact we use a similar form of broadcasting earlier with the constant

Â we're adding to a vector being the parameter b in logistic regression.

Â Here's another example.

Â Let's say you have a two by three matrix and

Â you add it to this one by n matrix.

Â 6:40

So the general case would be if you

Â have some (m,n) matrix here and

Â you add it to a (1,n) matrix.

Â What Python will do is copy the matrix m,

Â times to turn this into m by n matrix, so instead of this one by

Â three matrix it'll copy it twice in this example to turn it into this.

Â Also, two by three matrix and we'll add these so

Â you'll end up with the sum on the right, okay?

Â So you taken, you added 100 to the first column,

Â added 200 to second column, added 300 to the third column.

Â And this is basically what we did on the previous slide,

Â except that we use a division operation instead of an addition operation.

Â 7:34

So one last example, whether you have a (m,n) matrix and

Â you add this to a (m,1) vector, (m,1) matrix.

Â 7:47

Then just copy this n times horizontally.

Â So you end up with an (m,n) matrix.

Â So as you can imagine you copy it horizontally three times.

Â And you add those.

Â So when you add them you end up with this.

Â So we've added 100 to the first row and added 200 to the second row.

Â 8:08

Here's the more general principle of broadcasting in Python.

Â If you have an (m,n) matrix and you add or

Â subtract or multiply or divide with a (1,n) matrix,

Â then this will copy it n times into an (m,n) matrix.

Â And then apply the addition, subtraction, and

Â multiplication of division element wise.

Â 8:37

If conversely, you were to take the (m,n) matrix and add, subtract, multiply,

Â divide by an (m,1) matrix, then also this would copy it now n times.

Â And turn that into an (m,n) matrix and then apply the operation element wise.

Â Just one of the broadcasting, which is if you have an (m,1) matrix,

Â so that's really a column vector like [1,2,3], and you add,

Â subtract, multiply or divide by a row number.

Â So maybe a (1,1) matrix.

Â So such as that plus 100, then you end up copying

Â this real number n times until you'll also get another (n,1) matrix.

Â And then you perform the operation such as addition on this example element-wise.

Â And something similar also works for row vectors.

Â 9:38

The fully general version of broadcasting can do even a little bit more than this.

Â If you're interested you can read the documentation for

Â NumPy, and look at broadcasting in that documentation.

Â That gives an even slightly more general definition of broadcasting.

Â But the ones on the slide are the main forms of broadcasting that you end up

Â needing to use when you implement a neural network.

Â 10:03

Before we wrap up, just one last comment, which is for

Â those of you that are used to programming in either MATLAB or

Â Octave, if you've ever used the MATLAB or Octave function bsxfun

Â in neural network programming bsxfun does something similar, not quite the same.

Â But it is often used for similar purpose as what we use broadcasting in Python for.

Â But this is really only for very advanced MATLAB and

Â Octave users, if you've not heard of this, don't worry about it.

Â You don't need to know it when you're coding up neural networks in Python.

Â So, that was broadcasting in Python.

Â I hope that when you do the programming homework that broadcasting will allow you

Â to not only make a code run faster,

Â but also help you get what you want done with fewer lines of code.

Â 10:50

Before you dive into the programming excercise, I want to share with you just

Â one more set of ideas, which is that there's some tips and

Â tricks that I've found reduces the number of bugs in my Python code and

Â that I hope will help you too.

Â So with that, let's talk about that in the next video.

Â