Welcome back. Let's review what we have so far in our app.
We've created our app component that contains a few custom react components,
that we've built. And in the last video we were able to finally get our response from the 'Clarifai' API. If we
go back to 'Clarifai', we see that their face detection has a nice little square box that it detects.
But if you look at the API and the response that we receive, we actually don't get that square box.
All we get are these bounding box. And based on the face and how many faces it detects,
it gives you these bounding boxes.
And you can read through it, over here.
But these are just saying the, the top row is at 22 percent of the image.
So these are percentages that we can use. But in order for us to create our own box like this,
well, we need to just see, what styles are using but doing 'inspect'.
As you remember, we can see that they have a 'bounding box' clause here.
So our app we can do something similar and then we'll just have to calculate based on the response where
we can create these boxes because well, just having the number display to our users doesn't mean much,
it's nice to have this facebox.
So let's work on that in this video.
If we go back to our app, we first want to say that well, we're for sure going to add a new 'state' and
we'll call it 'box'. And this 'box' will just be an empty object to start off with.
And this 'box' will contain the values that we receive.
In the case of a response,
it'll be the 'top row', 'left column', 'bottom row' and then 'right column'.
So how can we do this?
This requires a bit of thinking on our part, but we can start off with creating a function. And this function
can say 'calculateFaceLocation'
and this 'calculateFaceLocation' will receive some 'data'.
And for now we'll just keep it as an empty function.
But we know that, we want to call this function 'calculateFaceLocation' based on the inputs that we get
from 'Clarifai'.
So in our case, with the response that they give us, we want to use this response, and call
'calculateFaceLocation' with the 'response' data.
And again because we're using classes here it will have to do,
'this' dot.
I also think this function looks a little bit messy right now.
So I'm going to move a few things around.
First off using ES6 we can just do arrow functions here.
That's much better.
And instead of using this 'error(err)', we can actually with a 'promise' whenever you have a dot 'then' you can do
a dot 'catch'.
And that is an 'err'(error) in case something fails.
In our case, we can just 'console dot log'.
This 'err'(error).
So this already looks a little bit cleaner.
But we need to define, oop!
and I forgot a bracket here, and now we also don't need this. OK.
So we know that this is going to get a 'response' which is the bounding box.
But this 'response' will be received while we console logged last time, which is getting the 'bounding box'.
For now we're not going to worry about having multiple faces in the picture we're just going to grab
the first 'bounding box'. Again if we go back to the 'Clarify' API, we see that depending on the regions we
actually get an array.
So we had multiple people in the image will have multiple faces.
In our case, we're just worrying about one, just trying to get one 'bounding box' around a face.
So we want to change this 'response' to now say 'data' and we can just have a constant called
'clarifaiFace' equals
this 'data', I'm going to make this a little bit smaller so we can see. Now within here, what do we want
to do first.
Well, we're going to want to do some DOM manipulation.
So the best thing we can do is say 'const image' equals 'document' dot 'getElementById' and we can create
an element or we can create an id called 'inputimage' and this 'imputimage' that we're grabbing will
be here.
This image that links the 'imageUrl'. So the image that gets displayed in our app. So we can create
an 'id' here and we'll say 'image' or 'inputimage'.
So now this 'inputimage' is right here we grabbed it and we want to, as you remember this 'bounding box'
is a percentage of the image.
So if it's point two two(.22) that means it's at 20 percent of whatever the height or width of the images.
In our case, we can say that we for sure want to have the width of our image by saying the 'image' dot 'width'.
And because this is going to be a string and we want to do some calculations on it we'll wrap it in
a number.
This way we make sure that it is a number, and then we'll do 'height' as well of the image that we just
grabbed.
And you know, we can just console log now to see that we've been doing everything right, 'width' and 'height'.
So let's try that. I'm going to save, go back to the image or to the app, grab this sample image, place it
in, press 'Detect' and let's look at the console.
'getelement', oh! I spelled that wrong.
Thank God for the console.
I'll admit, I can't spell today.
There you go.
Let's try that again.
Open it up, and we'll see,
five hundred and four hundred ninety seven.
All right.
We're grabbing the height and the width of the image, that's perfect.
And as you know this is 500 because we said it as 500, remember if we go to 'Face Recognition' and we have
with 500 pixels.
And the reason that I don't just do 500 here is that, well, if somebody comes along maybe a designer and
decides to change this, it won't get affected in here.
It's always going to be calculating it.
All right.
So we have the width and the height.
We also have the bounding box of 'Clarifai' which is right here.
So we're going to receive this, and we need to do a bit of calculation.
And I'm not going to lie.
This took me about an hour and I decided not to include it in this video because it was just me banging
my brains with a little bit of math.
So I am cutting to me trying to figure out the math to the solution. But I gonna explain exactly how
it works.
We want to return here, an object, and this object is going to be what's going to fill up the 'box' state.
Now this object will first need to figure out the first dot, the second dot, the third dot and the
fourth dot, around the face and then we're going to just wrap it in a border. Again if we go back to the
'Clarifai' API and inspect,
we see that the 'bounding box' is doing something similar.
And we're going to use some of these styles.
We'll say that the left, leftcolumn(lectCol) is going to be the 'clarifaiFace' that we receive right up here.
And as you remember this has a 'left column' property and now the left column(left_col) is the percentage of the
width.
So if we multiply by the 'width' that we have here,
well, which is 500, we're going to get the width of the actual displayed image and where the left
column should be.
So 'width' and the next line is going to be the 'topRow'
and top row is going to have the 'top_row' column or sorry the 'top_row' not the column and it's going to
have the 'height'. Again the top row will say that the height of it is the percentage of our image height.
Now the 'rightColumn' will be the subtraction of the 'width' from the 'clarifaiFace' dot 'right_column' dot 'width'
or multiplied by the 'width'.
So again because the right column is on this side, we want to get the number which is the total percentage
minus the width starting from the left hand side.
And finally with the 'bottomRow', will do something similar but instead with height and it'll be the
'clarifaiFace' 'bottom_row'
multiplied by 'height'.
All right.
Huh! that was a lot of math, that's a, that's as crazy as we're going to get with our math. And I'll show you
how I came up with these numbers, a little bit later on once we have the bounding boxes and I can show
you how you can play around in the console.
So now that we have these, we want to fill up the 'box' state with these values.
So maybe, I can create a, another function, another method on this class that says 'displayFacebox'. Not
the greatest name for a function but it's the one that I just came up with.
And this is going to receive this return value.
And it's going to say 'this' dot 'setState' to equal 'box', 'box'. And just a reminder with the ES6, you
can actually just do this.
But just because this is clear for now we'll keep it this way.
All right.
So we have a few errors here, so let's check them out.
Oh! and it's, because I did semicolons here,
sure some of you were watching this video and saying, why is he putting semicolon in there?
I don't like putting on the last one. Much better.
All right.
So the 'state' is now set with the 'box'.
Hmmm! but to see it so far everything is working.
Going back to our 'onButtonSubmit', so when we click the button, we want to calculate face location with
the response, this 'calculateFaceLocation' is going to run this function, it's going to return an object
and this object is needed by the 'displayFaceBox' method.
So it sounds like, whatever this returns, is this 'box' parameter.
So to make things clean, we can just printed right here saying 'this display FaceBox' and wrap this function
around.
So this 'calculateFaceLocation' takes a response,
returns this object and that returned object is now going into 'displayFacebox'.
Again the beauty of javascript is that, we can use functions like these and it reads nicely too.
Once we get the response, we calculate the inner function which is 'calculateFaceLocation' and then
'displayFacebox'.
God! that's a terrible name,
but like I said for now it's fine.
All right.
There was a lot.
Let's test out our code to make sure that we're on the right track.
I'm going to 'console log' here the 'box' object.
And if we go back to the face again, pasted in here
click and open up the console.
We have an object.
And all right, we have some numbers.
That's good.
Let's try and add some CSS so that these numbers appear on the face. It should correspond with the width.
But looking at these numbers, it doesn't look too bad. If you remember the image width is 500 pixels and if
the left column is at 131 pixels which is probably around here right.
And it doesn't look too bad.
Again let's have some CSS to verify that. We're calculating the right things.
Now, what we want to do is we definitely want to pass this 'box state' into our 'faceRecognition' component.
So we can just do 'box', is going to have 'this' dot 'state' dot 'box'.
And now within 'FaceRecognition',
we now have a new prop, that we can use and this prop can be used by creating a new 'div'.
And this 'div' is actually going to be completely empty, because we're not displaying anything other than
a border around this 'div'.
So what I'm going to do actually is, I'm going to copy what 'clarifai' has, by calling a 'bounding box' class
and we're going to create a new CSS file
'FaceRecognition.css' and just so don't forget, I'm going to import it right now.
And this class which has the 'bounding box',
will have
well, let's look at what they have over here.
I'm just going to copy this actually,
paste it in here.
And there's a few things that as you see because they're cross-talk they're not actually being
used.
So we can, we can remove those.
You also see that, they have the browser prefixes hear for now, we're not too worried about using other
browsers, so we can delete it. If you want to keep those on there that's fine too.
Now that we have that, let's see what happens.
I've added the 'bounding box' around the 'div'
and now if you look at 'div' class you also see that they've added a style, which is what we might want
to do, with our 'top', 'right', 'bottom', 'left' box properties that we just calculatd.
So let's do that.
I'm going to say 'style' equals, again,
we're going to return an object and this will have 'top' as 'box'
and I don't remember the name, so let's just come back here.
Left top row, left column, top row,
right column, bottom row.
Let's see, if I can remember that. Left column,
Right
is going to be 'box' dot
OK.
I definitely can't remember that. 'topRow' and then it's going to be 'bottom', which will have right column
and left which will have the bottom row.
When I get to add the 'box' because they're properties of the box object.
And you already saw that I made the mistake, over here you see that, left I have the bottom.
So let's fix that.
We want to have the top row to be the 'topRow' here.
We want the right column to be 'right column(rightCol)'.
And the bottom row to be the 'bottomRow'
and then finally left column(leftcol). Let's save and see what happens.
Want to go back to our face.
Let's see if I still have, no I don't have the link.
Let me copy and paste it.
Look at that.
It's working.
Let's just inspect this,
and first of all how cool is that, that we got this to work.
And it's wrapped around the face perfectly.
But I want to show you how I figured this up.
Now, when I first did this and again I'm sparing you for an hour of me trying to figure out the math
behind this, I initially had something where the box was a little bit off and all I did was going up
and down this way, and figuring out where the box should be.
And based on that looking at the math and saying, hmm! OK,
so that makes sense that the left column should be the left column percentage times of width, because now
it's calculating the total width times the percentage of where the left column should be.
And then let's say for example bottom row,
well I know that the bottom row should start at the bottom. So the percentage that I get of the bottom
row, I multiply by the height and then I have to subtract it from the total height because the calculation
starts from the top not from the bottom.
But you can play around with this and as you can see by pressing up and down arrows you can adjust these.
I'm really excited that we got that working in the first scope.
That looks amazing just to test this,
let's throw another picture to see if this box moves around with the picture.
Let's do something like Brad Pitt.
And I said Brad Pitt,
is there a Brat Pitt?
All right.
So a picture of him where he's not looking, he's not dead center.
How about this picture.
This looks like a good one.
Let's select it,
go to our app, enter it, detect.
Well, how cool is that?
It's detecting the face.
All right.
Just because a lot of fun, I'm going to pick one more. Let's do a hiker.
Maybe I'll do a climber, and let's see if they can detect this face.
That's going to be pretty tough.
That's a pretty big picture. Let's see.
Look of that.
It detects a face even though the face is very tiny,
it detects it.
This is very very cool.
All right. I want to emphasize here, that I made it look very very easy.
And one of the myths that I really want to bust here is that, as a developer or just because some experience
doesn't mean I can do this quickly. I've definitely practiced this and made sure that I don't waste your time
while you watch this video. And that I keep the errors and if I do anything by mistake I do it intentionally
to show you a point.
But it, like I said, it took me a while to figure out how this box works, mainly because I had to do a bit
of math and look at how 'clarifai' does it.
But the power comes with simply the response that we get from it. By using this response,
I can now add anything here.
Maybe I wanted to replace this with an emoji like a smiley face emoji.
That's easy to do.
It's simply adding a CSS style.
And just to make sure that this is responsive.
Yeah.
Everything works beautifully.
You now have an app that you can show off to your friends. But there's a few other things I want to fix
before we get on to working on the back-end.
And that is, I want to make sure that the 'sign out' feature works and we have a 'sign in' form.
So in the next video we're going to do just that.
I'll see you in that one.