Welcome back. We have our beautiful web site set up.
It's time to create some functionality, where we enter a url, press 'Detect' and an image appears with
a face detection box,
if a face exists in the picture. So how are we going to do that?
Well, the first thing we want to do, we want to make sure that there is some functionality here where
we can detect, what the user enters. And it's something that we've done before with react, when we had
our robo friends app.
We had our input that filtered through the robots.
So again, using the similar technique, using the 'ImageLinkForm', we can create this functionality.
So the first thing you might realize is that, yeah! we definitely need 'state', we need to create a 'state'
so that, our app knows, what the value is that the user enters, remembers it, and updates at any time it
gets changed.
So as you remember, in order to do that, we will define a 'constructor'
and within this 'constructor', we have to call 'super' to be able to use this.
And here, we can do 'this'
'state' equals and we'll call it,
'input', that's what the user will input.
And this input will have a 'onChange' or 'onInputChange', that we will use with arrow functions,
so that we don't get that error that we saw previously.
And as you know anytime there's some sort of an event listener on a web page, we receive an event and
we want to do something here.
So we'll do a 'console dot log', 'event' to make sure that we can detect it and on 'inputChange' in the
'imageLinkForm' we can pass it as a prop.
And we'll get the error 'onInputChange' is not defined.
Remember that because it's part of this class to access it, you need to save 'this dot onInputChange'
because 'onInputChange' is a property of the 'App'.
Let's save that, right now,
well it's not doing anything because we need to trigger this.
We go back to 'imageLinkForm' that's the css 'imageLinkForm.js'.
We can destructor the 'onInputChange' that we're getting.
So instead of doing 'props' dot 'onInputChange' down here, we just destructur it from the 'props' and we
can say on 'input' here 'onChange',
so 'onChange' is technically a react synthetic event that mimics what the html does with whenever input
changes.
And here the 'onChange', well what's it going to do, if we just do 'onInputChange' here,
let's see what happens?
I'm going to open up the console, click or press something, oop! go to the input and press something.
All right.
And looks like I'm getting something, a proxy event.
All right.
What if I go back into my
app dot js? and if you remember the way we get our value from the input is dot 'target' dot 'value'.
If I save that, and go back, there you go, we get our input.
Nice and easy, just like we did before with our initial react app.
Right, so we're getting the 'onInputChange' and we also want that when we click on 'Detect' that while we
detect an image, how are we going to do that?
For now, just because I like keeping things simple, building things one step at a time we can just 'console
log' and we'll say 'onSubmit' and it won't have any parameters for now.
We'll just say 'console dot log', click, and now 'onSubmit', we can pass to our 'imageLinkForm' again.
We will title it 'onButtonSubmit' here, just to make it clear.
And again, we'll pass it with 'this' dot 'onButtonSubmit'.
We save that, I'm going to put this on a new line just again so you can see,
now the 'imageLinkForm' will receive 'onButton
Submit' and we can say, here that, other than the 'className', they'll be a 'onClick' event that listens for the 'on
ButtonSubmit'.
Let's save, go back to our example,
click, look at that, that's working.
All right.
So there's a bit of functionality but the hard part is just getting started.
How do we display an image down here and have it detect the face? Well,
luckily for us, we can use an API, and this API I think is one of the coolest out there.
It's called 'clarifai' and as you can see from their tagline 'understand your image and video data'. You can
use this tool actually for free, up to a certain point.
If you look at the pricing over here for developers you have a free 5000 operations per month.
So you can submit 5000 pictures and it'll guess it for you.
Once your app becomes popular then you can start paying for it.
But we're going to be using a free API. And I want you to sign up for this, if you want to use it because
the API key that I'll use in this example, I'll delete it afterwards so that,
well, so that other people don't ah! don't use it
and then I reach my limit and I won't be able to use it anymore.
But it's very simple to sign up.
You just log in.
And once you create your account here, you'll be taken onto your profile.
And they might already have my first application for you.
If there's nothing here you just click on 'CREATE NEW APPLICATION' and here,
and your application, you can title it whatever you want but you don't need to touch anything.
If you go to API Keys, this key is your way of accessing the 'clarifai' API.
So if here I enter a url, I press detect, I can send that url with my API key that they provide for you
to their service and using the API key they'll say, oh! that's Andre, he's under the free plan.
He hasn't hit his 5000 limit yet for the month.
Yes.
Let's respond.
And in case you're wondering what they can actually do, if you go to their models you see over here that
they use machine learning to train something called 'Models'.
That is they train a computer by giving it many many pictures, tons of pictures, to memorize celebrity
faces, detect 'Apparel' like here, and it knows that these are sunglasses with 100 percent probability because
it's been trained probably millions of pictures of sunglasses to know what sunglasses are like. So you
can play around with a lot of the stuff that they have here.
It's really really nice what they have.
We're going to be using their face detection API. And their face detection API,
we simply,
if we go to the js(javascript), we simply run this function, to the 'clarifai' servers using our own API and it gives
us a response or an error.
Now this is always a lot easier, to demonstrate than to actually talk about, so let's implement it.
I'm using their face recognition.
So I'm just going to copy and paste this. Go back to my app and in app dot
js(app.js), I'll say that 'onButtonSubmit',
I want to run, the function that 'clarifai' just told me to use. But as you can see here, it has 'app
dot models dot predict'.
Well, how is my program going to know what 'app dot models dot predict' is?
Well, if you go to their API guide, they will show you exactly how to get started and how to use their API.
But you can see over here and they're 'Getting started', 'API clients' if we go to javascript, they'll say
install the client from npm.
So let's just follow along and install 'clarifai' and this is a package that they've built so they can
work with javascript, while that's loading, we can see that, we can import 'clarify'.
We can see here that they're using the common JS, 'require' way of importing,
but since we're using 'create react app', we can use 'import' here, and then we simply initialize your API
key with this. So that now we'll be able to have the app object. As you can see here we're using 'app
.models'.
But if I run this, I'll have no idea what 'app' is. Just to demonstrate for you, if I go back to the console
here,
'run npm start', I get the 'app is not defined', all right.
So again copy, what we see here will copy this, put it up in our configuration at the top of the file.
It asks for our API key.
In my case I'll use the one that they've given me.
Let me open that link in a new file.
I go to 'my first application', 'API keys'.
I'll copy and paste my API key. Paste it in here.
Like we talked about in the API section most useful APIs require a set up like this, where you have
to add an API key. So although these documents are specifically for 'clarifai' most APIs that you use are
going to be very very similar to this.
All right, so we have that, we also need to import,
if you remember, so going back here, I can just
import 'clarifai' and because, we can use the new javascript way of importing.
We'll just do that because it looks so much nicer.
All right let's save and see what happens.
All right.
It looks like it's working.
We see that we're just getting a, this is just a sample image that we're getting.
So we're using our API key and then submitting a url image and then having this dot then occur let's
console dot log the response, and see what we get.
I'm going to click submit(Detect), and I get status,
all right.
It looks like it's responding with something, that's good.
Very good.
All right, so for now, it looks like this is working.
We have an image, that we're receiving, when I put this on a new line so you can see.
Right.
So for now, let's have it, so we can display whatever image they just gave us in our example. Let's just copy
and see what this image is of.
So we want this to display at the bottom.
So how can we do that?
Well, we can start creating our last 'FaceRecognition' component. So we can move the brackets or the comments.
And now here, we'll import this new component, that we don't have yet and this would be called The 'Face
Recognition' component.
And again we'll create a new folder, that will say 'FaceRecognition'.
And within this new folder, (there)will be a new file 'FaceRecognition dot
js', using our very good tool,
copy and paste, going to copy and paste the 'Navigation' component
and now change the name of this to 'FaceRecognition'
component and what should this display?
Well for now we just want to display a static image to make sure that it works.
So I'm going to create
a 'div' with the 'className' of well, 'center' that we've already defined before,
if you remember in our app.css. And will have an image, with a source(src) of,
close the tag and we could just save that.
Let's see if that works.
All right.
That looks pretty good actually.
We got our image right at the bottom.
I get an error saying that we need an 'alt' tag.
So we'll just say 'alt'
and we'll just leave it blank for now.
Just so we have something.
All right.
So we have the basic look of the image and that's working but that's not really connected to the API.
When I click detect, well it's just hard coded.
So first let's look at the API and see how we can actually submit, image urls, if you go back to their
documents.
Again they're very very good with this.
If you scroll down,
under the 'Predict', you see that they have images and videos.
So we can use the 'Images', 'via URL' and we see that we have the javascript and it looks like the 'Predict'
function takes the 'clarifai' a model that they have.
So in this case they're sending the general model and the url. Right so it's very similar to what
we saw before.
So by the looks of it because we've already defined our API key we don't need this anymore.
All we need is to tell it, what model do we want to use.
But if you remember they have a ton of models. And actually want to show you a smart way of looking at
this.
They have this information in the docs but another good way of doing it is if you remember there was
a 'clarifai' npm package and this npm package has a github link and all npm packages usually do. In here,
if you go to visit their codebase, if you go into source, as you might remember, and index.js, this
is what they're giving us access to, and you see over here they have 'exports global dot clarifai' and here
all their models that we can use. So they're giving us the 'GENERAL_MODEL'.
But you can see, if we can get the 'FOOD_MODEL', the 'COLOR_MODEL'.
So let's use the 'COLOR_MODEL' for now, and in here, we will say
'COLOR_MODEL'.
And in order to have access to that, we have to do 'clarifai' dot 'COLOR_MODEL'.
So let's save that, and we (are) going to test this by opening up the console clicking detect,
I get a response and within their response, if I go to 'outputs', 'zero(0)', 'data', I get 'colors' and these
are all the hex colors predictions and you see the one with the highest probability, let's see what
color that is.
It's were pretty much black and if you look at this image yeah it's pretty much black.
All right so that is working.
And the reason I was able to find this is again in their API documentation it tells you if it's an it's
an outputs and an output returns an array with data inside it.
And because we selected the colors model we get colors. All right, so for now using the colors model,
let's build it up.
We want to have an 'imageUrl' state and it will be empty for now and this 'imageUrl' should
get displayed when we click on submit, so we can say that 'onButtonSubmit', we wanna 'this' dot 'setState' to
equal, to have image url updated with whatever the 'input' is.
So this way, we can pass the 'imageUrl' down to the 'FaceRecognition'.
So if I do 'imageUrl' equal to 'this' dot 'state' dot 'imageUrl', I can now use 'imageUrl' in
my 'FaceRecognition' component as the source(src).
Now we obviously need to receive this, so I'm gonna destructure it and use it as parameters.
And I get 'input is not defined',
That's because in here, I used input and well, it needs to be 'this' dot 'state' dot 'input'. Go back let's ah!, let's
find a new face actually. Let's do,
let's do a color.
Let's do a landscape, let's do one with a defining color.
That's a good one, let's do view image, copy this, if we do this and we don't get anything because well, if
you remember 'onInputChange', we've never actually updated the 'input' component.
We just console logged everything.
So now that we know that we can just say, 'this' dot 'set
State'
'input'
is the 'event' 'target' dot 'value'.
Let's see if that works.
And I go back, try it out and,
Oh boy!, that's a big picture we'll need to fix that,
but we got our image displaying. Now using this information,
it means that, we can give our url as the input over here.
We can just put 'this' dot 'state' dot 'input'
and open up the console.
All right,
looks like we're getting a response.
We're getting 'data' 'colors'.
We have this with 40 percent chance of this being the color.
Let's see what color that is.
All right that's green.
That's pretty cool.
So it's clearly working.
We're getting, I'm gonna close some of these.
All right, things are clearly working now, which is great.
Now the one trick I want to show you, because of the way react works,
if I would have done here, 'imageUrl', I would actually get an error,
So if I copy and paste, let's do an image 'Apple'
I get a 'Bad request' and that is because of the way 'set state' works.
All right.
So, while we actually want to use is the 'FACE_DETECT_MODEL' so using what we have now,
Let's see if we can get a response for, and if we go to our API we can look at the models,
and actually if we go to the top, you go to 'Models', and we go to 'Face Detection',
we see that we'll have a response that looks something like this.
What we're interested in is from the outputs, to get data and then regions that give us the regions or
the bounding box of the face. We'll see if we can get that working.
I've put the 'FACE_DETECTION_MODEL' and we don't know what the response is going to be yet so I'm just
going to leave it at that.
I'm going to find a face image,
let's do this picture,
copy and paste, detect,
oh boy! we need to fix that face.
But, we see that we're gtting output.
In the first array, we get 'data', 'regions'
and we have a 'bounding_box'.
So it's a little bit nested here.
So using this information, let's clean it up a little bit and then actually get what we need from here.
In our case, it looks like we'll need to get the 'response'
dot 'outputs'.
And then the 'zero(0)' the first array, then it was 'data', 'regions', 'zero(0)', 'region info', then 'bounding box'.
All right, let's do that.
So 'data regions' access the first array,
'region_info' and I think the last one was 'bounding box'.
Let's try that.
All right.
Let's give that a go.
And see if that works.
Detect
Look at that. We have our 'bottom row', 'left column', 'right column', 'top row'.
We still don't really know what these numbers mean.
We'll have to actually figure it out because as you can see it doesn't add anything to the face.
That's something that we'll have to do.
But it's working and we got our API connection.
We're actually doing something we're displaying the image.
So before we get into the next video let's fix this image issue to make sure that everything looks, well,
looks better than this.
So if we go to the 'FaceRecognition' here, there's a couple of things that we will need to fix.
I'm going to remove this 'console'.
We will do 'ma', which is a 'tachion' class and then we'll also wrap this in a 'div' and there's many ways
to do this.
Like I said before with styling, I'll show you the main way of doing it.
I going say 'absolute margin top two' just so there's a bit of a space in between the input and the image.
And now within the image, we want to make sure that the 'width' is always the same no matter the size of
the picture.
We'll do '500 pixels' but we also don't want it to get squished or have weird proportions.
So the way to fix that is to do 'auto'. So hight is automatically going to be adjusted based on the width.
So let's save and see what happens
now.
All right.
That's a, that's a lot better.
So that was pretty quick.
This was the main thing that really fixed it.
All right.
Awesome.
So we got ourselves a working app that returns real results.
And now we can work on having a box displayed around the face, if something gets detected. Just out of
curiosity.
If I do landscape again.
All right.
Everything's working fine.
Huh! that was a long one but things are starting to work.
I'll see you in the next video.
Buh-bye