Facial Recognition Inside AVCaptureSession, is it possible?

[CAM]

Is it possible to use CIDetector within an AVCaptureSession or is it just not fast enough. I have tried Setting the accuracy to CIDetector to low and i call it in this function

-(void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection {

I of course change the CMSampleBufferRef into a CIImage and rotate it to make it so it is the right orientation. Once i do that i run the the CIDetector code.

  NSArray *features = [detector featuresInImage:imag.CIImage options:nil];
    for (CIFaceFeature *f in features) {
        CGFloat faceWidth = f.bounds.size.width;
        
      
        
        if(f.hasLeftEyePosition){
            
            NSLog(@"left eye position x = %f , y = %f", f.leftEyePosition.x, f.leftEyePosition.y);
             
      
        }
        
        if(f.hasRightEyePosition){
            
            NSLog(@"right eye position x = %f , y = %f", f.rightEyePosition.x, f.rightEyePosition.y);
        
        }
        if(f.hasMouthPosition){
            
            NSLog(@"mouth position x = %f , y = %f", f.mouthPosition.x, f.mouthPosition.y);
         
           
        }
    }

My problem is, when I hold it up to my face, the Logs only seem to get called maybe once every minute, sometimes not at all. The picture is definitely being created properly because i got it to save them to my photo album and i checked every time.
Thanks in Advance

[Duncan C]

Originally Posted by CAM

Is it possible to use CIDetector within an AVCaptureSession or is it just not fast enough. I have tried Setting the accuracy to CIDetector to low and i call it in this function

-(void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection {

I of course change the CMSampleBufferRef into a CIImage and rotate it to make it so it is the right orientation. Once i do that i run the the CIDetector code.

  NSArray *features = [detector featuresInImage:imag.CIImage options:nil];
    for (CIFaceFeature *f in features) {
        CGFloat faceWidth = f.bounds.size.width;
        
      
        
        if(f.hasLeftEyePosition){
            
            NSLog(@"left eye position x = %f , y = %f", f.leftEyePosition.x, f.leftEyePosition.y);
             
      
        }
        
        if(f.hasRightEyePosition){
            
            NSLog(@"right eye position x = %f , y = %f", f.rightEyePosition.x, f.rightEyePosition.y);
        
        }
        if(f.hasMouthPosition){
            
            NSLog(@"mouth position x = %f , y = %f", f.mouthPosition.x, f.mouthPosition.y);
         
           
        }
    }

My problem is, when I hold it up to my face, the Logs only seem to get called maybe once every minute, sometimes not at all. The picture is definitely being created properly because i got it to save them to my photo album and i checked every time.
Thanks in Advance

I'm wondering about that as well. We have a prototype we're working with right now that uses a CIDetector set to high quality for still images, and it takes around 5 seconds to recognize a face, or sometimes less. It doesn't make sense that low accuracy would be slower.

My plan was to capture an image from a video capture session and pass that image to a CIDetector working in the background. Once I got a response from the detector, I would use those results on the current image stream. Hopefully the CIDetector would not lag too far behind, so it's results are still relevant.

If you look at Photo Booth in Mac OS 10.7, that's almost certainly what they are doing. The new morphs that apply to faces lag for a few seconds if the faces in the image move around.

[CAM]

Originally Posted by Duncan C

I'm wondering about that as well. We have a prototype we're working with right now that uses a CIDetector set to high quality for still images, and it takes around 5 seconds to recognize a face, or sometimes less. It doesn't make sense that low accuracy would be slower.

My plan was to capture an image from a video capture session and pass that image to a CIDetector working in the background.

I am doing the same thing, and i feel like it is too slow. I have tried something like this to see what it can do.

-(void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection {
    i++;
    if(i==100){
        i=0;
 

   UIImage *imag = [self imageFromSampleBuffers:sampleBuffer ];

        imag = [UIImage imageWithCGImage:imag.CGImage scale:imag.scale orientation:UIImageOrientationRight];
    [self performSelectorInBackground:@selector(markFacesWithImage:) withObject:imag];
}
}

I did this to see if the CIDetector was just running slow and that didn't seem to work either. I am thinking OpenCV might be the best to use, but I don't believe it can detect the eyes or mouth.