Sobel in C ; Sobel operator on YUV video in C
Implementation Sobel operator in C on YUV video File
Today, we discuss Sobel operator and how to apply on YUV
video file with step by step discussion. If you are not familiar with
the sobel operator or don’t know in detail, don’t worry, we first discuss what
is sobel operator followed by its C code.
Lets start our discussion.
First question is what is Sobel filter/operator?
The algorithm we will look at in this tutorial is an edge
detection algorithm, specifically an edge detection algorithm based on the
Sobel operator. This algorithm works by calculating the gradient of the
intensity of the frame at each point, finding the direction of the change from
light to dark and the magnitude of the change. This magnitude corresponds to
how sharp the edge is.
If you are not aware how to apply sobel operator on image,
please refer this link for that.
The Sobel Operator
To calculate the gradient of each point in the frame, the frame
is convolved with the Sobel Kernel. Convolution is done by moving the kernel
across the frame, one pixel at a time. At each pixel, the pixel and its
neighbours are weighted by the corresponding value in the kernel, and summed to
produce a new value. This operation is shown in the following diagram.
~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| k00 | k10 | k20
| | i00 | i10
| i20 | i30 | ..
|~~~~~+~~~~~+~~~~~+
|~~~~~+~~~~~+~~~~~+~~~~~+~~~
| k01 | k11 | k21
| | i01 | i11
| i21 | i31 | ..
|~~~~~+~~~~~+~~~~~+
|~~~~~+~~~~~+~~~~~+~~~~~+~~~
| k02 | k12 | k22
| | i02 | i12
| i22 | i32 | ..
|~~~~~+~~~~~+~~~~~+
|~~~~~+~~~~~+~~~~~+~~~~~+~~~
| i03 | i13 | i23 | i33 | ..
Applying the kernel to
the |~~~~~+~~~~~+~~~~~+~~~~~+~~~
to the first pixel gives
us | .. | .. | .. | .. | ..
an output value for that
pixel
|
o00
= k11 * i00 + k12 * i01 + k21 * i10 + k22 * i11
|
we treat it as though the kernel has been overlaid onto the
frame, with the center pixel of the kernel aligning we the first pixel in the
frame. Then we multiple each entry in the kernel by the value beneath it, and
sum them to produce a single single output value from that pixel.
For the pixels on the boundary, we just ignore any entries in
the kernel that fall outside.
Edge detection using the Sobel Operator applies two separate
kernels to calculate the x and y gradients in the frame. The length of this
gradient is then calculated and normalized to produce a single intensity
approximately equal to the sharpness of the edge at that position.
The kernels used for Sobel Edge Detection are shown below.
~~~~~~~~~~~~~
|-1 | 0 | 1 |
|~~~+~~~+~~~|
|-2 | 0 | 2 |
|~~~+~~~+~~~|
|-1 | 0 | 1 |
~~~~~~~~~~~~~
X gradient kernel
~~~~~~~~~~~~~
|-1 |-2 |-1 |
|~~~+~~~+~~~|
| 0 | 0 | 0 |
|~~~+~~~+~~~|
| 1 | 2 | 1 |
~~~~~~~~~~~~~
Y gradient kernel
The Algorithm
Now the algorithm can be broken down into its constituent steps
1. 1. Load
YUV File.
2. Load Frame of YUV video file.
3. Load Y , U and V Plane
separately
4. Iterate over every pixel in the Plane
for each Plane for each Frame
5. Apply the x gradient kernel
6. Apply the y gradient kernel
7. Find the length of the gradient
using pythagoras' theorem
8. Normalize the gradient length
to the range 0-255
9. Set the pixels to the new
values
10. Combined all three Plane and make a frame
11. Write this frame to video file yuv format
12. Save the YUV file.
This illustrates the main steps, though we miss out some
specifics such as what we do when we meet the boundary of the frame.
Formula for calculating gradient in X and Y direction ;
Let say (i,j) is the
middle pixel of 3*3 grid then we have the following formula
|
Gx =
- ( - Z1 + Z3 - 2*Z4 + 2*Z6 – Z7 + Z9
) / 8
&
Gy =
-(-Z1 – 2*Z2 – Z3 + Z7 + 2*Z8 + Z9 ) /8
|
Now time to move ahead, now we are looking for C code, right?
Here is the complete Code;
Code have Two procedure.
Save below procedure in new M file with name
Sobelr.h
|
/****************************
Sobel.h********************/
/*
Title : Implementation of
Soble Operator on Video in C
Object: Video file; YUV 4:2:0
Auth: Nitin Gupta
Date: 18/01/13 10:28
*/
#ifndef SOBEL_H
#define SOBEL_H
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include <time.h>
//define image height and width with number of frames
#define HEIGHT 480
#define WIDTH 640
#define NUM_FRAME 20
#define THRESH_HOLD_Y
20
#define THRESH_HOLD_U
1
#define THRESH_HOLD_V
1
//Input and output file name
char *Input_Filename = "front_camera_640x480_600_frames.yuv";
char *Output_Filename = "Output_front_C.yuv";
//Prototype of functions
//allocation of memory
void Sobel_Init ( unsigned char
**Input_Image_ptr, unsigned char
**Output_Image_ptr ) ;
//Intialization of memory
void Sobel_Assign( unsigned char *Input_Image_row_ptr[HEIGHT], unsigned char
*Output_Image_row_ptr[HEIGHT], unsigned char *Input_Image_ptr, unsigned char *Output_Image_ptr );
// For Loading Video File
void Load_Frame ( unsigned char
*Input_Image_ptr, FILE *File_reader, const int Frame_num ) ;
//Saving each frame to output file
void Save_Frame_In_Video ( unsigned
char *Output_Image_ptr , FILE * File_writer )
;
//Sobel operation; Applying Sobel operator
void Sobel_Operation ( unsigned
char *Input, unsigned
char *Output , const
int Width , const
int Height, const
size_t Thresh ) ;
//Sobel thrash-hold
char Sobel_Thrashhold ( char
Pixel_Value , const size_t Thresh ) ;
#endif
|
And save this file as Sobel.c
|
/*
Title : Implementation of
Soble Operator on Video in C
Object: Video file; YUV 4:2:0
Auth: Nitin Gupta
Date: 18/01/13 10:28
*/
#include "Sobel.h"
/********************* MAIN FUNCTION**********************************/
int main(int argc, char * argv[])
{
time_t start_time , end_time ;
/***********************************
Variable Declaration ******************************/
int Frame_num
= 0 ; //
loop variable
//File pointer
for reading and writting
FILE *File_reader, *File_writer;
//Image FRAMES
pointer's
unsigned
char *Input_Image_ptr;
unsigned char *Output_Image_ptr;
/*******************************************************************************************/
/***********************************
Opena and close file operation
******************************/
//Read Input
imgae file in binary mode
if ( ! (
File_reader = fopen(Input_Filename,"rb")
) )
{
printf ( "\nError
in opening input file" );
system("pause");
exit( 0 );
}
//Open Output
imgae file in binary mode
if ( ! (
File_writer = fopen(Output_Filename,"wb")
) )
{
printf ( "\nError
in opening output file" );
system("pause");
exit( 0 );
}
/*******************************************************************************************/
/***********************************
Assign memory to pointers operations
******************************/
Sobel_Init ( &Input_Image_ptr, &Output_Image_ptr ) ;
//start timer
time(&start_time) ;
/*******************************************************************************************/
/***********************************
ITERATING OVER EACH FRAM ONE BY ONE
******************************/
for
(Frame_num = 0 ; Frame_num < NUM_FRAME; Frame_num++ )
{
//Load
Frame
Load_Frame ( Input_Image_ptr, File_reader , Frame_num
) ;
// memcpy ( Output_Image_ptr , Input_Image_ptr
,(size_t)(1.5f * WIDTH * HEIGHT) * sizeof ( unsigned char ) ) ;
//Apply
Sobel filter
//Apply on
Y plane
Sobel_Operation ( Input_Image_ptr,
Output_Image_ptr , WIDTH , HEIGHT , THRESH_HOLD_Y ) ;
//Apply for
U plane
Sobel_Operation ( Input_Image_ptr
+ HEIGHT*WIDTH , Output_Image_ptr
+ HEIGHT*WIDTH , WIDTH/2 , HEIGHT/2,
THRESH_HOLD_U ) ;
//Apply for
V plane
Sobel_Operation ( Input_Image_ptr
+ (size_t)(1.25*HEIGHT*WIDTH) ,
Output_Image_ptr + (size_t)(1.25*HEIGHT*WIDTH) , WIDTH/2 , HEIGHT/2 ,THRESH_HOLD_V ) ;
//Save
frame in video
Save_Frame_In_Video (
Output_Image_ptr , File_writer ) ;
}
//stop timer
time(&end_time) ;
fclose (File_reader) ;
fclose (File_writer) ;
printf ("The
total duration of program excution is %.3f seconds ", difftime
(end_time,start_time) ) ;
system(Output_Filename)
;
system("pause");
return
0;
}
//allocation of memory
void Sobel_Init ( unsigned char
**Input_Image_ptr, unsigned char
**Output_Image_ptr )
{
//Assign memory
to image pointers for storing FRAMES in terms of bytes
//For input
image; SINCE EACH FRAM IS OF SIZE 1.5*WIDTH * HEIGHT
if ( !(
*Input_Image_ptr = (unsigned char *) malloc ( (size_t)(1.5f * WIDTH * HEIGHT) * sizeof ( unsigned char ) ) ) )
{
printf ("\nMemory
can not be allocated; :(" );
system("pause");
exit( 0 );
}
//for output
image
if ( !(
*Output_Image_ptr = (unsigned char *) malloc ( (size_t)(1.5f * WIDTH * HEIGHT)* sizeof ( unsigned char ) ) ) )
{
printf ("\nMemory
can not be allocated; :(" );
system("pause");
exit( 0 );
}
memset (*Output_Image_ptr , 0 ,
(size_t)(1.5f * WIDTH * HEIGHT)* sizeof ( unsigned char ) );
}
void Load_Frame ( unsigned char *Input_Image_ptr, FILE *File_reader , const int Frame_num
)
{
//size of file
in term of bytes
size_t read_size;
//read the
image and store bytes in Input_Imgae_ptr along with get its size in terms of
bytes in read_size
if (
( ( read_size = fread ( Input_Image_ptr, sizeof(unsigned char),
(size_t)(1.5f * WIDTH * HEIGHT), File_reader ) ) != ( 1.5*HEIGHT * WIDTH ) )
)
{
printf ("\nRead
operation can not be done; Error occured\n" );
printf("reached
end of video file at frame number %d\n", Frame_num);
printf("read size is %d\n",
read_size);
system("pause");
exit( 0 );
}
}
//Sobel operation; Applying Sobel operator
void Sobel_Operation ( unsigned
char *Input, unsigned
char *Output , const
int Width , const
int Height , const
size_t Thresh )
{
//row_num for
Number of Row and col_num for Number of Colomn
int
row_num, col_num;
//gradient of
image in horizontal and vertical direction and final gradient
float
Gradient_h, Gradient_v ;
char
Pixel_Value = 0 ;
//Applying
sobel operator; for each frame; only Y Plane
for
(row_num = 0; row_num < Height; row_num++)
{
for
(col_num = 0; col_num < Width; col_num++)
{
if ((row_num != 0) && (col_num != 0)
&& (row_num != HEIGHT - 1) && (col_num != WIDTH -1 ))
{
Gradient_v = -(-Input [
(row_num-1)*Width + (col_num-1 )] + Input [(row_num-1)*Width + (col_num+1) ]
- 2*Input [row_num*Width + (col_num-1)] + 2*Input [ row_num*Width +
(col_num+1) ] - Input[ (row_num+1)*Width + (col_num-1)] + Input [
(row_num+1)*Width + (col_num+1)] ) /8 ;
Gradient_h = -(-Input
[(row_num-1)*Width + (col_num-1)] - 2*Input [ (row_num-1)*Width + col_num ] -
Input [(row_num-1)*Width + (col_num+1)] + Input [(row_num+1)*Width +
(col_num-1)] + 2*Input [(row_num+1)*Width + col_num ] + Input [(row_num+1)*Width + (col_num+1)] )
/8 ;
//Assign
to image
Pixel_Value = ( char ) sqrtf ( Gradient_h * Gradient_h + Gradient_v
* Gradient_v ) ;
Output[ ( row_num - 1 )*Width + (col_num- 1 ) ] =
Sobel_Thrashhold ( Pixel_Value,
Thresh );
}
}
}
}
//Sobel thrash-hold
char Sobel_Thrashhold ( char
Pixel_Value ,const size_t Thresh )
{
if ( Pixel_Value >= Thresh )
return
(char )255 ;
else
return
(char )0 ;
}
//Saving each frame to output file
void Save_Frame_In_Video ( unsigned
char *Output_Image_ptr , FILE * File_writer )
{
//Wrtie each
frame to output video
fwrite( Output_Image_ptr, sizeof (unsigned char), (size_t)(1.5*WIDTH * HEIGHT), File_writer);
}
|
Got Questions?
Feel free to ask me any
question because I'd be happy to walk you through step by step!
References and External Links
For Contact us….. Click on Contact us Tab
Posts
-(-Input [ (row_num-1)*Width + (col_num-1 )]
ReplyDeleteWhy do u multiply it by width here?
very nice description. I am wondering how can be apply Hough Transform on top of it to detect straight lines? Can you share your thoughts and if possible any source or blog post for the same ?
ReplyDeleteJust keep writing this kind of posts
ReplyDeleteClan 7 con Hola amigos! 1 ćwiczenia
Very nice article,keep sharing it more.
ReplyDeleteThank you.
ServiceNow Developer Training