Implementation Sobel operator in Matlab on YUV video File

Posted by Unknown at 07:17 | 17 comments

Implementation Sobel operator in Matlab on YUV video File

This post is outdated click here for new one

Today, we discuss Sobel operator and how to apply on YUV video file with step by step discussion. If you are not familiar with the sobel operator or don’t know in detail, don’t worry, we first discuss what is sobel operator followed by its Matlab code.

Lets start our discussion.

First question is what is Sobel filter/operator?

The algorithm we will look at in this tutorial is an edge detection algorithm, specifically an edge detection algorithm based on the Sobel operator. This algorithm works by calculating the gradient of the intensity of the frame at each point, finding the direction of the change from light to dark and the magnitude of the change. This magnitude corresponds to how sharp the edge is.

If you are not aware how to apply sobel operator on image, please refer this link for that.

The Sobel Operator

To calculate the gradient of each point in the frame, the frame is convolved with the Sobel Kernel. Convolution is done by moving the kernel across the frame, one pixel at a time. At each pixel, the pixel and its neighbours are weighted by the corresponding value in the kernel, and summed to produce a new value. This operation is shown in the following diagram.

~~~~~~~~~~~~~~~~~~~            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~

| k00 | k10 | k20 |            | i00 | i10 | i20 | i30 | ..

|~~~~~+~~~~~+~~~~~+            |~~~~~+~~~~~+~~~~~+~~~~~+~~~

| k01 | k11 | k21 |            | i01 | i11 | i21 | i31 | ..

|~~~~~+~~~~~+~~~~~+            |~~~~~+~~~~~+~~~~~+~~~~~+~~~

| k02 | k12 | k22 |            | i02 | i12 | i22 | i32 | ..

|~~~~~+~~~~~+~~~~~+            |~~~~~+~~~~~+~~~~~+~~~~~+~~~

                               | i03 | i13 | i23 | i33 | ..

Applying the kernel to the     |~~~~~+~~~~~+~~~~~+~~~~~+~~~

to the first pixel gives us    |  .. |  .. |  .. |  .. | .. 

an output value for that

pixel

o00 = k11 * i00 + k12 * i01 + k21 * i10 + k22 * i11

we treat it as though the kernel has been overlaid onto the frame, with the center pixel of the kernel aligning we the first pixel in the frame. Then we multiple each entry in the kernel by the value beneath it, and sum them to produce a single single output value from that pixel.

For the pixels on the boundary, we just ignore any entries in the kernel that fall outside.

Edge detection using the Sobel Operator applies two separate kernels to calculate the x and y gradients in the frame. The length of this gradient is then calculated and normalized to produce a single intensity approximately equal to the sharpness of the edge at that position.

The kernels used for Sobel Edge Detection are shown below.

~~~~~~~~~~~~~

|-1 | 0 | 1 |

|~~~+~~~+~~~|

|-2 | 0 | 2 |

|~~~+~~~+~~~|

|-1 | 0 | 1 |

~~~~~~~~~~~~~

X gradient kernel

~~~~~~~~~~~~~

|-1 |-2 |-1 |

|~~~+~~~+~~~|

| 0 | 0 | 0 |

|~~~+~~~+~~~|

| 1 | 2 | 1 |

~~~~~~~~~~~~~

Y gradient kernel

The Algorithm

Now the algorithm can be broken down into its constituent steps

1.      1.  Load YUV File.
2.    Load Frame of YUV video file.
3.    Load Y , U and V Plane separately
4.    Iterate over every pixel in the Plane for each Plane for each Frame
5.    Apply the x gradient kernel
6.    Apply the y gradient kernel
7.    Find the length of the gradient using pythagoras' theorem
8.    Normalize the gradient length to the range 0-255
9.    Set the pixels to the new values
10. Combined all three Plane and make a frame
11. Write this frame to video file yuv format
12. Save the YUV file.

This illustrates the main steps, though we miss out some specifics such as what we do when we meet the boundary of the frame.

Let’s formulate the Formula;

So, we reach the required formula;

Here it is;

Gx = ( z7 + 2z8 + z9) – ( z1 + 2Z2 + z3 ) & Gy = (z3 + 2z6 + z9) – ( z1 + 2Z4 + z7)

Now time to move ahead, now we are looking for Matlab code, right?

Ok let’s start coding in Matlab;

For understand below code, you must be aware to the following function’s.

Imread( ) , rgb2gray( ) , size ( ) , figure , imshow ( ), double    fread () fwrite () fclose() fopen()  .  concatenation of vectors [ ]

Here is the complete  Code;

Code have two procedure.

Save below procedure in new M file with name Sobel_Operator.m

%Procedure; Sobel operator procedure; it takes one frame, apply sobel operator and return it function[ Gray_Edge_Image ] = Sobel_operator( orig_frame, a )             Gray_Edge_Image = zeros (size(orig_frame,1) , size(orig_frame,2)) ;             %make a temporary gray scale image             Temp_Image = double (orig_frame) ;              %Apply sobel operator             %variable for finding maximum value across Matrix             max_value = -1 ;             %Start iterate from first corner to last corner             % size(Temp_Image,1) gives number of rows; we subtract 2 bec. of leaving  boundry             for i = 2 : size(Temp_Image,1) - 1                 for j = 2 : size (Temp_Image, 2) - 1                         %calculating gradient in X direction                  X_Gradient = ( ( Temp_Image(i+1,j-1) + 2*Temp_Image(i+1,j) + Temp_Image (i+1,j+1) ) - ( Temp_Image(i-1,j-1) + 2*Temp_Image(i-1,j) + Temp_Image (i-1,j) ) );                         %calculating gradient in Y direction                  Y_Gradient = ( ( Temp_Image(i-1,j+1) + 2*Temp_Image(i,j+1) + Temp_Image (i+1,j+1) ) - ( Temp_Image(i-1,j-1) + 2*Temp_Image(i,j-1) + Temp_Image (i+1,j-1)));                         %obtaining Gradient for a pixel                         Gray_Edge_Image (i-1 , j-1) = sqrt ( Y_Gradient*Y_Gradient + X_Gradient*X_Gradient );                         if (max_value < Gray_Edge_Image ( i-1, j-1 ) )                              max_value = Gray_Edge_Image ( i-1 ,j-1 )  ;                         end                 end             end %converting whole image value in between 0 to 255; division makes it in range of [0,1] for i = 2 : size(Temp_Image,1)-1     for j = 2 : size (Temp_Image, 2)-1              Gray_Edge_Image ( i-1 ,j-1 ) =   ( ( Gray_Edge_Image ( i-1 ,j-1 )/ max_value ) * 255 );        end    end end

And here the main code; save this code in new M file with name Sobel_operator_on_Video.m

% clear all; %all global variable(s) height = 720;  %height of the video width = 1280;   %Width of the video num_frames =100;   %number of framse in video filename = 'A_280x720 _460_frms.yuv';  %file name of video filename_copy = 'copy_ A_280x720 _460_frms.yuv' ; %the new file %open video file of .yuv format fid_read = fopen (filename, 'rb'); fid_write = fopen (filename_copy,'wb') ; %loop over every frames of video for index= 1 : num_frames     %reading first frame ; deg will be the vector of frames and num will     %contain number of data read successfully ; In YUV 4:2:0; each frame is of size = 1.5*width*height     [deg,num]=  fread( fid_read, 1.5*width*height, 'uchar'); %read frame       %if all frame have been read than print msg and exit     if (num == 0)                 fprintf('read %d frames and exiting',index);                     break;     end     %reading frame for the component Y,U and V; seprately, starts from here     %since Y frame; from 1 to width*Height (of size M*N)     deg_y = deg(1:width*height);         %U frame; from 1+width*height (since, we read Y frame previously from 1 ) to width*Height + width*Height / 4 (of size M*N/4)     deg_u = deg(1+(width*height):(width*height)+(width*height/4));         %V frame; from 1 + 1.25*width*height (since, we read U frame previously upto 1.25*width*height ) to end (of size M*N/4)     deg_v = deg(1+(1.25*width*height) : width*height*1.5 );         %reshaping the Y, U and V from to its size M*N , (M/2)(N/2) and     %(M/2)(N/2) respectively;  reshape will convert 1D array to 2D array     %Reshaping Y frame      orig_y_imgtop1 = reshape(deg_y,width,height)'; % here ' means transpose;         %Reshaping U frame     orig_u_imgtop1 = reshape(deg_u,width/2,height/2)';         %Reshaping V frame     orig_v_imgtop1 = reshape(deg_v,width/2,height/2)';         %applying sobel operator on each frame and store result back to the     %respective arrays     orig_y_imgtop1 =  Sobel_operator (orig_y_imgtop1 );     orig_u_imgtop1 =  Sobel_operator (orig_u_imgtop1 );     orig_v_imgtop1 =  Sobel_operator (orig_v_imgtop1 );         %showing each frame on window seprately and pause for the next frame ;    % imshow (orig_y_imgtop1,[]) ; %here [] means,if it goes outside of 0 to 255 it convert in it    % imshow (orig_u_imgtop1,[]) ;    % imshow (orig_v_imgtop1,[]) ;    % pause;     %wrting back to new video file     %take the transpose     orig_y_imgtop = orig_y_imgtop1';         %convert 2D array to 1D array using (:)     deg_y_new = orig_y_imgtop(:);         %take the transpose     orig_u_imgtop = orig_u_imgtop1';         %convert 2D array to 1D array using (:)     deg_u_new = orig_u_imgtop(:);         %take the transpose     orig_v_imgtop = orig_v_imgtop1';         %convert 2D array to 1D array using (:)     deg_v_new = orig_v_imgtop(:);         %lets concatenate each frame into single vector     deg_new = [deg_y_new; deg_u_new; deg_v_new];         %now write into another yuv file     fwrite(fid_write,deg_new, 'uchar'); end %close everyting fclose (fid_read) ; fclose (fid_write) ;

Need more better solution, Click here

Got Questions? 

Feel free to ask me any question because I'd be happy to walk you through step by step! 

References and External Links

Wikipedia, Imread( ) , rgb2gray( ) , size ( ) , figure , imshow ( ), double    fread () fwrite () fclose() fopen()  .  concatenation of vectors [ ]

For Contact us….. Click on Contact us Tab