How can I read Intra coded frames from a video?

I am in need of reading the intra frames of a video to check whether it is edited or not. I could do it with single JPEG images as it can be read using jpeg_read. Read that videos vith MPEG codec is formed out of Intra, Predictive and Bidirectionally predictive frames. Is there any way in MATLAB to extract these frames from a video instead of simple RGB frames?

Answer 1

[Dinesh Iyer]

Zealous,

The notion I, P and B frames only applies to an encoded stream. It determines how the frames must be decoded and the level of compression of each frame. Once you decode the video, each of these frames are converted into either RGB or YUV colorspace. VideoReader currently supports only RGB frames.

If you are asking whether you can read the encoded frames as they are present in the video stream using MATLAB functions such as VideoReader, the answer is no. You can always use low-level file I/O such as fopen, fread and directly parse the bit-stream but that requires knowledge of the file format.

If you are asking whether it is possible to do an operation such as "read me all the P-frames in the video stream", the answer again is not directly. You can do this indirectly using tools such as ffmpeg and MATLAB. I can outline the steps but have no code to offer:

1. Download ffmpeg tools. it has a utility called ffprobe which dumps out info about the file.

2. Using the MATLAB's system command, execute:

ffprobe -show_frames -select_streams v:0 myfile.mpg > C:\Temp\myfileinfo.txt

This will generate a text file with information about each frame. You can do some clever text processing on this file using TEXTSCAN and look for PICT_TYPE=P and identify which frames are P-frames.

3. The number of occurrences of the string "PICT_TYPE=" indicates number of frames.

4. Use these indices to read frames using VIDEOREADER. This will decode the P-frames and return RGB data.

Hope this helps.

Dinesh

[zealous]

Thanks a lot. The reply is very helpful to understand the situation more clear. I think I can proceed with this method. But when I read the frame using video reader and save as JPEG, a decoding and a re-encoding takes place. Right? But I need the original encoded JPEG frame. Is there any way for that? Like, reading an image directly using jpeg_read() other than read using imread(), save as JPEG and then process.

[Dinesh Iyer]

Zealous,

When you read the frame you want, you do not have to save it as a JPEG. You can save it in an uncompressed format such as BMP or use a TIFF file.

Secondly, the nature of the compressed of the I-frames in the video file depend upon the codec used. The I-frames will not be JPEG encoded because each codec uses its own technique for intra-coding frames.

Dinesh

[zealous]

I am about to consider videos with MPEG codec only. They have JPEG encoded intra frames, as I understood. I am in need of the original compressed JPEG frames as it is embedded in the video.

[Walter Roberson]

MPEG-1 uses JPEG encoded intra frames; some of the later MPEG do not. http://www.eetimes.com/document.asp?doc_id=1275884

Note that JPEG encoded frame is not the same thing as jpg file format.

In any case, I am not aware of anyone having programmed MATLAB itself to do this. I have seen a number of students ask questions that imply that some professors believe that the code is available in MATLAB, but I did not find any implementations. It could be done; it is just tiresome to implement and likely to be error prone compared to calling upon an already-written and well-tested library.

[Dinesh Iyer]

It is not possible to get the original encoded frame using any of the video I/O functionality as the purpose of these is to decode the video and return decoded frames.

[zealous]

@walter Roberson "Note that JPEG encoded frame is not the same thing as jpg file format." How it differs? Atleast the key frames (I frames) has JPEG format, na?

[zealous]

@Walter Roberson @Dinesh Iyer Thanks for your helps. I have tried continueing with MPEG. But couldn't complete because I didn't get the originally encoded I frames. Hence chose some other algorithm.

[Walter Roberson]

jpg file format is in part governed by http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=54989, the JPEG File Interchange Format (JFIF) and that data tends to get wrapped by the standard about "Registration of JPEG profiles, SPIFF profiles, SPIFF tags, SPIFF colour spaces, APPn markers, SPIFF compression types and Registration Authorities (REGAUT)"

For example, how to include an EXIF header is part the formal structure of jpg files, but is not part of the section of the standard about DCT and so on. MPEG can incorporate the relatively low level JFIF without having to support jpg files.