分析H.264/AVC各帧间编码模式与最佳模式编码同一宏块（MB）时在比特率、参考帧选择及CBP（coded block pattern）上具有的相似性，对MB各最佳编码模式的比特率和CBP分布进行了统计，根据统计结果分析出比特率和CBP相结合能对编码模式进行比较准确的区分定位。并由此提出一种联合参考帧选择和模式选择的低复杂度RDO（rate distortion optimization）算法。算法利用P16×16模式编码得到的比特率和CBP来确定候选编码模式，用P16×16模式的参考帧来确定其余帧间模式的参考帧。最后将候选模式在候选参考帧中进行运动估计和编码模式的RDO，选出最佳编码模式。实验结果表明，提出算法与H.264/AVC参考软件JM12.0相比，平均比特率增加1.32%、 亮度PSNR减小0.0752dB，编码时间减少71.20%。

In this paper, the similarities of bits, reference frame selection and coded block pattern(CBP) of a coding macroblock(MB) between each inter mode and optimal mode have been investigated. The bits and CBP’s distribution characteristics of a MB which encoded by the optimal mode have been analyzed, and the result shows that the coding modes can be differentiated by bits and CBP. Based on these characteristics, a novel low complexity rate distortion optimization (RDO) algorithm which combination reference frame selection with mode selection RDO is proposed. This algorithm utilizes P16×16 mode’s reference frame to select the candidate reference frames for other inter coding modes, and the candidate modes are selected by bits and CBP of mode P16×16. The optimal mode will be selected in all candidate reference frames and candidate modes with RDO. The algorithm is evaluated using a wide range of test sequences of different classes. Comparing with RDO in JM12.0, the experiments results demonstrate a reduction coding time 71.20% in average with 1.32% of bits increase and 0.0752dB of Luma PSNR loss.