multi-res: work around reference mismatch
In some situations, believed to be an interaction between temporal
scalability and dropped frames, the references available to an
encoder may not be the same references available to its parent.
Previously, the code tried to force the reference frame chosen by
the parent to be used on this frame, even if it was disabled. This
was preventing the pick mode loop from running even once, which led
to a crash.
Attempts to reproduce this bug locally were unsuccessful, so it is
still undetermined what the underlying cause of this issue is. In
the specific case that was failing, the application did not set
any flags which influenced the reference selection on that frame.
ref_frame_flags indicated that the golden frame was disabled,
believed to be because the last frame updated the last and golden
frames, so golden was shut off by default. It's not clear why this
wouldn't have also been true in the lower res encoder, ie, why the
lower res encoder decided to use and/or was allowed to use the
golden frame. We weren't able to debug into the non-crashing
lower res encoder as the crash couldn't be reproduced locally.
Change-Id: Ifb265253d26963ac2afde0e20cf6792788be6af7
diff --git a/vp8/encoder/pickinter.c b/vp8/encoder/pickinter.c
index 6050d15..4206a46 100644
--- a/vp8/encoder/pickinter.c
+++ b/vp8/encoder/pickinter.c
@@ -610,8 +610,34 @@
MB_PREDICTION_MODE parent_mode = 0;
if (cpi->oxcf.mr_encoder_id && cpi->mr_low_res_mv_avail)
+ {
+ int parent_ref_flag;
+
get_lower_res_motion_info(cpi, xd, &dissim, &parent_ref_frame,
&parent_mode, &parent_ref_mv, mb_row, mb_col);
+
+ /* TODO(jkoleszar): The references available (ref_frame_flags) to the
+ * lower res encoder should match those available to this encoder, but
+ * there seems to be a situation where this mismatch can happen in the
+ * case of frame dropping and temporal layers. For example,
+ * GOLD being disallowed in ref_frame_flags, but being returned as
+ * parent_ref_frame.
+ *
+ * In this event, take the conservative approach of disabling the
+ * lower res info for this MB.
+ */
+ parent_ref_flag = 0;
+ if (parent_ref_frame == LAST_FRAME)
+ parent_ref_flag = (cpi->ref_frame_flags & VP8_LAST_FRAME);
+ else if (parent_ref_frame == GOLDEN_FRAME)
+ parent_ref_flag = (cpi->ref_frame_flags & VP8_GOLD_FRAME);
+ else if (parent_ref_frame == ALTREF_FRAME)
+ parent_ref_flag = (cpi->ref_frame_flags & VP8_ALTR_FRAME);
+
+ //assert(!parent_ref_frame || parent_ref_flag);
+ if (!parent_ref_flag)
+ parent_ref_frame = 0;
+ }
#endif
mode_mv = mode_mv_sb[sign_bias];
@@ -620,6 +646,16 @@
vpx_memset(&best_mbmode, 0, sizeof(best_mbmode));
/* Setup search priorities */
+#if CONFIG_MULTI_RES_ENCODING
+ if (cpi->oxcf.mr_encoder_id && cpi->mr_low_res_mv_avail
+ && parent_ref_frame && dissim < 8)
+ {
+ ref_frame_map[0] = INTRA_FRAME;
+ ref_frame_map[1] = parent_ref_frame;
+ ref_frame_map[2] = -1;
+ ref_frame_map[3] = -1;
+ } else
+#endif
get_reference_search_order(cpi, ref_frame_map);
/* Check to see if there is at least 1 valid reference frame that we need
@@ -671,25 +707,6 @@
x->e_mbd.mode_info_context->mbmi.ref_frame = this_ref_frame;
-#if CONFIG_MULTI_RES_ENCODING
- if (cpi->oxcf.mr_encoder_id && cpi->mr_low_res_mv_avail)
- {
- /* TODO: If parent MB is intra, child MB is intra. This is removed
- * now since it cause noticeable quality loss for some test clip.
- * Will come back to evaluate more.
- * if (!parent_ref_frame && this_ref_frame)
- * continue;
- */
-
- /* If parent MB is inter, and it is unlikely there are multiple
- * objects in parent MB, we use parent ref frame as child MB's
- * ref frame. */
- if (parent_ref_frame && dissim < 8
- && parent_ref_frame != this_ref_frame)
- continue;
- }
-#endif
-
/* everything but intra */
if (x->e_mbd.mode_info_context->mbmi.ref_frame)
{
