Return error for invalid st/lt sps parameters

When num_short_term_ref_pic_sets or num_long_term_ref_pics_sps in
sps is out of range, error is returned.

Bug: 77599438
Test: poc before/after
Change-Id: I652d27849438c3ea2d42fc6e9abb8aeb25ebdc22
(cherry picked from commit b93923e2883a2d1a4eb46290c3b15a84da808d66)