I did some digging for you online and found this from a website:
Plug any HDMI device in and use it through the Xbox One
You already knew that your cable box plugs right into the back of the Xbox One via HDMI-in, but did you know that literally any other HDMI device can also go in that slot? "There's always gonna be a little bit of lag. And so the problem with going out and promoting that feature is that it's a total 'your mileage may vary' thing," Penello told us. That said, if you can deal with the (nigh imperceptible) bit of lag, anything from your Apple TV to your Xbox 360 could be plugged right in and played through the Xbox One. At the very least, it frees up an extra HDMI port, right?
So yeah there you go, apparently the bit of lag you're noticing might be normal.