You may need to force close the CBS All Access app. 

On your Apple TV 4:
1. While on the Home screen, double-click the Home button on the Siri Remote. 
2. Swipe to the right on the touch surface to find the CBS app.
3. Swipe up on the touch surface to force the app to close.