To cancel your CBS All Access subscription, simply do the following:

1. Go to www.amazon.com/appstoresubscriptions and sign in with your Amazon username and password.
2. Next to the CBS All Access subscription, select "Actions". 
3. Select "Cancel your subscription".

Please note: deleting the app from your Fire TV will NOT cancel your subscription.