It should not have taken me this long to figure out.
If you're Android Auto connection doesn't drop, but gets weird sometimes, it's likely a settings problem.