People do say "can't", as in "can't [accurately] judge a book". This of course leads to the advice that one should not do so, but the phrasing with the word "can't" explains why.