"language brain" or "math brain" are obviously non-scientific terms.

I think most people would agree that problem solving, expressing ideas verbally, and expressing ideas in math are very different skills.