Why would he ask you to leave unless you were a liability?

It makes no sense as there are so many other positions that become available as a company grows.

Go learn Product. It doesn't take nearly as long, and you can have just as big of a positive impact.