It'd be cool if you could submit a specific Github commit (or similar) and then have the app store build it and distribute it. Then the user only needs to trust Apple or Google to not be malicious, which they kinda already need to do if they're using both an OS and app store from them.
But then apple couldn't gouge every app developer for an additional 2k. Right now you must buy a Mac to have the privilege of building and uploading iOS apps.