I ran into this too. It's because of the index that Sublime builds for its Goto Anything feature. If you never use Goto Anything, in your User Settings:

    "index_files": false