The fundamental problem with Mozilla is that it is trying to do database queries in its UI loop, and it wants every single piece of state safely on disk after every single click. This results in a huge amount of disk space to get written to disk as you visit every single click. Now I don't know about you, but if my computer crashes, do I really care if everything up to the last click is safely on disk? I wouldn't care at all if the last 10 or 15 minutes of browser history; I don't care if the link colors are a little off due to a some history getting lost on a system crash.
Compounding this is the fact that SQLlite was never intended to be a high performance database. It was designed for portability, and ease of setup. Which is fine, but it means that SQLlite uses many more I/O's and issues many more fsync()'s than would be strictly necessary. (In fact, Oracle doesn't issue a single fsync operation on a transaction commit; it uses direct I/O instead.)
So even if Firefox manages to get rid of all of the various problems that cause its UI thread to block, this fundamental design mistake will cause them to do excess I/O's, which burns battery and burns SSD write cycles. They would be much better off if they kept all of their state in memory, and 10-15 minutes, updated the on-disk database in a completely asynchronous fashion.
And if that means losing some history on a crash, is the fact that a user has visited one web site, but not another, really that important?
> The fundamental problem with Mozilla is that it is trying to do database queries in its UI loop, and it wants every single piece of state safely on disk after every single click
No, this is not true. Since Firefox 3.5 they have avoided this entirely:
So they've already done what you've suggested. It turned out to cause problems, and was replaced. AFAIK, there's basically no database queries in the Mozilla UI loop.
They are using asynchronous queries still, and the problem is that they aren't always asynchronous if there are any long-running queries happening in the background. That's the reason for the hang per the most recent bug.
If temp tables were causing problem, then there's something seriously wrong with SQLlite's in-memory temporary tables (or how they are being used). By definition they shouldn't be blocking...
Firefox was creating memory-backed temporary tables and syncing them to disk; however, the process of regularly syncing to disk while using the browser caused issues. It also added to start-up time requirements.
For example, if you don't want to hold the entire history in memory but only the most recent changes, you have to create a view spanning the temporary table and the on-disk table to do history queries (so the Awesomebar returns results from pages you just visited, as well as ones already written to disk) and things get complicated.
Define "regularly", he's talking about 15 minutes. I don't know that I'd even want it done that often. This is not critical data, let the filesystem layers do their job, at which point the RAM used by keeping a (very small, in any event) working set cached is also irrelevant as the filesystem cache takes over.
And if that means losing some history on a crash, is the fact that a user has visited one web site, but not another, really that important?
Yes, it is. Imagine you're typing a long comment, you've copied it into the clipboard just to be safe, and just before you submit, your OS crashes or you lose power.
Or, imagine you're looking for reference information on an obscure API, and you've finally found a useful page after 9 minutes of searching. Just before the 10 minute sync occurs, you lose power, and for the life of you, you can't remember how you got to that all-important page or what keywords to use. Having an up-to-date history would have let you continue working.
How often has that actually happened to you? I can honestly say that in 15 years of web browsing I've never had either of those problems, and don't know anybody who has.
Seems silly to significantly slow everything down for everybody because of a one in a trillion chance somebody will lose power at an inconvenient time.
And hey, maybe if the browser were faster, you'd have time to submit the comment or bookmark the page before the power goes out...
Just about everything important seems far-fetched until it happens (Who imagined spam botnets when designing SMTP? Who would've thought that Therac-25 operators would get fast enough at entering commands to trigger a race condition?).
How often has that actually happened to you?
Since Firefox syncs often, it can't. But, I have lost long comments and e-mails (before Gmail auto-saved drafts) to page/browser/OS crashes, and I've been saved by the history when I couldn't remember the one site that had some answer I needed.
I can honestly say that in 15 years of web browsing I've never had either of those problems, and don't know anybody who has.
Isn't that exactly the sentiment being complained about in the article?
Seems silly to significantly slow everything down for everybody because of a one in a trillion chance somebody will lose power at an inconvenient time.
And hey, maybe if the browser were faster, you'd have time to submit the comment or bookmark the page before the power goes out...
If anything, experience has taught me that catastrophic failures just love to occur at inconvenient times. For example, demonstrations work when you're alone or with other engineers, but as soon as you present to someone else everything stops working.
Besides, I'm not saying browsers should be slower. I'm saying that they should synchronize more often than once every 10-15 minutes and be prepared for edge cases. I'm of the view that software should, where possible, mimic physical objects -- changes are instantaneous and persistent -- with the reversibility of virtual objects.
Why you've been voted down so much is beyond me. Lots of IDONTLIKEIT in here and recently in general it seems.
> I'm of the view that software should, where possible, mimic physical objects -- changes are instantaneous and persistent -- with the reversibility of virtual objects.
Are you aware of Stanislav's Seven Laws of Sane Personal Computing or the Loper OS website in general? It's a nice read; thought-provoking, even if you don't agree with everything there.
Interesting. For the most part, I agree with the principles there. My long-term (i.e. long past my lifetime) goal is to make software like that seen on Star Trek (minus the homicidal holodeck characters) -- no companies controlling things, no proprietary standards, just simple interfaces to do anything and everything. In the mean time, though, I don't think that full system introspection is economically possible for all systems in all places.
It would be great if SQLlite could provide the option of "NOSYNC" tables that might lose recent updates after a crash. However, in order to ensure that the on-disk file is consistent it at least needs write ordering when it does decide to write to disk - "I don't care when this write gets to disk, as long as when it does, this other write has already gotten there too". There's no API to request that without also requesting data be flushed to permanent storage immediately - be it fsync, sync_file_range or O_DIRECT.
Compounding this is the fact that SQLlite was never intended to be a high performance database. It was designed for portability, and ease of setup. Which is fine, but it means that SQLlite uses many more I/O's and issues many more fsync()'s than would be strictly necessary. (In fact, Oracle doesn't issue a single fsync operation on a transaction commit; it uses direct I/O instead.)
So even if Firefox manages to get rid of all of the various problems that cause its UI thread to block, this fundamental design mistake will cause them to do excess I/O's, which burns battery and burns SSD write cycles. They would be much better off if they kept all of their state in memory, and 10-15 minutes, updated the on-disk database in a completely asynchronous fashion.
And if that means losing some history on a crash, is the fact that a user has visited one web site, but not another, really that important?