I stumbled upon a rather interesting article (as I understand it from the developer) about memory use in the third version of Firefox. It describes what has been done to optimize this use:
Reduced memory fragmentation Fixed garbage collection support for circular references Caching system is configured (memory consumption is drastically reduced when using the browser for a long time) Fixed storing compressed and animated images Memory leaks found Now we use a special set of tools for testing memory consumption.
Actually - a link to an article on the use of memory in Firefox 3 (English). It describes the technical details, like what kind of allocator is used, describes the testing methodology (I recommend reading the whole blog on this topic) and so on.