[Achon]

You could use a database management system, like one of oracles. Which would
use some local caching and no doubt utilize the OS virtual memory subsystem.

This would probably give reasonable speed and offer more space for your
ideas. It would also scale well and free up valuable memory for perhaps, even
more caching.