Thanks for your feedback. I have included the Java code in case anything caused the bad results, would be glad to hear about optimizations.

I am developing an open source middle ware solution client/server to connect to BerkeleyDB (OctopusDbm) and in paralel I am Founder and CEO of Buscaplus, an Internet search engine with so far like 1TB in BerkeleyDB databases, so I am kind of Berkeley lover. So I am a lover but also my requirements are very strict since I need lots of processing my little economic resources.

As part of OctopusDbm, which will be the cloud solution for Buscaplus to grow to millions of web pages and many TBs in size, I have to make a decision weather to keep the current Python client/server or move the server to something else, mainly in part since Octopus will have stored procedures in code or any other smart solution, and Python we all know is slow compared to C and Java. So I started to think about it some weeks ago, I was totally convinced that a pure 100% Java will do the job, and already planned for it. But did not do the job even for test #1, so my deception was very big.

Take the chance to ask you some questions:
- Which language would you choose to implement a server? Agree with me that C++ inside Linux?
- Is it possible to force BerkeleyDB JE to physically write to the file you define in the open() and not the software decide on file names and sizes of physical data files?

So that was the purpose of this post, letting people follows me what is going on, by no means telling the Internet community that BerkeleyDB JE is a bad product, which seems is the message you got from the post. Is a nice product for certain people with Java solutions, but not me.

Best regards

Did you not notice the java.util.collections API? That supports HashMaps and all the other collections types as well. Yes, the underlying database is a btree, but do you really care? Btrees have always been the fastest access method in the core product, even in very large databases (PBs). Your blog post is very short and very strongly worded. If you're going to make such strong statements it might help us to know more details about your testing, your platform, etc. What version of the products did you test? It is not even clear what you mean by “Python”, were you testing the Berkeley DB Python (bsddb) APIs? What version of JE? What version of DB? What version of Java, from what vendor, using what command line options, on what operating system with what file system, what hardware? What Python version? What version of bsddb? How many runs of the test did you perform? Where is the raw data? Did you calculate a standard deviation to see if the tests were highly clustered or being effected by something outside of the JVM?

Jorge, I have been working with Berkeley DB since its inception. I have been an employee working full-time on Berkeley DB products for eight years now. That's four years pre-Oracle and four years post-Oracle. If we “failed” or we “screwed” something up or we have no idea how Java works (I was an employee of Sun Microsystems and worked in JavaSoft before joining Sleepycat) then I'd like to learn more.

Please feel free to bring your tests out into the open and discuss them on Oracle OTN Java forum so that we can help you tune them. Maybe we can help you find that 4x (or better) performance, or maybe we'll learn more about how we've failed our community.

thanks, please do contact me directly if you'd like to talk more,

-greg