Blog
Adapting the Berkeley Database Java Edition for JSP (2010-02-14)
Overview
The Berkeley Database Java edition as the name might suggest, has its roots with the University of Berkeley. It was originally written in C, and has been ported to Java. The software is now owned by Oracle who has kept it as an open source, although it is available with an enterprise license also.
I initially stumbled upon this gem whilst pondering a problem that I had recently encountered with a client. The JSP website that I was building for them was supposed to be powered by an Oracle database, however mid project it was revealed that their technical department would not allow us access to their Oracle cluster. Enter the Berkeley Database JE...
I was looking for something that could operate locally, however that would offer better performance, and enterprise options than a simple file based solution such as XML persistence. This was partly due to the fact that the site would easily end up holding several hundred thousand records. The Berkeley Database JE offers all of this; performance delivered by a B-tree storage algorithm, hot backup, transactional read/writes, thread safety and more. The JAR file that sets you up is less than 1MB yet your database may grow up to 2TB in size.
In addition it also comes with a good API documentation maintained by Oracle, and is supported by a great community as demonstrated by the associated developer forum. The several questions I had regarding my implementation were all answered within hours of my threads being posted. Whilst the database does come with sample code, the one area that I found it lacking was with regards to my particular solution, JSP.
My Solution
The solution that follows is based on Direct Persistence Layer API and was implemented in conjunction with Weblogic 10g, Weblogic 9.2 (I ran it on both), and Solaris 10. The version of the database was 3.3.83. Whilst Berkeley JE does offer a more enterprise approach to file base persistence then lets say XML or CSV, one area in particular to be wary of is a load balanced production environment. Many enterprise solutions involve clustered environments where load balancing is a factor. This is a traditional issue for local file based solutions, as obviously the same server may not process any sequence of requests from a given client, and therefore not the same local database. While such an issue can be circumvented by housing the database on a network file server that all servers in the cluster can access via the same directory path, another issue then arises with file locking. The first server in the environment will lock the database file for modification consequently blocking all other servers from accessing the database. I see Berkeley JE 4 mentions high availability and load balancing as a new feature, however I'm unsure if the extent of this.
For the sake of demonstration I will keep my example simple. Lets say in the traditional database sense, we have just one database with one table person. This table holds regular user information that a website might store for someone. Of course Berkeley JE doesn't have traditional tables, and communicates in plain old Java objects a.k.a. POJO, so what we will really be reading and writing are person objects.
However first things first. The first thing you need to do to get things running smoothly, and incidentally the only thing that is missing really from the Oracle documentation is how to correctly reference the Berkeley DB from a web application. While you can take the code samples directly and probably get them to work as is with some basic JSP modification you may not be accessing the database correctly. The code samples are designed for J2SE applications with main methods. Therefore the sample programs are run, and then complete, usually within a short period of time. A web application however runs endlessly (hopefully), and therefore requires a different method for accessing the database environment.
For web applications, the database environment should only be opened once, and then stay open and available to all users of the site for simultaneous access. Only one instance of the Environment object can access a given database at one time, and opening and closing the environment has an overhead associated with it. It doesn't seem like much to start off with but as your database file size grows, instantiating the Environment object will take longer and longer. The correct solution to this problem is to open the database environment once when the web application is first started and then to store the reference to the Environment object in the application context.
In order to accomplish this we must configure a context listener for Weblogic. In this context listener we can define actions for when the application context is first created (i.e. when you click 'Start servicing all requests' in the Weblogic administration console, and also when the application context is destroyed ('Force stop', or 'Stop when all work completes'). For this to happen we must tell Weblogic that we have a context listener for it; we do this in the web.xml deployment descriptor. Following on we must also create a Java class to perform the appropriate actions upon context creation and context destruction.
<web-app xmlns="http://java.sun.com/xml/ns/j2ee">
. . .
<listener>
<listener-class>BerkeleyJEContextListener</listener-class>
</listener>
. . .
</web-app>
* BerkeleyJEContextListener.java
* Performs actions when the applicatoin context is both created and destroyed.
* In this case a reference to the Berekely Database JE environment is created
* and stored in the application context. The environment is also closed when
* when the application context is destroyed.
* Lives in the classes subfolder of the WEB-INF directory.
*/
import javax.servlet.*;
import java.io.File;
import com.sleepycat.je.DatabaseException;
import com.sleepycat.je.Environment;
import com.sleepycat.je.EnvironmentConfig;
import com.sleepycat.persist.EntityStore;
import com.sleepycat.persist.StoreConfig;
public final class BerkeleyJEContextListener implements ServletContextListener {
private static File envHome;
public BerkeleyJEContextListener() {
}
public void contextInitialized(ServletContextEvent servletContextEvent) {
ServletContext servletContext = servletContextEvent.getServletContext();
envHome = new File("/opt/myBerkeleyJESite/WEB-INF/classes/schemas/JEDB");
try {
EnvironmentConfig envConfig = new EnvironmentConfig();
StoreConfig storeConfig = new StoreConfig();
envConfig.setAllowCreate(true);
storeConfig.setAllowCreate(true);
Environment envmnt = new Environment(envHome, envConfig);
EntityStore store = new EntityStore(envmnt, "EntityStore", storeConfig);
servletContext.setAttribute("envmnt", envmnt);
servletContext.setAttribute("store", store);
String stat = "good";
servletContext.setAttribute("status", stat);
} catch (Exception e) {
String stat = e.getMessage();
servletContext.setAttribute("status", stat);
}
}
public void contextDestroyed(ServletContextEvent servletContextEvent) {
try{
ServletContext servletContext = servletContextEvent.getServletContext();
EntityStore store = (EntityStore) servletContext.getAttribute("store");
Environment envmnt = (Environment) servletContext.getAttribute("envmnt");
store.close();
envmnt.close();
servletContext.removeAttribute("envmnt");
servletContext.removeAttribute("store");
servletContext.removeAttribute("status");
}catch(Exception e){
System.err.println("Error destroying context attribute: " + e.getMessage());
}
}
}
As this class implements the ServletContextListener interface, it must be compiled with weblogic.jar in the class path. The default location for this file can differ depending on installations however the terminal command should look something like this.
From here we now have 3 application context variables that can be accessed from anywhere within the application in order to perform database actions; envmnt, store, and status. The latter I used as a diagnostic tool to deduce if the database environment had actually been setup correctly. If there were anything going wrong with your web application (especially NullPointerExceptions) the first place to look would be to a simple diagnostic page like this:
* status.jsp
* Prints the status of the Berkeley Java Database.
* Prints weather environment application context variable is null or not.
*/
<%@ page import="com.sleepycat.je.Environment, com.sleepycat.persist.EntityStore, com.sleepycat.je.EnvironmentConfig, com.sleepycat.persist.StoreConfig, java.io.File" %>
<%
response.setHeader("Cache-Control", "no-cache");
EntityStore stre = (EntityStore) application.getAttribute("store");
Environment evmt = (Environment) application.getAttribute("envmnt");
String status = (String) application.getAttribute("status");
response.getWriter().write(status+"<br />");
if(evmt == null){
response.getWriter().write("null");
}else{
response.getWriter().write("not null");
}
%>
Next we must integrate the application context variables created above with the proposed schema. Below is a simple example of the files required to read/write Person objects to the Berkeley database. Available methods are equivalent to the following traditional database operations: insert, update, delete, select record from a primary key, select records from a secondary key.
* Person.java
* Java bean to hold information regarding persons.
*/
package schemas.person;
public class Person{
private String email;
private String password;
private String memberType;
public Person(String email, String password, String memberType){
this.email = email;
this.password = password;
this.memberType = memberType;
}
public String getEmail(){
return email;
}
public String getPassword(){
return password;
}
public String getMemberType(){
return memberType;
}
}
* PersonDA.java
* The Data Accessor for the Person Entities.
*/
package schemas.person;
import java.io.File;
import com.sleepycat.je.DatabaseException;
import com.sleepycat.persist.EntityStore;
import com.sleepycat.persist.PrimaryIndex;
import com.sleepycat.persist.SecondaryIndex;
public class PlaylistItemDA {
// Open the indices
public PersonDA(EntityStore store) throws DatabaseException {
// Primary key for PersonEntityClass classes
pIdx = store.getPrimaryIndex(
String.class, PersonmEntity.class);
// Secondary key for PersonEntity classes
sIdx = store.getSecondaryIndex(
pIdx, String.class, "memberType");
}
// Index Accessors
PrimaryIndex<String,PlaylistItemEntity> pIdx;
SecondaryIndex<String,String,PlaylistItemEntity> sIdx;
}
* PersonEntity.java
* The Person entity class.
*/
package schemas.person;
import com.sleepycat.persist.model.Entity;
import com.sleepycat.persist.model.PrimaryKey;
import static com.sleepycat.persist.model.Relationship.*;
import com.sleepycat.persist.model.SecondaryKey;
@Entity
public class PersonEntity {
// Primary key is pKey
@PrimaryKey
private String email;
// Secondary key is the sKey
@SecondaryKey(relate=MANY_TO_ONE)
private String memberType;
//non-key memebers...
private String password;
//Keyed mutators/accessors
public String getEmail(){
return email;
}
public void setEmail(String data) {
email = data;
}
public void setMemberType(String data) {
memberType = data;
}
public String getMemberType() {
return memberType;
}
//non-keyed mutators/accessors.
public void setPassword(String data) {
password = data;
}
public String getPassword() {
return password;
}
}
* PersonStoreGet.java
* Containts methods for retrieving single or multiple person objects
* from the database.
*/
package schemas.person;
import com.sleepycat.je.DatabaseException;
import com.sleepycat.je.Environment;
import com.sleepycat.persist.EntityStore;
import schemas.person.Person;
import java.util.ArrayList;
import com.sleepycat.persist.EntityCursor;
public class PersonStoreGet {
private Environment envmnt;
private EntityStore store;
private PersonDA sda;
public PersonStoreGet(Environment envmnt, EntityStore store){
this.envmnt = envmnt;
this.store = store;
}
// Retrieve some an object from the store.
public Person run(String pKey) throws DatabaseException {
sda = new PersonDA(store);
PersonEntity sec1 = null;
// Instantiate and retrieve an entity class
sec1 = sda.pIdx.get(pKey);
Person p = new Person(sec1.getEmail(), sec1.getPassword(), sec1.getMemberType());
return p;
}
public ArrayList<Person> runMult(String sKey) throws DatabaseException {
sda = new PersonDA(store);
EntityCursor<PersonEntity> pi_cursor = null;
// Instantiate and retrieve an entity class
pi_cursor = sda.sIdx.subIndex(pKey).entities();
ArrayList<Person> arr = new ArrayList<Person>(0);
try {
for(PersonEntity secEn : pi_cursor){
Person p = new Person(secEn.getEmail(), secEn.getPassword(),
secEn.getMemberType()); arr.add(p);
}
}catch(Exception e){
if (pi_cursor != null) {
pi_cursor.close();
}
}finally{
pi_cursor.close();
}
return arr;
}
}
* PersonStorePut.java
* Containts methods for inserting, deleting or updating Person objects
* from the database.
*/
package schemas.person;
import com.sleepycat.je.DatabaseException;
import com.sleepycat.je.Environment;
import com.sleepycat.persist.EntityStore;
public class PersonStorePut {
private Environment envmnt;
private EntityStore store;
private PersonDA sda;
public PersonStorePut(Environment envmnt, EntityStore store){
this.envmnt = envmnt;
this.store = store;
}
// Populate or update (if the key already exists) the entity store
public void run(String email, String password, String memberType) throws DatabaseException {
sda = new PersonDA(store);
PersonEntity sec1 = new PersonEntity();
sec1.setEmail(email);
sec1.setPassword(password);
sec1.setMemberType(memberType);
sda.pIdx.put(sec1);
}
//delete a key from the database.
public boolean deleteKey(String pKey)throws DatabaseException {
sda = new PersonDA(store);
boolean retVal = sda.pIdx.delete(pKey);
return retVal;
}
}
Finally we can take our schema and use it in a regular JSP file. This JSP could be called by an AJAX function, a form submission or any other JSP invocation that takes place within the application context. Here is a simple example of tying it all together.
* TestPerson.jsp
* Reads and write Person objects to the Berkeley Java Database.
*/
<%@ page import="schemas.person.*, com.sleepycat.je.Environment, com.sleepycat.persist.EntityStore, java.util.ArrayList" %>
<%
EntityStore stre = (EntityStore) application.getAttribute("store");
Environment evmt = (Environment) application.getAttribute("envmnt");
PersonStorePut psp = new PersonStorePut(evmt,stre);
PersonStoreGet psg = new PersonStoreGet(evmt,stre);
//add a person to the database
psp.run("test_user@gmail.com", "mypassword", "vip");
//update that user in the database
psp.run("test_user@gmail.com", "myNewPassword", "vip");
//add another user to the database
psp.run("another_user@hotmail.com", "abc123", "vip");
//retrieve all users with the membership type vip
ArrayList<Person> arr = psg.runMult("vip");
//delete a user based on their email address
boolean retVal = psp.deleteKey("another_user@hotmail.com");
//retrieve a user based on their email address
Person person = psg.run("test_user@gmail.com");
%>