Welcome to the Java Programming Forums


The professional, friendly Java community. 21,500 members and growing!


The Java Programming Forums are a community of Java programmers from all around the World. Our members have a wide range of skills and they all have one thing in common: A passion to learn and code Java. We invite beginner Java programmers right through to Java professionals to post here and share your knowledge. Become a part of the community, help others, expand your knowledge of Java and enjoy talking with like minded people. Registration is quick and best of all free. We look forward to meeting you.


>> REGISTER NOW TO START POSTING


Members have full access to the forums. Advertisements are removed for registered users.

Results 1 to 3 of 3

Thread: Look Ma, no files! Portable object persistence

  1. #1
    Super Moderator Sean4u's Avatar
    Join Date
    Jul 2011
    Location
    Tavistock, UK
    Posts
    637
    Thanks
    5
    Thanked 103 Times in 93 Posts

    Default Look Ma, no files! Portable object persistence

    Where can I store my application data?
    I see a lot of people starting threads because they're struggling to persist (often) a little bit of data from one run of their Java application to the next. The offered solutions are often some kind of jiggery-pokery involving java.io.File. Not only is File handling a cross-platform morass (our favourite OSes can't even agree on path separators, let alone filesystem roots and GoodPlacesToPutApplicationData™), but every so often another limited-resource computing device appears, some of them without a natural filesystem at all.

    So if we can't store data in files, where can we store it? Enter the Java Preferences API in the java.util.prefs package. Preferences is a platform neutral way to store a small amount of simple data. It offers support to store Strings and built-in data types and retrieve them without having to worry how they are persisted. You simply obtain a Preferences instance, store your value-to-be-persisted on a key, exit your app and sleep easy. The next time you start your app, you obtain the Preferences instance and invoke get([key you used previously], [default if nothing's there]). Ideal - for simple use.

    Preferences was designed to be simple and lightweight, so it doesn't include any support for storing Java Objects. Worse, it imposes severe constraints on the size of values that can be stored. My Java 6 API docs say that a String stored by a Preferences instance can be up to 8,192 characters long (Preferences.MAX_VALUE_LENGTH). And only built-in datatypes and Strings may be stored! So if we can accept the constraints of the Preferences class, how would we use it to store arbitrary objects? Here's my first stab at it.

    An Object to String bridge
    We're going to store Objects as Strings. Java offers Serialization as a means of storing and retrieving objects to and from streams, so that's what we're going to do - serialize an Object to a stream. What kind of stream? A stream that creates a valid String object. Then we're going to store that String in a Preferences instance. When we later retrieve the String from the Preferences instance, we have to go back across the bridge - deserialize the String - back into our original Object-that-was-stored.

    We know we can create Strings from byte arrays, and we know we can get a byte array from a String, so it seems like we should be able to use ByteArrayInputStream and ByteArrayOutputStream to bridge between byte arrays and streams. So far so good: Serialization does Object-to-Stream and the ByteArray{In|Out}putStreams will do Stream-to-byte-array. The problem with this first bridge should be evident from the API docs for the String(byte[]) constructor:

    Constructs a new String by decoding the specified array of bytes using the platform's default charset. ...
    The behavior of this constructor when the given bytes are not valid in the default charset is unspecified.
    That's clearly not good. ObjectOutputStream creates a stream of bytes that is a valid object, not a valid String in any Charset. So to bridge this gap we need another pair of Streams between streams of valid Object bytes and streams of valid String bytes. FilterInputStream and FilterOutputStream are a pair of handy Streams for just this purpose.

    What I chose to do (I can post the code if anyone is interested) was to extend Filter{In|Out}putStream to accept a stream of arbitrary bytes on one side and output a 'hex' representation of those bytes on the other side. Every byte that is written to my HexOutputStream (the class I extend from FilterOutputStream) by ObjectOutputStream is converted to a 2-character hex representation, which is in turn written to the ByteArrayOutputStream as 2 bytes. OK, so I'm obviously doubling the size of the stored object, but I can absolutely guarantee that a string of bytes which are '0' to '9' and 'a' to 'f' are valid String objects in any Charset.

    So far so good - my Object to String bridge is now
    mySeriliazableObject -> ObjectOutputStream -> HexOutputStream -> ByteArrayOutputStream -> new String(byte[])
    and my String to Object bridge is
    String.getBytes() -> ByteArrayInputStream -> HexInputStream -> ObjectInputStream -> mySerializableObject
    Let's have a demo.

    A suitable problem
    I'm not very good at remembering birthdays, particularly bad when it's the birthday of my Significant Other. I could write a command-line application to keep a note of the birthday, but I'm a platform butterfly and I know I'm going to change my PC soon, but I can't be sure whether I'll be using an Android phone, an Ubuntu nettop, a LEGO NXT brick, an iPad or a Windo... no, let's not go too far. I want this app to remember the special birthday, so I create a serializable class so that I can create birthday instances from the keyboard and store and retrieve them from some persistence backend. I don't care what the persistence back-end is, so Serializable at least bridges my class to streams. Streams are good. I can write stuff into them, read stuff out of them, and I don't care what's at the other end.
    Here's Birthday.java:

    package com.javaprogrammingforums.domyhomework;
     
    import java.io.*;
     
    import java.util.regex.*;
     
    public class Birthday implements Serializable
    {
      public final static long serialVersionUID = 42l;
      public final static Pattern PAT_BIRTHDAY = Pattern.compile("^([0-9]{1,2})[a-z]{0,} ([JFMASOND][a-z]{2,8})$");
      public enum Month
       {
        January(31), February(29), March(31), April(30), May(31), June(30), July(31), August(31), September(30), October(31), November(30), December(31);
        private final int MAX_DAYS;
        Month(int maxDays)
        { MAX_DAYS = maxDays; }
        public void validate(int day) throws IllegalArgumentException
        {
          if (day < 1 || day > MAX_DAYS)
            throw new IllegalArgumentException("Not a valid day in " + this + ": " + day);
        }
       };
      private final int day;
      private final Month month;
      public Birthday(int dayOfMonth, Month m) throws IllegalArgumentException
      {
        m.validate(dayOfMonth);
        day = dayOfMonth;
        month = m;
      }
      public static Birthday parse(String s) throws IllegalArgumentException, NullPointerException
      {
        if (s == null)
          throw new NullPointerException();
        Matcher mat = PAT_BIRTHDAY.matcher(s);
        if (!mat.matches())
          throw new IllegalArgumentException("Bad format for birthday: '" + s + "'");
        try
        {
          Month m = Enum.valueOf(Month.class, mat.group(2));
          return new Birthday(Integer.parseInt(mat.group(1)), m);
        }
        catch (Exception e)
        {
          throw new IllegalArgumentException("Bad month: '" + mat.group(2) + "'", e);
        }
      }
      public String toString()
      {
        return month + " " + day;
      }
    }
    Marvellous, a Birthday class that stores only a day-of-month and a month as an int and an Enum. Should be plenty good enough for a demo of Object-into-Preferences.

    Now for the application. The application PreferencesSOBirthday starts up, tries to retrieve a Birthday from the Preferences back-end, displays it as a reminder if there's one in there. If there isn't a Birthday object in the Preferences instance, it asks you for one and stores it. Nowhere in the code will you find any mention of a File. For that matter, nowhere in the code will you find any explicit choice of persistent storage at all - all it does is to obtain a Preferences instance that's suitable for the user/application, and stores the object. The next time you run the application - tadaa - the object is magically there!
    package com.javaprogrammingforums.domyhomework;
     
    import java.io.*;
     
    import java.util.prefs.Preferences;
     
    public class PreferencesSOBirthday
    {
      private final static String PREFS_KEY_BIRTHDAY = "significant.other.birthday";
      public static void main(String[] args) throws Exception
      {
        /* may throw SecurityException - just die */
        Preferences prefs = Preferences.userNodeForPackage(PreferencesSOBirthday.class);
        if (args.length > 0 && "reset".equalsIgnoreCase(args[0]))
        {
          System.out.println("Command line argument 'reset' found - data removed.");
          prefs.remove(PREFS_KEY_BIRTHDAY);
        }
        // fetch the string-encoded object
        String sSOBD = prefs.get(PREFS_KEY_BIRTHDAY, null);
        if (sSOBD != null)
          try
          {
            /* check preferences string actually contains a Birthday */
            getBirthday(sSOBD);
          }
          catch (Exception e)
          {
            System.out.println("Ouch, my brain hurts!");
            sSOBD = null;
            prefs.remove(PREFS_KEY_BIRTHDAY);
          }
        while (sSOBD == null)
        {
          System.out.print("OMG are you in trouble, I don't know your SO's birthday - enter it now: ");
          BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
          String sBirthday = br.readLine();
          if (sBirthday == null)
          {
            System.out.println("OK, be like that then");
            System.exit(0);
          }
          try
          {
            Birthday b = Birthday.parse(sBirthday);
            sSOBD = toPreference(b);
            prefs.put(PREFS_KEY_BIRTHDAY, sSOBD);
          }
          catch (Exception e)
          {
            System.out.println(e);
            sSOBD = null;
          }
        }
        System.out.println("Be prepared - your Significant Other's birthday is " + getBirthday(sSOBD));
      }
      /* String to Birthday bridge */
      private static Birthday getBirthday(String s) throws NullPointerException, IOException, ClassNotFoundException
      {
        return (Birthday)new ObjectInputStream(new HexInputStream(new ByteArrayInputStream(s.getBytes()))).readObject();
      }
      /* Birthday to String bridge */
      private static String toPreference(Birthday b) throws IOException
      {
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        ObjectOutputStream oos = new ObjectOutputStream(new HexOutputStream(baos));
        oos.writeObject(b);
        oos.close();
        return new String(baos.toByteArray());
      }
    }
    I've tested this on Ubuntu and Windows XP. The code was compiled on Ubuntu and I copied the class files over to Windows XP and they worked exactly the same there. You'll have to run the application once to be told there's no data and enter a date - the application exits. Run it again, it always 'remembers' the date. Run it with a single command line argument of 'reset' and it'll remove the object (OK, the encoded string value) from the Preferences instance. Here's a command line to run the app:
    java com.javaprogrammingforums.domyhomework.PreferencesSOBirthday
    Remember to run the application at least twice, or you won't see persistence 'in action'. Here's a command line to reset (remove) the stored data:
    java com.javaprogrammingforums.domyhomework.PreferencesSOBirthday reset

    Where next?
    It shouldn't be difficult to turn this trivial example into a general-purpose Object Persistance engine based on the Preferences API, but I would caution against it because of the arguments put forward by Sun in the first article linked above. Preferences really isn't for arbitrary Object storage, it's for storing simple configuration data. Still, if a solution such as the one above fits your requirements and you are disciplined enough to avoid the temptation to base a mega-project on it instead of using a proper persistence back-end, then I think it has some kilometrage. Enjoy.

    edit: Some time later I realised you can't actually run this without the HexStreams classes. I'm not going to post those right now, because I think they're not necessary to understand what's going on. Attached instead is an executable jar file - execute it with 'java -jar SOBirthday.jar'

    nother edit: Just seen an article at IBM by an author who can read the API docs and notice the putByteArray() method in Preferences! That would make the code slightly less complicated, but not much. The IBM article also suggests breaking your objects up into under-8KB chunks, but I think if you're going to go that far, you probably need a proper DB: http://www.ibm.com/developerworks/ja...api/index.html
    Attached Files Attached Files
    Last edited by Sean4u; August 19th, 2011 at 11:08 AM. Reason: spotted another method at IBM

  2. The Following 5 Users Say Thank You to Sean4u For This Useful Post:

    copeg (August 19th, 2011), Gigggas (April 11th, 2012), JavaPF (August 19th, 2011), newbie (August 19th, 2011), Norm (September 29th, 2011)


  3. #2
    mmm.. coffee JavaPF's Avatar
    Join Date
    May 2008
    Location
    United Kingdom
    Posts
    3,336
    My Mood
    Mellow
    Thanks
    258
    Thanked 294 Times in 227 Posts
    Blog Entries
    4

    Default Re: Look Ma, no files! Portable object persistence

    Great post Sean4u. I've promoted it to a front page article.
    Please use [highlight=Java] code [/highlight] tags when posting your code.
    Forum Tip: Add to peoples reputation by clicking the button on their useful posts.

  4. #3
    Super Moderator Sean4u's Avatar
    Join Date
    Jul 2011
    Location
    Tavistock, UK
    Posts
    637
    Thanks
    5
    Thanked 103 Times in 93 Posts

    Default Re: Look Ma, no files! Portable object persistence

    I've just started a little project at Google Code based on this idea:

    juppmap - Java object persistence with java.util.prefs.Preferences - Google Project Hosting

    JuppMap uses putByteArray() instead of the redundant HexStream above, copes with large objects (4MB seems no problem in tests) and uses Java Generics to avoid any nasty casting. There's also a compressing JuppMap that uses GZIP...Stream, though the backend is still 6-bits to the byte.

    There are a few demo utility programs to demonstrate use of JuppMap, including a short address book program and a utility to list all JuppMaps in the Preferences store and all their keys. There are javadocs too!

    If there's something I should be doing at Google Code (but haven't) with JuppMap, let me know. Should I be posting a downloadable jar file?

Similar Threads

  1. Seraching through files in a folder for a pattern match inside the files.
    By dazzabiggs in forum What's Wrong With My Code?
    Replies: 4
    Last Post: May 2nd, 2011, 08:35 AM
  2. Reading from ResultSet to Object and from object Object Array
    By anmaston in forum What's Wrong With My Code?
    Replies: 4
    Last Post: April 7th, 2011, 06:11 AM
  3. Replies: 1
    Last Post: March 22nd, 2011, 06:59 PM
  4. Table with two primary keys(hibernate persistence)
    By jan_ed123 in forum Java Theory & Questions
    Replies: 0
    Last Post: August 20th, 2010, 02:36 AM
  5. Persistence causing problems with JButton 2D Array
    By easyp in forum What's Wrong With My Code?
    Replies: 1
    Last Post: April 21st, 2010, 12:21 PM