Lists and Nulls in Google App Engine Datastore

Do you know what happens when you store empty List into Google App Engine Datastore ? And do you know what happens if you store a List containing one null value ?

Let's look at a piece of code:

    DatastoreService ds = DatastoreServiceFactory.getDatastoreService();
    ArrayList emptyList = new ArrayList();
    ArrayList hasNullList = new ArrayList();
    hasNullList.add( null );

    Entity ent = new Entity("Test");
    ent.setProperty("nullProp", null);
    ent.setProperty("emptyList", emptyList);
    ent.setProperty("hasNullList", hasNullList);

    System.out.println("ENTITY - before: " + ent);

    Key key = ds.put( ent );
    ent = ds.get( key );

    System.out.println("ENTITY - after: " + ent);

The output of the code is following:

 ENTITY - before: <Entity [Test(no-id-yet)]:
     nullProp = null
     hasNullList = [null]
     emptyList = []
 >
 
 ENTITY - after: <Entity [Test(1)]:
     nullProp = null
     hasNullList = [null]
     emptyList = null
 >

So it means, that GAE does not store an empty list at all ! Instead of an empty list, the property is null.

This is not so big surprise and is not against anything in the official documentation. But the problem is the following:

Can we filter 'null' and '[null]' values separately ?

We will answer quickly - just extend our first example by these lines:

    System.out.println( "count(emptyList==null) = "
        + ds.prepare( 
            new Query("Test").addFilter("emptyList",
                Query.FilterOperator.EQUAL, null)
         ).countEntities());

    System.out.println( "count(hasNullList==null) = "
        + ds.prepare(
            new Query("Test").addFilter("hasNullList",
                Query.FilterOperator.EQUAL, null)
         ).countEntities());

And the output is:

 count(emptyList==null) = 1
 count(hasNullList==null) = 1

This means that it is not possible (this way) to distinguish between null and [null] properties!

So the summary is: do not store nulls into Lists, otherwise you will get into a problem in future.