Kerflyn's Blog

Well… It's a blog!

Java 8: Now You Have Mixins?

with 18 comments

Java 8 starts to emerge. It comes with a full new feature: functional programming with the lambda expressions. Here I’ll discuss about a feature that is part of the Lambda project (JSR-335): the virtual extension methods, also called public defender methods. This feature will help you to provide a default implementation for the methods that you have declared in your interfaces. This allows, for example, the Java team to add method declarations in existing interfaces, like List or Map. On their side, the developers do not need to redefine their different implementations in Java libraries (like Hibernate). Thus, Java 8 will be theoritically compatible with existing libraries.

The virtual extension methods are a way to provide multiple inheritance in Java. The team who works on Lambda project argues that the kind of multiple inheritance provided with virtual extension methods is limited to behaviour inheritance. As a challenge, I show that with virtual extension methods, you can also emulate state inheritance. To do so, I describe in this article an implementation of mixin in Java 8.

What a mixin is?

The mixins are kind of composable abstract classes. They are used in a multi-inheritance context to add services to a class. The multi-inheritance is used to compose your class with as many mixins as you want. For example, if you have a class to represent houses, you can create your house from this class and extend it by inheriting from classes like Garage and Garden. Here is this example written in Scala:

val myHouse = new House with Garage with Garden

To inherit from a mixin is not really a specialization. It is rather a mean to collect functionalities and add them to a class. So, with mixins, you have a mean to do structural factorization of code in OOP and to enhance the readability of classes.

For example, here is a use of mixins in Python in the module socketserver. Here, the mixins help to declare four socket based servers: a UPD and a TCP services using multi-processes, a UPD and a TCP services using multi-threading.

class ForkingUDPServer(ForkingMixIn, UDPServer): pass
class ForkingTCPServer(ForkingMixIn, TCPServer): pass

class ThreadingUDPServer(ThreadingMixIn, UDPServer): pass
class ThreadingTCPServer(ThreadingMixIn, TCPServer): pass

What are virtual extension methods?

Java 8 will introduce the notion of virtual extension method, also called public defender method. Let’s called them VEM.

A VEM aims to provide a default implementation for a method declared in a Java interface. This helps to add a new method declaration in some widely used interfaces, like the interfaces of the JDK’s Collection API. Thus, all the existing libraries like Hibernate that reimplements the Collection API do not need to rewrite their implementation, because a default implementation is provided.

Here is an example of how you will declare a default implementation in an interface:

public interface Collection<T> extends Iterable<T> {

    <R> Collection<R> filter(Predicate<T> p)
        default { return Collections.<T>filter(this, p); }

}

A naive emulation of mixin in Java 8

Now, we will implement a mixin by using the VEMs. But I have to warn you first: Please, don’t use it at work! Seriously, DON’T USE IT AT WORK!

The implementation below is not thread-safe, subject to memory leak, and completely depends on the way you have defined hashCode and equals in your classes. There is another major drawback that I will discuss later.

First, we define an interface with a default implementation that seems to be based on a stateful approach.

public interface SwitchableMixin {

    boolean isActivated() default { return Switchables.isActivated(this); }

    void setActivated(boolean activated) default { Switchables.setActivated(this, activated); }

}

Second, we define a utility class that owns a map to store associations between the instances of our previous interface and their state. The state is represented by a private nested class in the utility class.

public final class Switchables {

    private static final Map<SwitchableMixin, SwitchableDeviceState> SWITCH_STATES = new HashMap<>();

    public static boolean isActivated(SwitchableMixin device) {
        SwitchableDeviceState state = SWITCH_STATES.get(device);
        return state != null && state.activated;
    }

    public static void setActivated(SwitchableMixin device, boolean activated) {
        SwitchableDeviceState state = SWITCH_STATES.get(device);
        if (state == null) {
            state = new SwitchableDeviceState();
            SWITCH_STATES.put(device, state);
        }
        state.activated = activated;
    }

    private static class SwitchableDeviceState {
        private boolean activated;
    }

}

Here is a use case for the implementation above. It highlights the inheritance of the behavior and of the state.

private static class Device {}

private static class DeviceA extends Device implements SwitchableMixin {}

private static class DeviceB extends Device implements SwitchableMixin {}
DeviceA a = new DeviceA();
DeviceB b = new DeviceB();

a.setActivated(true);

assertThat(a.isActivated()).isTrue();
assertThat(b.isActivated()).isFalse();

“And Now for Something Completely Different”

This implementation seems to work well. But Brian Goetz, the Oracle’s Java Language Architect, has suggested me a puzzle where the current implementation does not work (assuming thread-safety and memory leak are fixed).

interface FakeBrokenMixin {
    static Map<FakeBrokenMixin, String> backingMap
        = Collections.synchronizedMap(new WeakHashMap<FakeBrokenMixin, String>());

    String getName() default { return backingMap.get(this); }
    void setName(String name) default { backingMap.put(this, name); }
}

interface X extends Runnable, FakeBrokenMixin {}

X makeX() { return () -> { System.out.println("X"); }; }

    X x1 = makeX();
    X x2 = makeX();
    x1.setName("x1");
    x2.setName("x2");

    System.out.println(x1.getName());
    System.out.println(x2.getName());

Can you guess what this should display?

Solution to the puzzle

At first sight, there seems to be no problem with this implementation. There, X is an interface with a single method, because the method getName and setName have a default implementation and not the method run from Runnable. So we can generate an instance of X from a lambda expression, that will provide an implementation to the method run, as the method makeX does. So, you expect that this program displays something like:

x1
x2

If you remove the calls to getName, you expect to display something like:

MyTest$1@30ae8764
MyTest$1@123acf34

Where the two lines indicates that makeX produces two separate instances. And this what the current OpenJDK 8 produces (here I have used the OpenJDK 8 24.0-b07).

However, the current OpenJDK 8 does not reflect the eventual behavior of Java 8. To do so, you need to run javac with the special option -XDlambdaToMethod. Here is the kind of output you get once the special option is used:

x2
x2

If you remove the calls to getName, you will get something like:

MyTest$$Lambda$1@5506d4ea
MyTest$$Lambda$1@5506d4ea

Each call to makeX seems to give a singleton instantiated from the same anonymous inner class. Nevertheless, if you look at the directory holding the Java binaries (having carefully removed all *.class files before), you will not find a file named MyTestClass$$Lambda$1.class.

The complete translation of the lambda expression is not really done at compile time. In fact, this translation is done at compile time and runtime. The compiler javac put in place of lambda expression an instruction recently added to JVM : the instruction invokedynamic (JSR292). This instruction comes with all the necessary meta-information to the translation of the lambda expression at runtime. This includes the name of the method to call, its input and output types, and also a method called bootstrap. The bootstrap method aims to define the instance that receive the method call, once the JVM execute the instruction invokedynamic. In presence of lambda expression the JVM uses a particular bootstrap called lambda metafactory method.

To come back to the puzzle, the body of the lambda expression is converted into a private static method. Thus, () -> { System.out.println("X"); } is converted in MyTest into

private static void lambda$0() {
    System.out.println("X");
}

This can be seen if you use the decompiler javap with the option -private (you can even use the -c option to see the more complete translation).

When you run the program, once the JVM tries to interpret the invokedynamic instruction for the first time, the JVM call the lambda metafactory method, describes above. In our example, during the first call to makeX, the lambda metafactory method generates an instance of X and links the method run (from the interface Runnable) dynamically to the method lambda$0. The instance of X is then stored in the memory. During the second call to makeX, the instance is brought back. There you get the same instance as during the first call.

Fix? Workaround?

There is no direct fix or workaround for this problem. Even if it is planned to activate -XDlambdaToMethod by default for Oracle’s Java 8, this is not supposed to appear in the JVM specification. This kind of behavior might change over the time and among vendors. For a lambda expression, what you can only expect is to get something that implements your interface.

Another approach

So our emulation of mixins is not compatible with Java 8. But there is still a possibility to add services to an existing class by using multiple inheritance and delegation. This approach is referenced as virtual field pattern.

So lets start again with our Switchable.

interface Switchable {    boolean isActive();
    void setActive(boolean active);
}

We need an interface based on Switchable with an additional abstract method that returns an implementation of Switchable. The inherited methods get a default implementation: they use the previous getter to transmit the call to the Switchable implementation.

public interface SwitchableView extends Switchable {
    Switchable getSwitchable();


    boolean isActive() default { return getSwitchable().isActive(); }
    void setActive(boolean active) default { getSwitchable().setActive(active); }
}

Then, we create a complete implementation of Switchable.

public class SwitchableImpl implements Switchable {


    private boolean active;


    @Override
    public boolean isActive() {
        return active;
    }


    @Override
    public void setActive(boolean active) {
        this.active = active;
    }
}

Here is an example where we use the virtual field pattern.

public class Device {}


public class DeviceA extends Device implements SwitchableView {
    private Switchable switchable = new SwitchableImpl();


    @Override
    public Switchable getSwitchable() {
        return switchable;
    }
}


public class DeviceB extends Device implements SwitchableView {
    private Switchable switchable = new SwitchableImpl();


    @Override
    public Switchable getSwitchable() {
        return switchable;
    }
}
DeviceA a = new DeviceA();DeviceB b = new DeviceB();

a.setActive(true);

assertThat(a.isActive()).isTrue();
assertThat(b.isActive()).isFalse();

Conclusion

In this post, we have seen two approaches to add services to a class in Java 8 with the help of virtual extension methods. The first approach uses a Map to store instance states. This approach is hazardous for more than one reason: it can be not thread-safe, there is a risk of memory leaks, it depends on the way your vendor has implemented the language, etc. Another approach, based on delegation and referenced as virtual field pattern, uses an abstract getter and let the final implementation describes which implementation of a service it uses. This last approach is more independent from the way the language is implemented and more secure.

The virtual extension method is something new in Java. It brings another mean of expression to create new patterns and best pratices. This article provides an example of such a pattern, enabled in Java 8 due to the use of virtual extension methods. I am sure you can extract other new patterns from them. But, as I have experienced here, you should not hesitate to share them in a view to check their validaty.


EDIT: after sharing in the lambda project mailing list, some modifications has needed to be reported in this post, in a view to reflect the new behavior of Java 8. Thank to Brian Goetz from Oracle and Yuval Shavit from Akiban for their suggestions.

  • [2012-07-11] changed title + renamed “How to emulate mixin in Java 8?” to “Naive emulation of mixin in Java 8″ + added a puzzle
  • [2012-07-12] added a section + removed the conclusion
  • [2012-07-22] added the solution to the puzzle
  • [2012-07-24] added fix/workaround section + some correction
  • [2012-08-13] completed all sections

About these ads

Written by fsarradin

2012/07/09 at 06:55

18 Responses

Subscribe to comments with RSS.

  1. but the state is a static field in a utility class (external to the object)

    Kontra Gergely

    2012/07/10 at 00:15

    • The state is not a static field. This is the association between instance and state that is stored in a static field, not the state. But, yes, the state is external to the object.

      fsarradin

      2012/07/10 at 06:35

  2. Puzzle: your solution … the print out is wrong. :D

    In the code of the puzzle, the print out is defined as:
    System.out.println(x1.getName());
    System.out.println(x2.getName());

    But your solution suggest this:
    System.out.println(x1);
    System.out.println(x2);

    And I think you should add a “How do we fix this” section. :)

    Michi

    2012/07/23 at 22:16

    • Oops! Indeed :/

      I have done some corrections. You can check the content. Thanks :)

      fsarradin

      2012/07/24 at 06:56

  3. So why would the JDK implement lambdas as cached objects, while they are being advertised to the public as sugared anonymous classes? It is an instance where the principle of least surprise is utterly violated.

    I assume they replaced SAMs with Functional Interfaces to be able to do these optimizations, but they clearly forgot about external state. To me it sounds like premature optimization.

    Actually default methods are not required to show this. There may already be libraries out there that associate external state with Functional Interface instances. Passing in a lambda could break existing code!

    Ali Lahijani

    2012/08/13 at 10:07

    • Ali, your comment is interesting. For the external state, I cannot give an answer and I suggest that you contact directly the mailing list of the Lambda Project. Nevertheless, do you an example or two of a library based on such external states?

      For the way the lambdas are translated, indeed, to use cached objects (and this is the case only for completely independent lambdas) is an optimization. In a sense, it is welcome because when you are playing with functional programming, you tend to generate a wide range of functions, if they are not cached. And the JVM suffers in being efficient in this case, even if it provides some other optimizations for this. But, on the other hand, I was as surprised as you were once I discovered that some lambdas are cached. I am agree with the fact that this can lead to errors difficult to understand.

      fsarradin

      2012/08/13 at 19:16

  4. The virtual field pattern reminds me of “Property” classes that is used in the JavaFX API. Nothing new.

    Michi

    2012/08/13 at 14:26

  5. I’d thought that lambda conversions were limited to single-method interfaces. So, how does interface X even qualify for the conversion?

    • In fact, it is not “single-method interface” but “single abstract method (SAM) interface”. As far as I understand it, by “single abstract method interface”, you have to read: an interface where it misses the implementation of one method to represent a complete implementation of a class. For example, those interfaces below are SAM interfaces:

      interface A { void m(); }

      interface B { void n(); void o() default { … } }

      A is a SAM interface because you should define m() to be complete. B is a SAM interface because you should define n() to be complete, knowing that there already is an implementation available for o().

      If you apply a lambda to a SAM interface, the lambda is directly mapped on the abstract method.

      fsarradin

      2012/08/15 at 01:02

      • A functional interface is defined as an interface that has EXACTLY one abstract method (an abstract method is a method that lacks a body). So this means you can have tons of default methods and one abstract method, that’s a functional interface (aka SAM).

        enkkforpr0nnkk

        2013/09/21 at 01:04

  6. very useful informative article for me.. thanks for sharing.. :)

    About Java

    2012/12/26 at 14:57

  7. Isn’t calling it “FakeBrokenMixin” and saying “our emulation of mixins is not compatible with Java 8″ a bit misleading?

    The problem seems to be that makeX() returns the same object twice; the fact that it does so appears to have nothing to do with the fact that it’s using a mixin emulation technique. (If I’ve misunderstood this, please do correct me). Would the same problem occur – cached object returned – even if we weren’t using mixins?

    The interesting and surprising thing here seems to be the behaviour of lambdas.

    bacar

    2013/04/19 at 11:00

    • I am really interested in this answer.

      Enkk

      2013/09/21 at 03:39

    • I ‘m not sure this answer your question. But it shows the optimization introduced by Java 8, in a context with no mixin (knowing that the test below succeeded).


      public Function constant() {
      return _ -> 1;
      }

      public Function constant(int value) {
      return _ -> value;
      }

      @Test
      public void test_references() {
      Function c1 = constant();
      Function c2 = constant();
      Function c3 = constant(1);
      Function c4 = constant(1);

      assertThat(c1).isSameAs(c2);

      assertThat(c1).isNotSameAs(c3);
      assertThat(c3).isNotSameAs(c4);
      }

      Indeed, the problem may appear when we play with the references on lambda expressions. This is the case in my mixin implementation. But it could happen with everything playing with lambda references.

      fsarradin

      2013/09/23 at 20:58

  8. Please, do not write code like this one! It is like asking for troubles. Don’t keep state in singleton maps. If you want to inherit state use abstract classes no default methods, and dont keep state of your object out of it. Default methods are to be used to inherit behaviour, and they should act only on other methods defined in the interface.

    Kamil

    2014/04/14 at 10:47

  9. This is something new for me. I have read couple of Java 8 tutorials about new features but this is clever use of that. By the way, I have also shared couple of Java 8 lambda expressions examples, which complements your post. Your reader may find that useful.

    Regards
    Javin


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 507 other followers

%d bloggers like this: