OpenJDK Challenge

Internal Toolkit Implementation Tutorial

Introduction

By using the public and semipublic peer interfaces, it is possible to implement graphics backends for AWT and Java2D, as described in External Toolkit Implementation, a walkthrough. However, this requires implementing the complete graphics stack, including the Toolkit, Graphics, and a part of or all of fonts and image support. This can be quite a large task. On the other hand, OpenJDK already has a lot of code that can be reused to implement most of the above mentioned subsystems. This tutorial describes the internal interfaces and classes that are relevant for implementing a graphics backend for a new system.

Summary

Toolkit implementation

The starting point for implementing a graphics backend should be to implement java.awt.Toolkit. In OpenJDK there is a base implementation of this class called sun.awt.SunToolkit, from which all of the internal toolkit implementations are derived, and upon which new implementations may be based. This provides implementations for the image subsystem as well as a couple of useful default implementations for other areas. It is still necessary to provide implementations for most of the AWT widget peers. (However, it will be possible in the future to plug in portable platform independent Swing based widget peers.)

Methods with usable default implementation in SunToolkit

  
    public FontMetrics getFontMetrics(Font font);
    public String[] getFontList();
  

Those two methods provide the necessary font support and hook into the font subsystem of OpenJDK. This only needed to be implemented when a new font subsystem is needed.

  
    public PanelPeer createPanel(Panel target);
    public CanvasPeer createCanvas(Canvas target);
  

These calls provide default implementations for the Panel and Canvas peers. Both use the default peers for lightweight components, so the functionality might be limited. Some applications expect Panel and Canvas to be real heavyweights, so one might need to override this.

  
    public Image getImage(String filename);
    public Image getImage(URL url);
    public Image createImage(String filename);
    public Image createImage(URL url);
    public Image createImage(byte[] data, int offset, int length);
    public Image createImage(ImageProducer producer);
    public int checkImage(Image img, int w, int h, ImageObserver o);
    public boolean prepareImage(Image img, int w, int h, ImageObserver o);
  

These methods implement the image related methods of java.awt.Toolkit and hook into the image subsystem of OpenJDK. Unless a new image subsystem is needed, these methods need not be override.

  
    protected EventQueue getSystemEventQueueImpl();
  

This method provides a default system event queue, that should be perfect for most toolkit implementations.

  
    protected synchronized MouseInfoPeer getMouseInfoPeer();
  

This provides a default MouseInfoPeer implementation. However, this is implemented to map to native methods which need to be implemented by each backend. It is a good idea to override this and provide a custom MouseInfoPeer.

  
    public boolean isModalityTypeSupported(Dialog.ModalityType modalityType);
    public boolean isModalExclusionTypeSupported(Dialog.ModalExclusionType exclusionType);
  

These predicates provide default implementation for both abstract methods in java.awt.Toolkit. One may override these, depending on the modality support in the backend.

  
    public KeyboardFocusManagerPeer createKeyboardFocusManagerPeer(KeyboardFocusManager manager) throws HeadlessException;
  

This creates a default KeyboardFocusManagerPeer, which might or might not be ok for a new backend.

  
    public Dimension getScreenSize();
  

This provides a default implementation of getScreenSize(). However, it only maps to two new abstract methods getScreenWidth() and getScreenHeight(), which one needs to implement.

Additional abstract methods in SunToolkit

SunToolkit also defines a couple of additional abstract methods over java.awt.Toolkit for extended functionality. One needs to provide implementation of those methods in a new backend. The following lists summarizes the new abstract methods.

  
    public abstract TrayIconPeer createTrayIcon(TrayIcon target)
        throws HeadlessException, AWTException;
    public abstract SystemTrayPeer createSystemTray(SystemTray target);
    public abstract boolean isTraySupported();
  

These methods provides tray icon support. For systems without tray icon support, it is ok to return false in isTraySupported(), and null in the other two methods.

  
    protected abstract int getScreenWidth();
    protected abstract int getScreenHeight();
  

Determines the screen width and height.

  
    protected abstract boolean syncNativeQueue(final long timeout);
  

Platform toolkits need to implement this method to synchronize with the native queue. Refer to the API documentation for details.

  
    public abstract void grab(Window w);
    public abstract void ungrab(Window w);
  

Implements mouse grabbing and ungrabbing for Swing's PopupMenu. Refer to the API documentation for details.

  
    public abstract boolean isDesktopSupported();
  

This predicate should return true if the Desktop API is supported, false otherwise. When the Desktop API is not supported, the method createDesktopPeer() (declared abstract by java.awt.Toolkit) can return null.

Provide top level AWT widget peers

As part of implementing the Toolkit, one also has to provide peer implementations for the 3 top level AWT widgets:

  
    public abstract WindowPeer createWindow(Window target)
        throws HeadlessException;
    public abstract FramePeer createFrame(Frame target)
        throws HeadlessException;
    public abstract DialogPeer createDialog(Dialog target)
        throws HeadlessException;
  

These are mandatory, because without top level widgets it is not possible to create a graphical application. Please refer to the API documentations of the widget peers and External Toolkit Implementation, a walkthrough for details.

It should be noted, that the top level widgets also provide the entry points for the graphics pipeline.

There is no common base implementation for these interfaces.

Provide AWT widget peers

Also, as part of implementing the Toolkit, one can provide AWT widget peers for all other AWT components. This is only needed for supporting the AWT widgets. For Swing applications these are not used. Refer to the API documentation for details.

In the future, it will be possible to use portable Swing based widget peers.

Providing a GraphicsEnvironment

The other central entry point for implementing a graphics backend is the java.awt.GraphicsEnvironment and its related classes java.awt.GraphicsDevice and java.awt.GraphicsConfiguration. In OpenJDK there is a common base implementation for java.awt.GraphicsEnvironment, called sun.java2d.SunGraphicsEnvironment, which provides convenient default implementations for a couple of subsystems. For the other two classes there is no default implementation.

Useful default implementations

  
    public synchronized GraphicsDevice[] getScreenDevices();
    public GraphicsDevice getDefaultScreenDevice();
  

These provide default implementations for screen device creation and handling. These use the two new abstract methods getNumScreens() and makeScreenDevice() to actually create the screen devices.

  
    public Graphics2D createGraphics(BufferedImage img);
  
This method creates a Graphics2D for drawing on BufferedImages. This hooks into the graphics pipeline of OpenJDK. This does not need to be overridden, unless one provides a BufferedImage implementation in the GraphicsConfiguration.createCompatibleImage() methods, that are not compatible with the standard pipeline.

  
    public Font[] getAllFonts();
    public String[] getAvailableFontFamilyNames(Locale requestedLocale);
    public String[] getAvailableFontFamilyNames();
  

These routines Implement font support of the GraphicsEnvironment class. This hooks into the font subsystem of OpenJDK, and does not need to be overridden, unless one provides a new font subsystem.

New abstract methods

  
    protected abstract int getNumScreens();
    protected abstract GraphicsDevice makeScreenDevice(int screennum);
  

These are used to create the screen devices.

  
    public abstract boolean isDisplayLocal();
  

This is used to determine if a display is local or remote.

GraphicsDevice and GraphicsConfiguration

As part of implementing the GraphicsEnvironment, one also must provide implementations of java.awt.GraphicsDevice and java.awt.GraphicsConfiguration. There are no common base implementations for these two classes, so one must follow the API specification for the public classes.

Implement graphics pipeline

ComponentPeer (especially a new WindowPeer implementation) requires the implementation of getGraphics():

  
    Graphics getGraphics();
  

The declaration of this method is slightly misleading, because most applications expect the subclass Graphics2D, so it makes sense to provide a Graphics2D object here. (It makes sense in future versions of the peer interface to require Graphics2D).

In theory, it is possible to implement a Graphics/Graphics2D based solely on the public API specification (as shown in the Caciocavallo prototype for the external API). However, this is a really huge task. It makes a lot of sense to reuse existing code from the OpenJDK graphics pipeline because a lot of portable generic and less generic rendering loops already exist.

The central class of the graphics pipeline is SunGraphics2D, a concrete final implementation of java.awt.Graphics2D, which implements a lot of generic functionality (bookkeeping of graphics state) and hooks into a rendering pipeline for actual rendering.

The usual implementation of getGraphics() looks like this:

  
  public Graphics getGraphics()
  {
    SunGraphics2D sg2d = new SunGraphics2D(surfaceData, foreground, background,
                                           font);
    return sg2d;
  }
  

where foreground and background are java.awt.Color objects specifying the foreground and background colors respectively and font is a java.awt.Font object specifying the font. The most important argument here is the surfaceData object, which is an implementation of sun.java2d.SurfaceData.

Implement SurfaceData

SurfaceData is the central class that defines the rendering pipeline for a specific drawing surface. A drawing surface is usually something like an on-screen window area, an off-screen backing image (volatile image) or similar. Each drawing surface usually has exactly one SurfaceData object associated.

Abstract methods to be implemented by specific SurfaceData implementations

  
    public abstract GraphicsConfiguration getDeviceConfiguration();
  

This must return a GraphicsConfiguration object that describes the graphics configuration of the destination drawable.

  
    public abstract Rectangle getBounds();
  

Returns the bounds of the drawable.

  
    public abstract Object getDestination();
  

Returns the destination, that is either the AWT component or the image to which this SurfaceData draws. This is used only by Swing's RepaintManager.

  
    public abstract SurfaceData getReplacement();
  

Changes in the configuration of a surface require the invalidation and creation of a new SurfaceData object. Please refer to the API documentation of this method for details. This probably do not have to be changed.

  
    public abstract Raster getRaster(int x, int y, int w, int h);
  

This method returns a Raster object which provides access to the pixel data of the surface. This probably do not need to be provided, as long as all rendering with new rendering loops. However, if one would like to use the (slow) generic loops that operate on Rasters, then one can save time by provding this.

Pipes and Loops

Most likely want, validatePipe() will need to be overriden in a new SurfaceData subclass:

  
    public void validatePipe(SunGraphics2D sg2d);
  

This method is supposed to setup the various XXXpipe objects in the SunGraphics2D instance.

What are pipes and loops?

To put it simple, pipes are objects that are used by SunGraphics2D to perform the actual rendering. Pipes can do the rendering directly, i.e., EscherRenderer is a PixelDrawPipe that implements primitive drawing operations by calling directly into the Escher X11 API, or they can apply some transformation to the drawing call and delegate to some other pipe. This usually happens in complex drawing calls. Some pipes use loops for rendering. Loops are generic (or less generic) implementations for some rendering primitives like lines, rectangles, etc.

The method validatePipe() should check the state of the SunGraphics2D object (the various state fields) and setup the pipeline accordingly. Usually, in the primitive case (no composite, paint, etc) one would setup drawpipe and fillpipe that use graphics primitives of the graphics backend. In the complex cases, one usually wants to use the pipes that are setup by the superclass.

It is most useful to study existing implementations of this method, for example the XDrawableSurfaceData.validatePipe() of the Caciocavallo-NG prototype or the implementations included in OpenJDK.

Setup blit loops

There is one special type of loops that one needs to take care of, that are the blit loops (defined by sun.java2d.loops.Blit). Blits perform copying of rectangles of pixels from one surface to another (or within one surface). This is used by copyArea(), the various drawImage() methods and some of the other loops.

One detail sets blits apart from other loops. They are not simply installed as a value of a certain field; they are registered into a special datastructure, which enables them to be looked up by the rendering pipeline, according to their supported source and destination surface types. This registry data structure is a kind of hierarchy. The top node is always a generic (but slow) blit registered, which serves as fallback if no other blit operation can take over a certain operation. Then, one can register blit operations for more specific surface types.

One can reuse a set of standard surface types (defined in sun.java2d.loops.SurfaceType) when they match a given surface, or one can derive new subtypes for surfaces that are not predefined. This is done not by subclassing SurfaceType, but rather by calling deriveSubType() on an existing SurfaceType, such as in the following code (from the Caciocavallo prototype):

  
  static final SurfaceType Escher =
    SurfaceType.Any.deriveSubType(DESC_ESCHER_INT_RGB);
  static final SurfaceType EscherInt =
    Escher.deriveSubType(DESC_ESCHER_INT);
  static final SurfaceType EscherIntRgb =
    EscherInt.deriveSubType(DESC_ESCHER_INT_RGB);
  

This defines a subhierarchy, that uses the type 'Any' as supertype (this is the most generic type), and derives a generic 'Escher' supertype (for all Escher-based surfaces) as well as more generic subtypes 'EscherInt' and 'EscherIntRgb' (for integer-based and integer-based-RGB Escher types).

To register specific Blit loops, one can call in the static initializer of SurfaceData subclass a static register() method of a new BlitLoops class like this (from XDrawableSurfaceData.java):

  
  static
  {
    EscherBlitLoops.register();
  }
  

and then define this register method like this (from EscherBlitLoops.java):

  
  static void register()
  {
      GraphicsPrimitive[] primitives =
      {
        new EscherBlitLoops(XDrawableSurfaceData.EscherIntRgb,
                            XDrawableSurfaceData.EscherIntRgb, true),
        new EscherBlitLoops(SurfaceType.Any, XDrawableSurfaceData.EscherIntRgb, true)
      };
      GraphicsPrimitiveMgr.register(primitives);
  }
  

This creates an array with blit loops to be registered. In this case one has one loop that is specific to copy from EscherIntRgb to EscherIntRgb surfaces only, and one more generic loop to transfer from any other surface (e.g. BufferedImage) to an EscherIntRgb surface. Finally, the blit loops are registered by calling GraphicsPrimitiveMgr.register() with this array as argument.

Volatile image support

The ComponentPeer interface also mandates the implementation of createVolatileImage(int,int):

  
    VolatileImage createVolatileImage(int width, int height);
  

as does the GraphicsConfiguration class:

  
    public VolatileImage createCompatibleVolatileImage(int width, int height);
  

Infact, the authors recommend implementing the ComponentPeer method as follows:

  
  @Override
  public VolatileImage createVolatileImage(int width, int height)
  {
    GraphicsConfiguration gc = awtComponent.getGraphicsConfiguration();
    return gc.createCompatibleVolatileImage(width, height);
  }
  

However, the GraphicsConfiguration method createCompatibleVolatileImage() is not an abstract method, so it is not necessary to override it. The default implementation creates a SunVolatileImage which in turn uses a VolatileSurfaceManager to perform its platform specific tasks.

So in order to support VolatileImages, one only need create a subclass of sun.awt.image.VolatileSurfaceManager and a subclass of sun.java2d.SurfaceManagerFactory. Then one only needs to tell the SurfaceManagerFactory to use the new factory to create VolatileSurfaceManager instances for the new volatile images. The authors recommend doing this in the static initializer of a new GraphicsEnvironment implementation:
  
    static {
        SurfaceManagerFactory.setInstance(new MySurfaceManagerFactory());
    }
  

The VolatileSurfaceManager class declares the following abstract methods:

  
    protected abstract boolean isAccelerationEnabled();
    

to determine if acceleration is enabled or not;

  
    protected abstract SurfaceData initAcceleratedSurface();
  

to initialize and return the accelerated surface;

  
    public abstract SurfaceData getPrimarySurfaceData();
  

to returns the SurfaceData object that is used to draw on the volatile image; and

  
    public abstract SurfaceData restoreContents();
    

to restore the content if it has become invalid.

There are also a lot of other useful methods that one might or might not want to override. See the excellent API documentation for VolatileSurfaceManager for details.

Fonts

The Toolkit is also resposible for creating a Font peer. However, this interface is not actively used anymore. There is an internal API for implementing fonts, which consist of a couple of classes and interfaces:

  
sun.font.Font2D
sun.font.FontManager
sun.awt.FontConfiguration
    

In order to implement a new font backend one must provide a FontManager and all the referenced support classes.

It is possible to reuse most of the existing FontManager code. For the most portable approach, just implement the methods described in the FontManager interface. Currently, there is an ongoing work to make this interface more portable and more documented.

To use the new FontManager implementation, pass a system property to the VM:

  
java -Dsun.font.fontmanager=your.fontmanager.ClassName
  

One can also set this property in the static initializer of a GraphicsEnvironment implementation:

  
  static
  {
    System.setProperty("sun.font.fontmanager", "your.fontmanager.ClassName");
  }

Relevant classes


java.awt.GraphicsConfiguration
java.awt.GraphicsDevice
java.awt.peer.WindowPeer
java.awt.peer.FramePeer
java.awt.peer.DialogPeer
java.awt.peer.*
sun.awt.SunToolkit
sun.awt.KeyboardFocusManagerPeerImpl
sun.awt.image.(Volatile)SurfaceManager
sun.java2d.SunGraphicsEnvironment
sun.java2d.SurfaceData
sun.java2d.loops.Blit
sun.java2d.pipe.PixelDrawPipe
sun.java2d.pipe.PixelFillPipe
sun.java2d.pipe.ShapeDrawPipe
sun.java2d.pipe.GlyphListPipe
sun.java2d.SurfaceManagerFactory
sun.font.FontManagerBase
sun.awt.FontConfiguration