In this article, Anders Isberg from Sony Ericsson’s Technology Reseach department explains more about WebGL and what to think of when you develop 3D web applications for touch-enabled devices. If you scroll down, you will also find three WebGL examples that you can browse to from the Android browser, if you have the latest software on your 2011 Xperia™ phone. You can also check out how it looks in the video above.

***

Earlier this year, I made a blog post about an internal WebGL research project  on how WebGL could be implemented in phones. Based on the feedback we received back then, we decided to launch support for WebGL in the latest software upgrade  we rolled out for the 2011 Xperia™ phones during this autumn and winter. Now let’s take a closer look at what WebGL support means for mobile.

Anders Isberg, Sony Ericsson.

What is required to implement a WebGL application?
Unlike other 3D web technologies, WebGL will not require any plug-in modules. Once the device is WebGL enabled, no further installations are required, and developers can go ahead and explore the WebGL™ functionality. To learn how to develop WebGL applications, we recommend visiting one of the following online resources that gives an introduction on WebGL application development, for example, Learning WebGL or the Khronos WebGL Tutorial.

Due to the fact that WebGL is a low level API, the technical threshold is rather high if you are new to developing WebGL applications. However, there is no reason to be scared by this. There are several JavaScript frameworks available that make it easier to create WebGL applications. Several of these can be found on the Khronos WebGL Wiki.

Navigating and interacting with 3D web applications
Most of the existing WebGL applications available on the web are tailored for a desktop environment. Consequently, those applications are making use of the W3C DOM Mouse and Keyboard events to navigate and control the WebGL application. However, on a mobile device, the touch events must be used to navigate and control the 3D scene. The Android web browser supports three basic touch events that can be used to navigate and control the WebGL application:

• touchstart: generated when the finger is placed on a selected DOM element.
• touchmove: generated when a finger is moved over a DOM element.
• touchend: generated when the finger is removed from a DOM element.

The following code example shows how an event listener is assigned to touchmove event on a WebGL accelerated HTML Canvas element:

// Add an event listener on the WebGL™ canvas
webglcanvas.addEventListener('touchmove', onTouchMove);
function onTouchMove (event) {
// Cancel the default event handler
event.preventDefault();
// Readout the current X and Y coordinate relative the viewport
var currentX = event.targetTouches[0].clientX;
var currentY = event.targetTouches[0].clientY;
}

In many cases, the same code used to handle mouse movements can be reused for touch events. There are also frameworks available to simplify the handling of touch events. For example, there is a jQuery plugin that unifies touch and mouse events across platforms, as well as a Mobile jQuery that has support for high level events like swipe.

Considerations when developing for mobile
Devices such as Xperia™ PLAY, Xperia™ arc, Xperia™ neo and Xperia™ pro are equipped with a 1 GHz Snapdragon™ processor and an Adreno™ 205 graphics processor, so even if the processing power is not in parity with desktop computers, there are enough processing capabilities to create astonishing 3D web applications. Check out the video in the beginning to see some examples of WebGL running on a 2011 Xperia™ phone. You can also check out these examples by browsing to them from the Android browser on your 2011 Xperia™ phone, if you have the latest software. Just click one of the links to the demos below. Please be patient while the demos are loading, it might take some time. 

• WebGL-based interactive 3D demo of the Sony Ericsson logo. 
• WebGL-based interactive 3D demo of Walt Disney. 
• WebGL-based interactive 3D demo of two Ninjas.

When working with WebGL in mobile devices, it is important to take a look under the hood to get good performance. It is essential to consider how the shaders are written, how many polygons that are used in the 3D model, and to select textures carefully. At Developer World, we have published a blog post called Optimising shader performance for the Xperia™ PLAY, which includes several useful tips that can be used to enhance the performance.

One limitation in the Android browser is that web workers are not supported. For some WebGL frameworks that could be a limiting factor. For example, the WebGL framework three.js implements resource loading with web workers. In the WebGL examples listed earlier you can see how this can be solved by using a JavaScript shim layer that emulates web workers. More information about this topic is available at the Google Code Web Workers Emulation project site.

Hopefully this has given you enough information (and confidence!) to get you started with your own WebGL project! Good luck!

Anders Isberg
Master Engineer Technology Research
Sony Ericsson

More information:

• Learn more about WebGL  in Anders Isberg’s previous post.
• Check out the Learning WebGL website. 
• Find out more in the Khronos WebGL tutorial.
• Find more information about our latest software upgrade for the 2011 Xperia™ phones.
• Check out the WebGL-based interactive 3D demo of the Sony Ericsson logo. 
• Check out the WebGL-based interactive 3D demo of Walt Disney. 
• Check out the WebGL-based interactive 3D demo of two Ninjas. 
• Find out more about the WebGL framework three.js.
• Learn more about the Google Code Web Workers Emulation project.
• Learn how to optimise the shader performance for the Xperia PLAY.

Developers for video-related apps will love multimedia experience this Android™ smartphone is capable of delivering – recording camera and video in HD, true colour presentation on the Reality Display with Mobile BRAVIA® Engine, HD TV viewing via a built-in HDMI connection, and a front-facing camera for video chat. The highlight of the experience is the introduction of Sony’s groundbreaking 3D sweep panorama technology, which enables image capture and viewing in 3D by connecting the phone to a 3D TV using the HDMI output.

The multimedia interaction continues with Google Talk™ with Video Chat*, which is pre-installed for instant chatting with friends and family. Facebook inside Xperia™ is also included, which allows users to see, share, like and comment on all their favorite stuff – tracks from the music player, photos from the gallery, and even apps and games from the phone.

The fine performance of Xperia™ neo V lies in its 512 MB RAM and 1 GHz Snapdragon processor with Adreno 205 GPU (based on the Qualcomm MSM8255 platform). The smartphone runs the latest Android platform version (2.3), has a 5 megapixel camera with 16x digital zoom, and a 3.7 inch capacitive, multi-touch screen.

*for phones with front-facing cameras

More information

• Read more about the Xperia™ neo V on the consumer website.
• Watch the YouTube clip of the Xperia™ neo V.
• Download the Xperia™ neo V whitepaper.

• The latest version of Android software (Gingerbread) including Google Talk with Video Chat for smartphones with front-facing cameras.
• Updated Facebook inside Xperia™ which allows users to like, share, and discover on Facebook faster and easier; users also have the ability to share games, apps and songs directly to Facebook™.
• Enhanced camera capabilities from Sony, including 3D sweep panorama
• Connectivity to USB peripherals (mouse, keypad, or game controller) through the Sony Ericsson LiveDock™ multimedia station.
• Swipe to write which allows for text input by swiping the finger from one letter to the next.
• Screen capture from anywhere on the phone.

The Xperia™ software update will be available for most Sony Ericsson devices launched in 2011. The software upgrade will be available in select markets through a phased roll out starting from October 2011, and applicable Xperia™ phone owners will receive a notification instructing them to download the software over the air via 3G or WiFi. Alternately, they have the option to connect to a computer and download it via PC Companion.

For more details on the Xperia™ software upgrade features, check out our YouTube video, or the official press release.

More information

• Read the official Sony Ericsson press release for the Xperia™ software upgrade.
• Watch the YouTube clip demoing the features of the Xperia™ software upgrade.
• Learn more about Facebook inside Xperia™.

Wouldn’t it be cool to have a 3D UI in your Android™ application? I’m Erik Hellman, Lead Software Architect for Android development at Sony Ericsson, and I’ll explain how to make this possible. In this article, I’ll describe how to incorporate 3D with Android’s UI API and OpenGL ES 2.0 graphics, and I provide a bitmap code example that can be used as input to a texture in OpenGL ES 2.0.

Being a game developer for Android is getting easier every day. The devices are getting high-end GPUs, loads of memory and have nice multi-touch displays. There are also several game engines announcing their support for Android, making the time-to-market for game developers much less than it used to be. Creating advanced games for Android no longer requires a huge project involving hundreds of people and spanning over 2 years or more.

While game engines can help us create advanced 3D games, it is not as easy to create an application that uses 3D graphics to enhance the user experience. The engines are focused towards gaming and usually lack a tight integration with the existing Android API and services.

In this post, I will explain how you can combine the best from Android’s UI APIs with hardware accelerated OpenGL ES 2.0 3D graphics. At Droidcon 2011 in Berlin, I presented a session on this topic and showed some demos of how 3D graphics can enhance your applications.

The main problem with doing 3D graphics in Android applications is that the Android API for UI (android.widget.* and android.view.*) has no solution for manipulating the View objects in 3D. OpenGL ES 2.0 on the other hand can do 3D very well, but has no knowledge about Android’s touch and input APIs, lacks layout managers and is really bad at rendering content such as text. The first thing I’ll show is how you can combine the two technologies in the best possible way.

All user-interface components in Android are based on the View class. Regardless if you show a button, text or a list of objects, you will use one of the many sub-classes of View. When a View, and all its children, is drawn there are two things happening. All Views in the hierarchy are first measured and then the layout manager in Android positions them as specified. This is done by a call to the measure and layout method in the View class. All this usually happens behind the scenes in standard Android 2D UI, but if you wish to use your Android View in a 3D scene you need to do this by yourself.

Once measure() and layout() have been called we can draw the View using the draw() method. This method takes a Canvas as a parameter. The Canvas, in turn, can be created to draw on a Bitmap. This is where we tie it all together in a nice utility method:

   bitmap.eraseColor(Color.TRANSPARENT);

The above method lets you reuse the same Bitmap instance and thus reduces the amount of memory allocations you have to do (which takes time).

The Bitmap that we just drew our View to can be used as input to a texture in OpenGL ES 2.0 that can be placed on a 3D model (or, in the simplest case two triangles forming a flat quad).

For examples of how to create a simple OpenGL ES 2.0 application where we use bitmaps as textures, I suggest looking at the samples in the official Android SDK.

This method mainly solves the problem of mixing Android UI content with OpenGL ES 2.0 scenes. There is still a lot you need to build yourself, such a scene graph, handling user input and animations. However, you now have enough to start experimenting and I’ll be posting a follow up on this where we look at how to build a simple scene graph and handling user input.

Note: Droidcon Berlin is a two day barcamp and conference that explores all aspects of developing for Android mobile devices. Happening in Berlin at the Urania Berlin on March 23 and 24, droidcon Berlin will host workshops on Android apps and game development. Over 1200 people are participating in this conference.

More information

• View OpenGL ES 2.0 samples from Android SDK>>>
• View the OpenGL ES 2.0 reference pages from the Khronos Group>>>