Magnifying Glass AIR 2 application or how to communicate with a Java program from AIR

My favorite feature in Adobe AIR 2 is, by far, Native Processes: the ability to launch and control and communicate with a native process. It could be any executable on the machine where the AIR application is installed. I think this feature opens up a whole new range of AIR applications. When you add this feature to the ability to create socket servers, you have a powerful platform to build RIA applications for desktops.

Once I heard that this feature would make it in AIR 2 I was very excited. Why? Well, back in 2008 when we launched AIR 1.0, my fellow evangelist Serge Jespers created one of the coolest AIR applications for the AIR Tour. It was the smallest video player in the world. Basically it let you watch videos in the application icon from the Dock.

The application is extremely cool, but it has a small issue: it is too damn small to be able to see what’s going on. Being an engineer, I spent some time trying to find an engineering solution. Of course, I could have asked Serge to rewrite the application to make it bigger, but this wouldn’t have been an engineering solution. It would have been something that an accountant or manager would come up with. My solution is to build a second AIR application that can be used to magnify the video played inside the icon. This application would act like a digital magnifying glass.

With AIR 2 I’m finally able to implement the magnifying glass app pretty easily. Below you can see a screenshot of my application in action. It has two windows. The first window is the view port of the magnifying glass. You can see how many frames per second it processes, you can control the amount of zooming, and you can drag it around your screen. The second window displays the magnified image.

The internals

How did I do it? The application has two main parts. One part is the AIR application itself. It renders the UI, controls the view port and the zoom factor, and scales the image. The second part is a Java program that captures a screenshot of a portion of the screen. The Java program is controlled by the AIR application.

Using the NativeProcess and NativeProcessStartupInfo classes from AIR 2, you can launch an executable. In order to communicate with the executable you can use standard input and standard output. I wrote the Java program to output the bytes of the screenshot to standard output. It listens to standard input for commands, such as take a shot, set the viewport, or terminate the program. I compiled the Java program as an executable JAR file and placed in the AIR application root folder.

In order to capture the output of the Java program all you have to do is to register a listener on the NativeProcess instance for the standard output events. When you want to send commands you write bytes to the standardInput property of the same object. Here is a snippet of code, for the complete code have a look at the ScreenShotService class from the AIR application.

private var nativeProcess:NativeProcess;
private var npInfo:NativeProcessStartupInfo;
//setting the arguments for starting the Java program
var arg:Vector.<String> = new Vector.<String>;
arg.push("-jar");
arg.push(File.applicationDirectory.resolvePath("screenshot.jar").nativePath);
arg.push("130");
arg.push("100");

npInfo = new NativeProcessStartupInfo();
//setting the path to the native process
npInfo.executable = new File("/Library/Java/Home/bin/java");
npInfo.arguments = arg;

nativeProcess = new NativeProcess();
nativeProcess.addEventListener(ProgressEvent.STANDARD_OUTPUT_DATA, onStandardOutputData);
//start the process
nativeProcess.start(npInfo);

/**
 * Read the data from the standard ouput.
 * Before reading a png, first you have to read the length of the image
 */
private function onStandardOutputData(e:ProgressEvent):void {
    //reading the available bytes from the standard output buffer of the process
    nativeProcess.standardOutput.readBytes(_processBuffer, _processBuffer.length, nativeProcess.standardOutput.bytesAvailable);
    ...
}

//sending take command to the Java process
nativeProcess.standardInput.writeMultiByte("take\n", "utf-8");

This is the relevant Java code (you can find the complete code inside the source folder of the application, ScreenShot.java):

/**
 * @param width of the screen capture
 * @param height of the screen capture
 * @param args
 */
public static void main(String[] args) {
     if (args.length == 2) {
         width = Integer.parseInt(args[0]);
         height = Integer.parseInt(args[1]);
     }

    ScreenShot s = new ScreenShot();
    BufferedReader in = new BufferedReader(new InputStreamReader(System.in));
    String text = "";
    String[] tokens;

    while (true) {
        try {
            text = in.readLine();
            if (text.equals("take")) {
                s.capturePortion(x, y, width, height);
            } else if (text.equals("terminate")) {
                return;
            } else if (text.length() > 0) {
                tokens = text.split("\\|");
                if (tokens.length < 4)
                    continue;
                x = Integer.parseInt(tokens[0]);
                y = Integer.parseInt(tokens[1]);
                width = Integer.parseInt(tokens[2]);
                height = Integer.parseInt(tokens[3]);
            }
        } catch (IOException e) {
            System.err.println("Exception while reading the input. " + e);
        }
    }
}

/**
 * Capture a portion of the screen
 */
public void capturePortion(int x, int y, int w, int h) {
    try {
        if (robot == null)
            robot = new Robot();
        BufferedImage img = robot.createScreenCapture(new Rectangle(x, y, w, h));
        ByteArrayOutputStream output = new ByteArrayOutputStream();
        ImageIO.write(img, imageType, output);

        DataOutputStream dataOutputStream = new DataOutputStream(System.out);
        //output the buffer size
        dataOutputStream.writeInt(output.size());
        //output the buffer
        dataOutputStream.write(output.toByteArray());
        dataOutputStream.flush();

        output.close();
    } catch (AWTException e) {
        System.err.println("Exception while capturing screen. " + e);
    } catch (IOException e) {
        System.err.println("Exception while writting the image bytes. " + e);
    }
}

I am by no means a designer. Still, I think I managed to get a decent look of the main application window using Adobe Illustrator and Flash Catalyst. I created the design in Illustrator, and then using Flash Catalyst I transformed the graphics into a Flex application. And finally using Flash Builder 4 I added the logic.

Source code and native installers

You can download the Flex project from here, Mac installer from here, and Windows executable from here. This program requires Java 5 or newer and the Adobe AIR 2 runtime.

Things to know when working with Native Processes in AIR

In order to enable this feature you need to add the extendedDesktop profile to the application descriptor file. Add this tag as a child of the application tag:

   1: <supportedProfiles>extendedDesktop</supportedProfiles>

When using this feature you can’t package your application as an AIR file for distribution. You have to use the native installer. The easiest way to do this is to export for release from Flash Builder (you get the AIR file you normally use to distribute your application). And then you use adt at the command line to create the native installer. If you want a Mac installer you do it on a Mac, if you want a Windows installer you have to do it on a Windows. The command looks like this:

   1: adt -package -target native myApp.exe myApp.air

More on how to create native installers for AIR applications here (make sure you use the adt from AIR 2 and not one from an older version).

If you see an error like in the picture below when you install an application using the generated native installer, you should create a file named .airappinstall.log in your home folder. This log file can tell you what was wrong. In my case the error was “failed while validating native package: Error: Missing digested package file: .DS_Store starting cleanup of temporary files” (I fixed the problem by deleting the .DS_Store file from the source folder).

Finally, you can check at runtime if the application has extended desktop capabilities by using this:

if (NativeProcess.isSupported)
    //extended desktop profile is available
else
    //extended desktop profile is not available

What’s next?

If you haven’t already, download the Adobe AIR 2 runtime and SDK and play with the new features. You can find a nice article about the new features from AIR 2 on Christian Cantrell’s blog.

I already have another idea: what about an AIR application that does screen sharing? Keep an eye out I might be able to pull it off!

PS. Many thanks to my friends Chicu and Raul from the Romanian AIR team for their help.

Later Update: My friend Benjamin Dobler created a nice screen recording application with AIR 2 (it captures the sound as well). Although for now the source code is not available, I still think it is worth having a look.

November 30, 2009 | Filed Under AIR, Flex, Tutorials, Video | 20 Comments

Flex Builder 3 Alpha for Linux is out

If you use Linux then I have good news for you: we just released (November 24th 2009) a new version of the Flex Builder 3 plug-in for Linux. You can grab the bits from here.

There are no new features added to this update. However this version was tested on the latest Linux distros and the hardcoded expiration time has been extended beyond December 1st (this is the timeout for the previous release).

Click here to see the release notes.

November 25, 2009 | Filed Under Others | 4 Comments

Acrobat.com reloaded

We just launched a new version of Acrobat.com our online solution for office applications. There is a text editor (Buzzword), a spreadsheet editor (Tables), a presentation tool (Presentation), and a web conference tool (Meetings with screen sharing, voice and video). And because this is hosted in the Cloud, you can uset it wherever you have a web browser and Flash Player, you can upload files to your account and share them with other users, you can work on the same document at the same time (text documents, spreadsheets), and you can create PDF files from the documents you create using this suite, or from documents from your computer.

Acrobat.com is offered as a free product and premium one. Some of the differences between them are related to how many people can connect to your meeting room or how many PDF files you can create. But believe me, even the free version offers a lot of functionality. I’ve been using this suite for the past year and I’m quite happy.

There is an AIR application that enables “dragging and dropping files and folders from your local computer directly into your Acrobat.com account, and browsing, previewing, sharing or publishing files easily from your computer”.

If you have a BlackBerry or an iPhone you can try the Acrobat.com mobile application. Using this app you can view or fax documents from your account, and you can upload document images from your phone and have them automatically stored as a PDF (OCR is performed).

And finally if you run on Windows, you can install the Acrobat.com Outlook add-in. When you want to send a document to others, you can have the document uploaded to your Acrobat.com account instead of loading your mail server with this task. Or you can invite people to your meeting room right from Outlook.

Try it and don’t forget to drop a comment!

November 22, 2009 | Filed Under Others | 1 Comment

Memory profiler for AIR apps created with HTML/CSS/JS

If you create AIR applications using HTML/CSS/JavaScript then you should have a look at the memory profiler feature from Aptana Studio for AIR 2. Here is a screenshot of the memory profiler:

You can see a screencast over here and you can download Aptana Studio from here and try this feature for yourself.

November 18, 2009 | Filed Under AIR | 2 Comments

Transylvania Flex Group hosts a new RIA event

On November 21st in Cluj, Romania there will be a new RIA event hosted by the local Flex User Group. The name of the event is Flex vs Silverlight vs AJAX. My friend who organizes the PHP Geek meetings from Cluj, Mihai Brehar will be the moderator.

The event starts at 10:30 AM and ends at 2.20PM. Although the event is free, you have to register. The event’s location is Betfair Romania, 69-71 Brancusi street, Cluj-Napoca.

November 18, 2009 | Filed Under Conferences | 2 Comments

Flash Player 10.1 and AIR 2 first public betas

We just released the first public betas for the two client runtimes: Flash Player 10.1 and Adobe AIR 2. You can grab them from here and here.

The cool thing about this release is that there are versions available for all three operating systems Win, Mac OS, and Linux (openSUSE, Fedora, and Ubuntu) right from the beginning.

For now Flash Player 10.1 is available only for x86-based computers/netbooks. Later on we will have other releases with support for mobile.

Here you can see public demos and interviews for Flash Player 10.1. You can learn more about features like hardware video decoding, graphics acceleration, instance management, multi-touch support, or microphone access.

You can install sample applications for AIR 2 from here. Here is a list with the new AIR 2 features:

Support for the detection of mass storage devices.
Advanced networking capabilities like secure sockets, UDP support, and the ability to listen on sockets.
Support for native code integration.
The ability to open a file with its default application.
Multi-touch and gesture support.
New APIs for access to raw microphone data.
Webkit update with HTML5/CSS3 support.
Global error handling.
Improved cross-platform printing
Improved security and support for enterprise and government standards.

You can read more about this here and here.

If you have feedback, please don’t be shy!

November 17, 2009 | Filed Under AIR | 12 Comments

Cool 3D Game created in Flash made in Japan

My fellow Japanese evangelist, Teiichi Ota, showed us a cool Flash 3D game made in Japan. You can actually try it for yourself despite having the instructions in Japanese. Here is how to play the game (click here to open the game):

1) You get presented with 10 questions. You’re supposed to click on one of the three circles. Just click them randomly.
2) After the 10 questions you get to name your robot using 3 letters. This is easy.
3) Now you have a robot, with all the custom parameters automatically set.
4) You are presented with 3 buttons:

[Random Match] [Search for Opponent from the Ranking List]
[Challenge the Boss (very strong)]

5) Click whatever button you want.
6) The fight starts. You get to click 3 buttons each once in the battle. Each button represents a special move or weapon.

The image quality is excellent, the design is very clean, and I have to say I’m really impressed by this game. Behind this application, there are IMG SRC/Non-Grid, Kaibutsu, and Masayuki Kido (he’s the one who created the 3D library used by this game).

Not so long ago, Mariko Nishimura (Field Marketing Manager with Adobe Japan) posted a series of articles (Community Power at Adobe MAX and What’s going on now in the Japanese Flash world) that shed light on the Japanese Flash developers. I think their work and especially the quality of their projects may be a surprise for many of us.

November 16, 2009 | Filed Under Others | Leave a Comment

Adobe and RIM collaboration

Today RIM (an Open Screen Project partner) made some interesting announcements for Flash developers at the annual Blackberry Developer Conference. With the upcoming version of Creative Suite there will be additional support for the Blackberry platform.

At the same time we announced together with RIM that we will be optimizing the Adobe AIR runtime and the Flex Mobile Framework for the Blackberry platform. This actually makes RIM the first OEM to announce support for these technologies.

You can find additional information on Mark Doherty‘s blog and Ryan Stewart’s blog.

Isn’t this great news for the Flash community?

November 10, 2009 | Filed Under Others | Leave a Comment

PHP and Flex Webinars

Last week I visited Zend headquarters, and I had an interesting talk over there. One effect of this meeting is this: we will start to do webinars together with Zend.

If you want to find more about Zend Studio and other products related to PHP from Zend, or learn more about the integration between the Flash Platform and PHP (Flash Catalyst, Flash Builder, Flex framework) you shouldn’t miss this opportunity. I know that webinars don’t offer the same experience as in-person events. On the other hand, you can attend them from the comfort of your own home, there is no traveling involved and no need to convince your boss to let you attend the event.

We haven’t set the first event date yet, but it should be in the first week of December, and probably it will be in the evening (Central European Time).

Keep an eye on my blog for the exact date and time.

November 9, 2009 | Filed Under AIR, Conferences, Flex, PHP | 1 Comment