How To Automate Desktop Application Using Java

By Vitthal Kavitake

Abstruse

Automating Coffee Applications is a very vast domain as i solution does not fit all. This paper tries to consolidate all the learning in the process of coming up with an API that can be hands used to automate whatsoever Java application. The paper follows a Problem-Solution approach where at each phase of evolution, hurdles are identified and the best solution is selected from the options available.

What can We Achieve Using These Automation Techniques?

Java Automation technique deals with code that can be used to control the execution of specific UI operations in Java. This is helpful in the following scenarios:

1. To exam Coffee Thick Client Applications where repetitive testing tin can be automated (UI automation testing).

2. To obtain specific information by performing repetitive operations (complex or simple) on the UI.

Both of in a higher place scenarios are doable in but a unmarried click or by running a single program. All the operations on the application get performed automatically and the user gets the end result.

So at that place are basically two usages of this technique.

To automate UI testing
To perform the same sequence of operation with unlike data

At that place are multiple approaches that can be used to achieve this. Each approach comes with its own pros and cons. Each approach is discussed in detail in the next section.

Approaches for Automating

The following approaches are available for automating a Java Awarding:

Using robot class to automate
Using reflection to launch the awarding
Injecting code into another application

We will discuss each of these approaches in detail.

i.Using Robot Class (Imperfect Approach)

Robot is i the class provided by Java to simulate mouse click / keyboard events. We tin use this course to automate not only Coffee but whatever application.

However, with Robot, outcome arises while providing the location coordinates. Robot basically simulates a mouse click at the electric current position of the mouse. Hence difficult coding the mouse location is not a good thought as the location depends on whether the application is opened to full screen. If not total screen, then what is the x, y position of the top-left end of the application? Moving the application a little on the screen changes all the x, y positions of all components. Also if screen resolution is changed and so also x, y positions of each component volition be changed. Another disadvantage is that users will have to keep the application on top and await until the automation finishes. This blocks valuable user fourth dimension.

Here are some pros/cons of this approach in brief:

Automation Possible Scenarios

Automating applications volition be possible by using this approach if nosotros consider the following scenario.

Screen resolution is same across all desktops
Each awarding opens with fixed width and meridian or awarding opens in maximized state.
If application is non opening in maximized state then (x,y) co-ordinate for the specific application remains the same for each run.
User should not perform any other operation while automation is running.

Bold the to a higher place conditions are met, we tin automate non merely java, but any technology awarding. This technique is solely dependent upon where the application is on the screen. So any application in any engineering science is possible to automate.

Advantages

Perform Mouse click at specified location.
Dispatch keyboard events (information technology is assumed that focus is already present at required field.)

Disadvantages

When the application is moved in the screen then this method cannot be used
User needs to await till the functioning is completed
If something pops up while automation is running then automation will fail
If screen resolution is changed then this approach will fail
Background processing of the automation is not possible
Retrieving values from whatever field is not possible

And then simulating clicks on the UI is not a expert arroyo and we volition accept to get inside Java to get more accessibility and admission bodily objects to perform the required operation.

Here the question arises as to how we tin admission the objects of a third party application. The adjacent two approaches explain how we tin can go most it.

2.Using Reflection (Imperfect Approach)

The second arroyo manipulates how Java applications become launched using the JVM.

This approach uses the Reflection API. In this approach, instead of JVM launching the application, it is routed through the custom launcher. Custom launcher reads the principal jar file and as well loads all dependency jar files. Then it loads the entry class (course with primary method) and executes main method from the class.

As custom launcher has launched the application using Reflection, it has control over the JVM on which the application is running and and then information technology can admission any of the UI component objects from the awarding. Once the object from the UI is attainable, it is easy to perform any required functioning on the object. These operations can include setting text, getting text, dispatching events, etc.

Though at this stage Java Reflection looks similar a favorable arroyo, further investigation reveals issues while automating applet applications. Specifically in scenarios where an applet takes dynamic parameters from the web page or where the applet is launched using complex logic on a web page, information technology is non feasible to utilise this arroyo. Also the Coffee applications, which are wrapped as exe, are also non possible to load. For thick client applications where launching java applications has complex steps, this approach cannot be used or information technology may require more complex coding to be done.

Automation Possible Scenarios

Thick client applications, which tin can exist run using a simple command java –jar UIApplication.jar , tin be automated easily. If the command has more properties set in social club to run the application with –D options then it will be hard to automate, or it volition need more effort to make information technology work.

Applet applications with simple tag <Applet grade="my.applet.instance.ClassName" width="500" height=="300" /> tin be automated. If the applet tag is generated using JSP or if it is retrieved from the application server after few steps so it becomes difficult. This approach goes to discarding country when a user wants to automate an applet, which takes input as parameter similar token from an HTML or JSP page.

The same is true in the case of JNLP. Applications with a simple static JNLP file tin be automatic, simply difficulties start when JNLP files are generated dynamically.

Advantages

Perform Mouse click for specified object
Dispatch keyboard events for specified objects
Retrieving value from specified objects
Does not depend on screen resolution/position of application on screen
Background processing is possible
As this runs in groundwork, user tin simultaneously work on other things

Disadvantages

All Applets cannot exist automated using this approach
Dynamically generated JNLP files cannot be automated
Thick customer apps with complex launching logic will exist hard to automate
Thick client apps, which use batch file to launch and having circuitous launch logic cannot exist automated
Jar files wrapped equally exe cannot exist automatic
It is required to give extra permission for socket connection and listening to AWT events and depending on the app may require giving all permission as well

3. Injecting Code into Running JVM (Best Suited Approach)

Java provides 2 approaches to inject code into running JVM. These are the concepts of Java instrumentation and attaching Instrumentation Agent.

We volition discuss these concepts in cursory here.

Coffee provides a form chosen java.lang.instrument.Instrumentation , which can be used to inject code in running JVM. Code can be injected in 2 ways.

3.1. Using Agentmain Method

For more detail nearly using this method, refer to Coffee Instrumentation from the Oracle docs. This type of injection injects the code into running JVM. Hither the assumption is that the tertiary party application is already upward and running. Code tin can be injected into the running application by creating a Java amanuensis jar file with the agentmain method. Once the code is injected, the agentmain method gets called. Automation related code can be executed considering the agentmain method as the entry point. At present the lawmaking written in agentmain and the tertiary party application both reside in the same JVM. So automation code can directly access objects from the running third political party application. More than detail almost accessing objects from a 3rd party awarding and performing operations on it are explained in after sections.

Yet there are some restrictions to using this method e.thousand., not all JVMs support this blazon of code injection. Also we crave the JDK to be installed in club to employ this type of code injection.

3.ii. Using Premain Method

For more than detail most using this method, read Java Instrumentation from the Oracle docs. This blazon of injection injects the code while starting JVM. Here, the Java agent runs fifty-fifty before third party awarding. As the method suggests, the premain method gets called even earlier the main method of the 3rd political party application. Here, the same as agentmain , Automation related code is executed from the premain method. These two run in the same JVM so Automation code can get ahold of UI objects easily. More detail about accessing objects from third party applications and performing operations on it are explained in later sections.

Advantages/Automation Possible Scenarios

All kinds of Java applications are possible to automate using this arroyo. The only pre-requisite is that before launching the app, surroundings variable JAVA_TOOL_OPTIONS is required to set with value as –javaagent:"<PATH TO JAVAAGENT>" . The variable can be gear up equally process level, user level, or system level depending upon the requirement.

Advantages

Perform Mouse click for specified object
Dispatch keyboard events for specified objects
Retrieving value from specified objects
Does not depend on screen resolution/position of application on screen
Background processing is possible
Every bit this runs in background, user can simultaneously work on other things
All kind of desktop Swing/AWT applications tin can exist automated
All JNLP applications can exist automated
All applet applications can be automated
Java applications wrapped as exe can be automatic
Dynamically generating JNLP applications can exist automated
Two or more applets embedded in one html can be automated

Disadvantages

It is required to give extra permission for socket connection and listening to AWT events and depending on the app may require giving all permission likewise.

Using Pre-Defined Environment Variables

This is required in the case of using the premain method. This section talks almost attaching the Automation Amanuensis we accept created to whatever Coffee Application.

Nosotros will accept to add Coffee Agent along with application. To do this, we demand to pass the JVM parameter '-javaagent:<Coffee agent path>'

If you are running the Java application using Java control, then it is simple, just add these JVM parameters along with what you have already.

e.1000.) if you are running the Java program like this: coffee –jar MyApp.jar then it will be: java –javaagent:"C:AutomationAgent.jar" –jar MyApp.jar

If you are running the Java program like this: java –cp someclasspath com.MyClass then information technology volition exist: java – javaagent:"C:AutomationAgent.jar" –cp someclasspath com.MyClass

How do we set this for the applets that run in a browser?

Set up environs variable, _JPI_VM_OPTIONS=-javaagent:<java agent path>

It'south the aforementioned in the case of a JNLP, prepare environs variable,

JAVA_VM_ARGS=-javaagent:<java amanuensis path>

All the same these environment variables practise not work in a few scenarios, similar if JNLP itself has the JVM argument set, and then this will be ignored. If the browser doesn't create different java.exe process for applets, then this will not work.

The alternative for this is the following:

Set, JAVA_TOOL_OPTIONS=-javaagent:<java agent path>

This works for whatever kind of Java application, be information technology Applet or JNLP or running through any command.

Accessing Objects of Third Party Applications

Every bit nosotros are dealing with UI applications only, nosotros tin can use Abstract Window Toolkit provided past Java to achieve this. From the premain method, we can add an event listener, which volition receive window opened and window closed events.

Toolkit.getDefaultToolkit().addAWTEventListener(new AWTEventListener(){ … }, AWTEvent.WINDOW_EVENT_MASK/*For listening to window events*/);

By adding to a higher place snippet of code, we tin mind to all windows that are being opened. And from the event nosotros tin get the target component. Events include the pinnacle level window object. In one case nosotros go the top level window object, nosotros tin can iterate top level and all its children to notice child components and so on. So in this way we tin can get all the UI objects. After getting the handle for all the objects, nosotros tin perform whatsoever operation. East.yard. we can dispatch whatever event that could simulate the deportment we desire to perform.

As we demand to perform some operation, it is time specific or provisional operations. We will have to pattern few APIs, which can exist executed in society to achieve the functionality.

Exposing API to Perform Operations

Consider a simple scenario, we desire to set text to a text field and then click on a button.

So here we need two APIs, one to set text to text field and the other to click on the button. Nosotros will betrayal below two APIs to perform the operations.

setText(JTextField textField,String text)

buttonClick(JButton push)

These functions now need to exist called from a different JVM. How do we achieve this Inter-JVM call?

The most popular approach is using the RPC (Remote Procedure Call) server-client machinery. Another benefit of using this mechanism is that the client tin be on any platform like Java, .Net, C, etc.

XML-RPC Role

XML-RPC is used to phone call the API from a different JVM or unlike platform program. The application that is required to automate will deed as an XMLRPC server and the plan, which calls the API exposed for automation will act as the XMLRPC client and phone call the XMLRPC server APIs.

A challenge while using XMLRPC is which port should we use to run the XMLRPC server. The port cannot be fixed every bit this may lead to a conflict later on in the application when some other functionality may attempt to utilize the same port. Also, if iii to four applications are running simultaneously and then the same port cannot be used for each awarding.

The approach used here is to assign the gratuitous port numbers randomly to the application.

So the next question is while assigning port numbers randomly, how will the client come to know which port to ship the request to?

To solve this problem, we have come up up with the concept of a MappingServer .

Mapping Server

A MappingServer, as the proper noun suggests, is one of the XMLRPC servers. When the awarding runs and assigns the port numbers randomly, information technology gives a call to MappingServer with port number and PID details. The MappingServer then stores these details and whenever the client wants to connect; they volition request the MappingServer to get the port provided a particular PID. Then this way, the challenge of identifying which port the application is running on is solved.

Block Diagram

The block diagram below helps us understand the flow and working of the each of the components.

Diagram

Footstep ane and 2 are simultaneous steps every bit launching and code injection happens at the same time.

Component Identification Mechanism

The adjacent trouble that we confront is regarding the API nosotros have exposed for accessing the component i.eastward.

'public void setText(JTextField textField)'

This API cannot be directly called, as this cannot identify the textfield in the different JVM.

The challenge is to identify the right component. Coffee doesn't provide any unique mechanism to place components. If you are aware of hashCode and so maybe you lot're thinking that we can employ hashCode. Merely nosotros cannot, as they volition be unique for each component, simply will differ for every run for the same component.

Next we evaluate ii approaches to uniquely identify a component.

The first approach is assigning numbers in BFS (Latitude Offset Search) order to each component. Then each time the application runs, the same club will be followed. But this fails when there is a dynamic addition of components on the UI.

The second approach is to identify components using searchPath . The searchPath is defined right from the parent till the component. e.thousand.) suppose JFrame has JPanel and JPanel has two children – JtextField and JButton. Then searchPath for JTextField will be [(frame)JFrame].[(panel0)JPanel].[(myText)JTextField] and for the push information technology volition exist [(frame)JFrame].[(panel0)JPanel].[(submit)JButton].

If there are two buttons or identical components in the aforementioned container and then at that place are chances that both components searchPath will become the same. In this case, once again we requite indexes to place these components uniquely. And then for the kickoff button the id would exist [(frame)JFrame].[(panel0)JPanel].[(null)JButton]{0} and for second push the id would be [(frame)JFrame].[(panel0)JPanel].[(naught)JButton]{1}.

This is just an illustrative example. And then, in this fashion we can identify the component uniquely.

Thus we change the signature of the APIs exposed. The new signatures will be,

setText(Cord searchPath,String text)

buttonClick(String searchPath)

Nosotros will have to start become the object co-ordinate to the search path and then perform the required operation.

Conclusion

Thus, in conclusion, nosotros take walked through the procedure of automating any kind of Coffee applications and in the process have identified and overcome the various hurdles faced in the path. The paper illustrates the best approach to selection in order to automate any kind of Java application. The paper talks in brief about the approaches that are not suitable as well. In the walkthrough, nosotros accept concluded that Injecting lawmaking into a third political party application using java instrumentation is the best suited approach for automating Java applications.

References

http://docs.oracle.com/javase/6/docs/api/java/lang/instrument/parcel-summary.html

http://docs.oracle.com/javase/tutorial/reflect/

This article was written by Vitthal Kavitake. The information in this presentation is accurate as of its publication engagement; such information is subject to change without notice. The author acknowledges the proprietary rights of other companies to the trademarks, product names and such other intellectual property rights mentioned in this document. Except as expressly permitted, neither this presentation nor any part of it may be reproduced, stored in a retrieval system, or transmitted in any form or past any means, electronic, mechanical, printing, photocopying, recording or otherwise, without the prior permission and/or any named intellectual property rights holders under this presentation.