2009-10-24

Troubleshooting two common JMS exceptions

I've been working with a particular architecture lately.

It's composed of an application server (WebLogic Server 10), hosting webservices, an ESB (ALSB 2.5) and an EAI / BPM (WLI 8.1 SP5).

The point we are going to focus on is the JMS communication between WLI 8.1 SP5 & WLS 10.

WLI is hosting a JMS server and WLS 10, is hosting a foreign JMS Server, representing the WLI JMS Server.

First exception

From time to time, we happened to have this exception showing up in the logs :

<23 oct. 2009 21 h 50 CEST> <Error> <JTA> <BEA-114089> <User [<anonymous>] is not authorized to invoke ackPrepare on a Coordinator.>
<23 oct. 2009 21 h 54 CEST> <Error> <JTA> <BEA-110495> <User [<anonymous>] is not authorized to invoke AckRollback on a Coordinator.>

These particular errors were happening when WLS was sending a JMS message towards WLI

(which was pending since the transaction wasn't commited nor rollbacked yet) and then tried to commit the transaction.

Then we saw the first message (BEA-114089). WLS, seeing that it couldn't commit, tried to rollback but it still wasn't allowed to do such a thing.

If we take a look at the official documentation :

BEA-114089
Error: User [principal] is not authorized to invoke ackPrepare on a Coordinator.
Description
The user with identity [principal] should not be attempting to directly invoke internal methods on the Coordinator object. This might subvert transactional integrity.
Permission was denied, and the attempt was ignored.
Cause
This might be a symptom of a potential security attack, or alternately, a problem with the "system" security identity of the server.
Action
Verify that the client software is not directly invoking an internal WebLogic system object.

And

BEA-110495
Error: User [principal] is not authorized to invoke AckRollback on a Coordinator.
Description
The user with identity [principal] should not be attempting to directly invoke internal methods on the Coordinator object. This might subvert transactional integrity.
Permission was denied, and the attempt was ignored.
Cause
This might be a symptom of a potential security attack, or alternately, a problem with the "system" security identity of the server.
Action
Verify that client software is not directly invoking an internal WebLogic system object.

We see that at some point, these explanations are not very useful.

I then made a sandbox to reproduce this problem.

I played with cross-domain security and security interoperability mode.

Before playing with these two options, I strongly recommend you to read this official documentation :

Finally, I came to the conclusion that these errors only happened when I had cross-domain security enabled (with identical credentials on both domains)

and that the security interoperabilty mode was set to "Performance" on WLS 10. (by default, this is the mode defined on WLS 8.1).

In spite of the fact that official documentation says "Every participating server must set the Security Interoperability Mode parameter to the same value",

in my case, it worked with Performance on WLS 8.1 (WLI) and Compatibility on WLS 10 ...

Second Exception

During my tests I also encountered an annoying error :

<WSEE:13>
Error invoking fr.edf.test.transaction.SampleWebService (POJO): weblogic.jms.common.JMSException<ComponentHandler.handleRequest:115>
<WSEE:13>weblogic.jms.common.JMSException: Connection not found<ComponentHandler.handleRequest:115>
weblogic.jms.common.JMSException: Connection not found
        at weblogic.jms.dispatcher.DispatcherAdapter.convertToJMSExceptionAndThrow(DispatcherAdapter.java:110)
        at weblogic.jms.dispatcher.DispatcherAdapter.dispatchSync(DispatcherAdapter.java:45)
        at weblogic.jms.client.JMSConnection.setupJMSSession(JMSConnection.java:499)
        at weblogic.jms.client.JMSConnection.createSessionInternal(JMSConnection.java:467)
        at weblogic.jms.client.JMSConnection.createQueueSession(JMSConnection.java:432)

I had to google some time to figure out that this error came from the fact that both my admin servers were named "admin".

Seen in the official doc :

Please note the following limitations for inter-domain transactions:

The domains and all participating resources must have unique names. That is, you cannot have a JDBC data source, a server, or a domain with the same name as an object in another domain or the domain itself.

Well, that's it for now. I'm sure there are lots of potential other errors, but I hope that, at least, these two explanations may help some fellows out.

Mots clés Technorati : jms,jta,transaction,wls 8.1,wls 10,BEA-110495,BEA-114089,JMSException,Security Interoperability,cross-domain security

2009-09-20

How to suspend / resume WLI 8.1 EventGenerators programmatically ?

You sometimes may have to programmatically pause your EventGenerator (EG) consumption.

And this, for various reasons, such as pausing all processes while one specific is running, or coding your own administration application.

BEA | Oracle doesn't come with a simple way of doing so. And the reason is obvious : theoretically, you don't have to manipulate those objects,

through another way as the WLI console.

However, this post will help you to do so.

First way : JMX

Since JMX wasn't officially finished while WLI 8.1 was released, BEA chose to implement its own version.

I quote : "Prior to 9.0, WebLogic Server used its own JMX implementation based on the JMX 1.0 specification."

As a result, it's not fun to juggle between the official JMX and the BEA implementation.

Second way : Using the official but not-normally-callable API

As you may have guessed, we are going to use the WLI console functions since those suspend & resume functions are already used there.

If you take a look at your WLS 8.1 installation directory, you will find the WLI console WAR in the directory WLS_HOME \ weblogic81 \ integration \ lib \ console.

If you open this WAR, you will see some interesting classes, such as EventGeneratorUtils and so on.

But these are only classes, available in the WEB-INF directory of the WAR, thus no easy way to know the dependencies between them.

The best solution would be to have a JAR.

Well, if you have created a WLI domain, you will find exactly what you need !

Let's assume we have an administration server whose name is "AdminServer" (pretty good name, huh ? :) )

Then check in the extract directory, the directory corresponding to the WLI Console :

DOMAIN_HOME \ AdminServer \ .wlnotdelete \ extract \ AdminServer_WLI Console_wliconsole \ jarfiles

And you should see something like that :

Two directories in which you have a directory called "jarfiles". There's an interesting file "_wl_cls_gen.jar".

It contains all the classes used for the WLI Console, and then all the classes used for managing EG lifecycles.

Just add this Jar & Log4J in your IDE classpath project and you're good to go.

To go straightforward, here is the class I wrote :

import java.util.HashMap;
import java.util.List;
import java.util.Map;
import javax.management.MBeanException;
import org.apache.log4j.Logger;
import com.bea.wli.oam.eventgenerators.EventGeneratorUtils;
import com.bea.wli.oam.eventgenerators.EventGeneratorWrapper;
import com.bea.wli.oam.eventgenerators.file.FileEventGeneratorHelper;
import com.bea.wli.oam.eventgenerators.jms.JmsEventGeneratorHelper;
import com.bea.wli.oam.eventgenerators.timer.TimerEventGeneratorHelper;
/**
* @author mbutton
*
*/
public class EGManager {
    private transient Logger logger = Logger.getLogger(getClass());
    public static final String TIMER_EG = "TIMER";
    public static final String JMS_EG = "JMS";
    public static final String FILE_EG = "FILE";
    private static Map timerEG;
    private static Map jmsEG;
    private static Map fileEG;
    /**
     * Constructor that initialises the maps representing the declared EGs.
     * @throws EGManagerException
     */
    public EGManager() throws EGManagerException {
        try {
            indexTimerEventGenerators();
            indexJMSEventGenerators();
            indexFileEventGenerators();
        } catch (MBeanException e) {
            logger.error("Problème lors de l'indexation des EventGenerators", e);
            throw new EGManagerException("Problème lors de l'initialisation : les EventGenerators n'ont pu être récupérés.");
        }
    }
    /**
     * (non-javadoc) Indexes the TIMER EG family
     *
     * @throws MBeanException
     */
    private void indexTimerEventGenerators() throws MBeanException {
        List timerEGs = TimerEventGeneratorHelper.getAllTimerEventGenerators();
        timerEG = indexEventGenerators(timerEGs);
    }
    /**
     * (non-javadoc) Indexes the FILE EG family
     *
     * @throws MBeanException
     */
    private void indexFileEventGenerators() throws MBeanException {
        List fileEGs = FileEventGeneratorHelper.getAllFileEventGenerators();
        fileEG = indexEventGenerators(fileEGs);
    }
    /**
     * (non-javadoc) Indexes the JMS EG family
     *
     * @throws MBeanException
     */
    private void indexJMSEventGenerators() throws MBeanException {
        List jmsEGs = JmsEventGeneratorHelper.getAllJmsEventGenerators();
        jmsEG = indexEventGenerators(jmsEGs);
    }
    /**
     * (non-javadoc) Indexes a List with the EG name as a key and the
     * EventGeneratorWrapper as the object.
     *
     * @param eventGenerators
     *            the EG list to be indexed
     * @return a Map containing the indexed information.
     */
    private Map indexEventGenerators(List eventGenerators) {
        Map indexedEG = new HashMap();
        int size = (eventGenerators != null) ? eventGenerators.size() : 0;
        EventGeneratorWrapper currentEGWrapper = null;
        for (int i = 0; i < size; i++) {
            currentEGWrapper = (EventGeneratorWrapper) eventGenerators.get(i);
            indexedEG.put(currentEGWrapper.getName(), currentEGWrapper);
        }
        return indexedEG;
    }
    /**
     * Method used to suspend a running EventGenerator
     *
     * @param eventGeneratorType
     *            the type (File, JMS, ...) of the EG
     * @param eventGeneratorName
     *            the EG name as it appears through the WLI console
     * @throws EGManagerException
     */
    public void suspendEventGenerator(String eventGeneratorType, String eventGeneratorName) throws EGManagerException {
        checkEventGeneratorType(eventGeneratorType);
        EventGeneratorWrapper eg = getEventGeneratorWrapper(eventGeneratorType, eventGeneratorName);
        if (eg != null) {
            try {
                EventGeneratorUtils.suspendGenerator(eg);
            } catch (Exception e) {
                logger.error("Impossible d'arrêter l'EventGenerator " + eventGeneratorName, e);
                throw new EGManagerException("Impossible d'arrêter l'EventGenerator " + eventGeneratorName);
            }
        } else {
            throw new EGManagerException("L'event Generator de type " + eventGeneratorType + " et de nom " + eventGeneratorName + " n'a pas été trouvé.");
        }
    }
    /**
     * Method used to resume a suspended EventGenerator
     *
     * @param eventGeneratorType
     *            the type (File, JMS, ...) of the EG
     * @param eventGeneratorName
     *            the EG name as it appears through the WLI console
     * @throws EGManagerException
     */
    public void resumeEventGenerator(String eventGeneratorType, String eventGeneratorName) throws EGManagerException {
        checkEventGeneratorType(eventGeneratorType);
        EventGeneratorWrapper eg = getEventGeneratorWrapper(eventGeneratorType, eventGeneratorName);
        if (eg != null) {
            try {
                EventGeneratorUtils.resumeGenerator(eg);
            } catch (Exception e) {
                logger.error("Impossible de relancer l'EventGenerator " + eventGeneratorName, e);
                throw new EGManagerException("Impossible de relancer l'EventGenerator " + eventGeneratorName);
            }
        } else {
            throw new EGManagerException("L'event Generator de type " + eventGeneratorType + " et de nom " + eventGeneratorName + " n'a pas été trouvé.");
        }
    }
    /**
     * (non-javadoc) From the indexed EG list, it returns the
     * EventGeneratorWrapper matching the EventGenerator looked for.
     *
     * @param eventGeneratorType
     *            the type (File, JMS, ...) of the EG
     * @param eventGeneratorName
     *            the EG name as it appears through the WLI console
     * @return the matching EventGeneratorWrapper
     */
    private EventGeneratorWrapper getEventGeneratorWrapper(String eventGeneratorType, String eventGeneratorName) {
        EventGeneratorWrapper result = null;
        if (TIMER_EG.equals(eventGeneratorType)) {
            result = (EventGeneratorWrapper) timerEG.get(eventGeneratorName);
        } else if (JMS_EG.equals(eventGeneratorType)) {
            result = (EventGeneratorWrapper) jmsEG.get(eventGeneratorName);
        } else if (FILE_EG.equals(eventGeneratorType)) {
            result = (EventGeneratorWrapper) fileEG.get(eventGeneratorName);
        }
        return result;
    }
    /**
     * (non-javadoc) Method used to check that the given EventGenerator type is
     * correct.
     *
     * @param eventGeneratorType
     *            the type (File, JMS, ...) of the EG
     */
    private void checkEventGeneratorType(String eventGeneratorType) {
        if (eventGeneratorType == null
                || eventGeneratorType.equals("")
                || (!eventGeneratorType.equalsIgnoreCase(TIMER_EG) && !eventGeneratorType.equalsIgnoreCase(JMS_EG) && !eventGeneratorType
                        .equalsIgnoreCase(FILE_EG))) {
            throw new IllegalArgumentException("Le type de l'EventGenerator '" + eventGeneratorType + "' n'est pas correct.");
        }
    }
    /**
     * @author mbutton
     * Exception used to raise an error while manipulating EventGeneratorWrappers to the front layer.
     */
    public class EGManagerException extends Exception {
        private static final long serialVersionUID = 1L;
        /**
         * Constructor
         * @param message
         */
        public EGManagerException(String message) {
            super(message);
        }
    }
    /**
     * @param args
     */
    public static void main(String[] args) {
        EGManager egm;
        try {
            egm = new EGManager();
            egm.suspendEventGenerator(EGManager.TIMER_EG, "TimerSampleEG");
        } catch (EGManagerException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
    }
}

There you go !

To sum up, you have two methods on the EGManager : suspendEventGenerator & resumeEventGenerator.

Both take a EventGenerator type as the first parameter and the name of the EventGenerator as a second argument.

This class was designed only for Timer EG, JMS EG & File EG. But you may add other types and translate the French messages :)

But there is one limitation to that implementation. Actually, those objects are sensed to be called during a proper session via the WLI Console.

It means you have to be logged on.

In my case, we were creating another console, then I just had to secure my WebApp and everything went ok after that.

To do so, let's check my other post "Reminder on how to quickly secure a web app on WebLogic Server".

Update : I also had to call these methods from WLI processes. The solution I chose was to add the following annotation to each process that had to manipulate EG.

* @common:security run-as="weblogic"

Of course, "weblogic" has to be a valid principal.

For instance, in a real-world example :

/**
* @jpd:process process::
* <process name="PlayWithEventGenerators">
* <clientRequest name="Client Request" method="clientRequest"/>
* <perform name="Do The Business" method="perform"/>
* </process>::
* @common:security run-as="weblogic"
*/
public class PlayWithEventGenerators implements com.bea.jpd.ProcessDefinition
{ [...] }

Mots clés Technorati : WLI,EventGenerator,Suspend,Resume,Programmatically,8.1,WebLogic Integration,WebLogic

Reminder on how to quickly secure a web application on WebLogic Server

This post is not about a highly technical subject, but I guess it's nice to have it explained in a few words, for beginners,

and have the real stuff at a glance for experimented users who just want a reminder.

Actually, note the security has nothing to do with your code !

All this aspect relies on configuration + the login page and the error page (JSPs).

Here's what you have to do.

Assume you have a webapp called "MyWebApp".

In the WEB-INF directory, you'll have a web.xml and, if you want to deploy it on WebLogic server, another XML file : weblogic.xml.

Note : Here's presented the authentication through a html form.

In blue, the important stuff.

Web.xml :

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE web-app PUBLIC "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN" "http://java.sun.com/dtd/web-app_2_3.dtd">
<web-app id="WebApp_ID">
    <display-name>MyWebApp</display-name>
    <welcome-file-list>
        <welcome-file>index.jsp</welcome-file>
    </welcome-file-list>
    <security-constraint>
        <web-resource-collection>
            <web-resource-name>All</web-resource-name>
            <url-pattern>/*</url-pattern>
            <http-method>GET</http-method>
            <http-method>POST</http-method>
        </web-resource-collection>
        <auth-constraint>
            <role-name>webuser</role-name>
        </auth-constraint>
    </security-constraint>
    <login-config>
        <auth-method>FORM</auth-method>
        <form-login-config>
            <form-login-page>/login.jsp</form-login-page>
            <form-error-page>/login_failed.jsp</form-error-page>
        </form-login-config>
    </login-config>
    <security-role>
        <role-name>webuser</role-name>
    </security-role>
</web-app>

Weblogic.xml :

<?xml version="1.0" encoding="UTF-8"?>
<weblogic-web-app xmlns="http://www.bea.com/ns/weblogic/90"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.bea.com/ns/weblogic/90 http://www.bea.com/ns/weblogic/920/weblogic-web-app.xsd">
    <weblogic-version>10.0</weblogic-version>
    <context-root>TestInterface</context-root>
    <security-role-assignment>
        <role-name>webuser</role-name>
        <principal-name>Administrators</principal-name>
    </security-role-assignment>
    <jsp-descriptor>
        <page-check-seconds>-1</page-check-seconds>
    </jsp-descriptor>
    <container-descriptor>
        <session-monitoring-enabled>true</session-monitoring-enabled>
    </container-descriptor>
</weblogic-web-app>

<html>
<head>
    <title>Security WebApp login page</title>
</head>
<body>
<blockquote>
<h2>Please enter your username and password:</h2>
<p>
<form method="POST" action="j_security_check">
<table border=1>
    <tr>
      <td>Username:</td>
      <td><input type="text" name="j_username"></td>
    </tr>
    <tr>
      <td>Password:</td>
      <td><input type="password" name="j_password"></td>
    </tr>
    <tr>
      <td colspan=2 align=right><input type=submit value="Submit"></td>
    </tr>
</table>
</form>
</blockquote>
</body>
</html>

login_failed.jsp :

<%@ page
    language="java"
    contentType="text/html;charset=UTF-8"
%><%
response.setStatus(200); // To prevent IE from catching the response with its own error page
%>
<html>
    <head>
        <title>Security WebApp login error page</title>
    </head>
    <body bgcolor="#cccccc">
        <blockquote>
            <h2>Vous n'êtes pas autorisé à accéder à l'application.</h2>
        </blockquote>
    </body>
</html>

If you want to know more, take a look at the official documentation.

Mots clés Technorati : secure,weblogic,webapp,web application,j_security_check,role-name,auth-method,url-pattern

2009-04-27

How to create a distributed WebLogic domain ?

Due to a lot of questions about that, I'll blog about the methodology to create

a simple domain, with a cluster distributed on several machines.

Architecture Schema

Usually, a best practice is to keep the admin server away from managed servers.

Because, if a strong load happens and overloads a managed server, it will consume all the resources available

on the machine, including those needed for the admin server to be safe & running.

Then consider we have three machines, one admin server and four managed servers :

Install the binaries

This part is the easiest one. On each machine, you'll have to install the binaries.

I strongly recommend that you use the silent mode, which is faster and requires no manual intervention.

http://e-docs.bea.com/common/docs100/install/silent.html

Another advice : it may sound obvious, but you have to use the same WLS version on every machine ...

Create the domain

You've got two possibilities for a domain creation :

Using the Config Wizard

This tool allows you to simply create a domain, by answering the asked questions, through a GUI or a command line.

It's based on the default WebLogic template, which is almost empty :)

(you may later base your domain on your own template, but that's another topic).

Using WLST

Using WLST, the concept is the same, you will base your domain on the default WLS template and

then, you'll script the instructions to create your domain.

Note : you may also use WLST with WLS 7 & 8

Unfortunately, this tool was hosted on projects Dev2Dev, which is not available anymore.

If you're interested in getting it, I may send it to you through email (about 2 539k).

Populate the domain

When you create a domain, as explained above, it will automatically create an administration server but

nothing else, unless you specify it. You can do so, either during the creation process, by chosing to

customize instead of selecting default, or by creating a light domain, launching the admin server,

and then creating all the wanted resources.

No matter what is the solution you've taken, you'll need to configure at least two kind of resources :

Machines (which only are a logical representation of your physical machines)
Managed Servers, which represent the JVM hosting your business logic.

It is mandatory that you define machines if you're willing to use the NodeManager. Else, even if you

don't want to use it, it's a good practice to define your machines. Will help you, when administrating

your domain, to see where your managed servers are hosted.

Then, once you've done those two steps, you may proceed to optional next steps (configuring datasources ...)

Distribute your domain

Now your domain is full and complete, we need to duplicate it on other machines.

Pack & Unpack commands

These commands were designed to package a domain and unpackage it on another machine.

You could do the same by using a zip or a tar.gz, but using these commands will ensure you that

you have taken only the necessary files for your domain to work (no log files and unnecessary stuff).

http://download.oracle.com/docs/cd/E12840_01/common/docs103/pack/commands.html

Pack / Unpack (WLST version)

You can also do that task by using WLST.

Here is the pack script (taken from the WLST project page on dev2dev) :

# pack.py: convert from pack command to wlst script
# This script shows how to convert form the pack command to a wlst script.
# Note that the domain and template settings must be single-quoted.
# Read in the domain that you want to templatize
readDomain(<domain>)
# Create a template from the domain
writeTemplate(<template>)
closeDomain()

And here comes the unpack script :

# unpack.py: convert from unpack command to wlst script
# This script shows how to convert from the unpack command to a wlst script.
# Note that the domain and template values, and the options to setOption, must be single-quoted
# Specify the template that you want to use
readTemplate('c:\wls9\user_templates\wlst_wls_template.jar')
# If you specified the -username and -password option in the unpack command,
# Specify them here. Otherwise, delete these lines`
# Note that the domain_name field here is just the name of the domain, not the full path as specified in writeDomain below
cd ('/Security/<domain-name>')
create (<user_name>,'User')
cd ('User/<user_name>')
set ('Password',<password>)
# analogous to unpack -java_home
setOption('JavaHome',<java_home>)
# analogous to unpack -server_start_mode
setOption('ServerStartMode',<server_start_mode>)
# analogous to unpack -app_dir
setOption('AppDir',<app_dir>)
# write the domain
writeDomain(<domain>)
closeTemplate()

By the way, I was talking about the fact you can zip your domain and install it elsewhere : that possibility should be considered as an

excellent way of saving snapshots of your domain (for instance, before installing a new resource ...).

If something should go wrong, you would simply have to delete your domain and unzip your backup to start as it was before.

NodeManager (optional)

The NodeManager is a tool that allows you several things, but the most important that come to my mind are :

=> to start servers from the console
=> to automatically restart a failed managed server
=> to migrate pinned services (JTA / JMS)

But some cilents don't want to use it because they prefer to use scripts. If this is your case, well, you're done.

Just start your managed servers on each machine and your cluster is up and running.

Else, read carefully the following lines !

For a domain to be managed by the NodeManager, you will have to enroll it first.

That means that you've got to help the NodeManager to be aware that a new domain was created.

(this step is NOT necessary when you're using the config wizard)

To see if the different nodemanagers are reachable, check the machines.

In case of a problem, you may see :

Of course, ideally, the status should be "Running".

Start your servers

If you chose to use the NodeManager, you only have to go on the administration console, and start your cluster.

Or you may start your managed servers using the start scripts. But be aware that with this solution, since your domain

has been duplicated on each machine, you could start all the servers on each machine. It's up to you to start only the relevant

servers on each machine.

Mots clés Technorati : weblogic domain,cluster,distributed,machine,nodemanager,nmenroll,pack,unpack,wlst

2009-04-24

SOA Suite 11g : first quick look

Working @ Oracle, it gives me some interesting advantages, like the ability to play with the 11th version of

the Oracle SOA Suite, before its release to the market.

Here's a little summary of what I've had the chance to play with.

But, be aware that what I'm exposing right here is NOT representating the final version.

It's only a beta but as far as I can tell, our guys have done a good job !

What's new ?

Well, of course OC4J is gone and WebLogic Server has replaced it.

Now, we've got a new WLS template, reflecting the SOA Suite products :

The installer has been updated (Xtreme Makeover :p) :

And some webpages have been changed as well :

I'm quite a fan of that new graphical aspect.

I unfortunately didn't have enough time to play more with it, so stay tuned for the next episode :)

However, what I think could be improved :

=> JDeveloper 11g still needs another WLS install to be able to launch (ideally, I would have expected a automated

installation with all the products and only one WLS install, but I'm sure it's on the list)

=> Enterprise Manager wasn't happy since the French resource bundle was missing. I would have prefered the English

one to be loaded by default if not found.

=> For now, loading times (when server starts up) are quite long.

=> I used to use my own JVM memory arguments by using the USER_MEM_ARGS variable. Now, it's impossible because,

it's already used as an internal variable. I would have liked another variable like SOA_MEM_ARGS and the possibility

to still have the USER_MEM_ARGS var overriding the other values.

Mots clés Technorati : SOA Suite 11g,first look,jdeveloper11g,weblogic server 11g

Building a high available and scalable architecture with WebLogic Server

For once, I'd like to talk about general points. As fas as I know, I still haven't seen any process

that has been respecting all these simple rules.

Determine the target

Define your capacity planning.

What kind of business are you dealing with ?

Is it fault tolerant ?

Is there money involved ?

Your application

First of all, make sure you know what EAI, ETL, SOA and so on refer to.

Because before you build your house, you'd better know what tools exist & could help you.

Designing your application

Separation of concern (a layer per function)

Use design patterns where applicable

Make your developpers write the documentation when they're coding. Else, you'll get some unmaintainable code.

Be sure that each class has its unit test !

No custom database access, use a connection pool.

Think beyond the scope of your application : are there some business services which could be reused later ?

Choose the technology that fits your needs (webservices are not THE ULTIMATE solution)

For instance, don't use Hibernate if you can't use it or tune it properly. Prefer a custom DAO layer if you feel more comfortable with it.

Generally, don't use a tool / framework just because it's cool or trendy. Take it if you really need it and

understand what it can bring to you.

Dependency injection : why not if you master the framework you're using.

What kind of data is going to be exchanged between your application and 3rd party app ? (size, protocol, frequence)

Do you need to manipulate huge loads of data ?

Make some POCs !!! And compare performance measures !

Tuning your application for production use

Prefer the request scope instead of session : it will help keeping your session light.

Make all the session objects serializable and don't forget to use "transient" objets where applicable.

Your session should not exceed 35kb (for a good replication performance)

Think about packaging. Don't forget that some people are going to manage your application ! Eventually, use deployment plans.

Defining WorkManagers

Even if you don't want to use them for now, define one for each subcomponent (webapp / EJB).

When starting, if WLS doesn't find it, it will use the default : you'll still be able to use them later.

Continuous Integration

Think about Maven !

Be sure that your code can be compiled EVERY DAY ! Use reports to detect as early as possible

the cyclomatic complexity, the lack of documentation, unit tests coverage and so on.

When integrating a new version, start from scratch : no iterative delivery.

You can even scratch your install and reinstall it from the beginning : a good way to know if you're at ease with WLST :)

Your architecture

Which frontend ?

Choose a configuration with a minimum intelligence (WebServer + WLS Plugin)

If you plan using a hardware load-balancer, make sure it handles passive cookies and that you have someone in your team

who knows how it works and how to configure it.

How many managed servers ?

Depends on the capacity planning.

Should involve several physical machines, to have better SLA and/or handle a hardware crash.

You may think about setting up a MAN or WAN if needed (site disaster).

Security configuration (SSL, Authentication, Connection Filters ...) ?

Set some connection filters to avoid man-in-the-middle attacks.

Use SSL if your data is important and must be encrypted but change the DemoCertificates & Keystores.

Remember that SSL has an impact on performance.

SSL might not be sufficient : use a real authentication if needed.

Creating your WebLogic domain

Admin Server

Chose a (free) default port and determine if you wish to dedicate a port for administration traffic.

Template and WLST

Don't forget to industrialize your building. If you have to setup a new configuration at once,

it will spare you a lot of time.

Managed Servers

Pack / Unpack commands to export / import the domain on different machines. (it's like a zip but without the unnecessary files).

Setting up a cluster

Use multicast only for backward compatibility.

Else Oracle recommends strongly the use of Unicast

Don't forget about the cluster address (even if WLS is completing it for you, it's good to know which servers are

defined in your cluster).

Creating your own certificates

Using keytool, manage your own certificates and keystores (trust & identity).

Once you've got your CSR, use a CA or make your own.

Setting up the NodeManager (machines)

I'm going to quote official documentation as it's very detailed.

http://edocs.beasys.com/wls/docs103/nodemgr/nodemgr_config.html

Tuning your domain performance

Defining relevant log levels

No debug on a production server. By cutting out the unnecessary I/O, you'll gain performance and limit the risks of a "disk "quota exceeded".

Keep the logs on a dedicated file system, so that when they dwell, they do not crash the whole system.

Enabling Native I/O

Sounds obvious, but a native socketmuxer can give you a large performance gain.

Configuring session replication

Depends if you have to, but if it's the case, be sure your session isn't too big, and that all the objects you want to replicate are serializable.

Chose wisely your replication group.

Tuning the thread pool

The thread pool has to be defined to a sufficient initial value (default is one and is obviously not enough).

Adjust this parameter according to the test results you got from your load tests.

Tuning the connection pool

One good way to size your connection pool is once again linked to the capacity planning. In a word, you have to put a quite high number

of connections. You launch a stress test and then you take a look at the max number of connections that were created.

It will give you the right number.

Configuring Panic & Overload modes

By default, no action is performed on an OOME and on stuck threads. So far, you can't do a lot of things, but it's better than nothing.

If your server faces a OOME (Out Of Memory Exception), you may request it to shut down. That way, if a NodeManager is up,

it can start your server up again. The main con is that you may not be aware of a server failure as it is fully automated.

Same thing for stuck threads : you can either chose to shut the instance down or switch it to an admin mode, that is to say,

serving only administration requests, for the time threads get unstuck.

Monitoring (optional but recommended)

JConsole

Starting with JVM 1.5, Sun has shipped with its JVM a very nice tool to monitor and supervize a JVM : the JConsole.

http://java.sun.com/developer/technicalArticles/J2SE/jconsole.html

JRockit Mission Control

BEA and now Oracle has its own JVM : JRockit. And the team (hi swedish fellows !) has developped a very cool tool as well.

It allows to monitor what's happening on your JVM, but can also record any activity for a later analysis.

WLDF

Built-in (and then free) tool to replace the "8.1 Performance Tab".

You can choose to monitor all the MBeans exposed by WebLogic Server, that is to say, almost everything !

http://e-docs.bea.com/wls/docs103/wldf_configuring/understand_wldf_config.html

And you may instrument your code as well, as explained in one of my previous posts.

SNMP

The use of external tools such as Nagios, Cacto, Tivoli or Mercury BAC is possible with the activation of SNMP.

http://edocs.beasys.com/wls/docs103/snmpman/index.html

Oracle Enterprise Manager

http://www.oracle.com/enterprise_manager/index.html

Summary : Application Delivery Process

It sounds obvious but it's not often the case : your application must follow a lifecycle such as :

=> Development / Unit Testing
=> Integration (continuous) : build from scratch the application and make sure it deploys correctly and works.
=> FuncTest : Play all the scenarii that are going to be played by users.
=> Benchmark : Once the application is approved : determine the capacity planning and perform
stress tests (very high charge on a short timeline) & load tests (110% of your estimated charge on a large timeline)
=> PreProduction : Deployment on this environment as if it were the final production environment.
=> GO-LIVE : Two options.
- Either you apply the same procedure you did for the PreProduction (with a nightly shutdown for instance)
- Or you switch the roles and make the PreProduction become the Production and vice-versa.
- Or best solution, you have two clusters with a load-balancer which is going to perform a "graceful migration".

Mots clés Technorati : best practices,weblogic,J2EE,Java EE,lifecycle,production,developpement,continuous integration

2009-03-27

WLI : JTA Timeout at about 300 seconds ?

I've been fighting against WLI on that matter for the last two days.

In the beginning, when I saw that stacktrace :

<27 mars 2009 12 h 38 IRST> <Error> <WLW> <000000> <Exception processing myProcess
javax.transaction.TransactionRolledbackException: EJB Exception: : javax.transaction.TransactionRolledbackException: EJB Exception: :

weblogic.transaction.internal.TimedOutException: Transaction timed out after 303 seconds

BEA1-0141042BE78F2733AB68
        at weblogic.transaction.internal.ServerTransactionImpl.wakeUp(ServerTransactionImpl.java:1600)
        at weblogic.transaction.internal.ServerTransactionManagerImpl.processTimedOutTransactions(ServerTransactionManagerImpl.java:1147)
        at weblogic.transaction.internal.TransactionManagerImpl.wakeUp(TransactionManagerImpl.java:1882)
        at weblogic.transaction.internal.ServerTransactionManagerImpl.wakeUp(ServerTransactionManagerImpl.java:1064)
        at weblogic.transaction.internal.WLSTimer.trigger(WLSTimer.java:31)
        at weblogic.time.common.internal.ScheduledTrigger.run(ScheduledTrigger.java:243)
        at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
        at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121)
        at weblogic.time.common.internal.ScheduledTrigger.executeLocally(ScheduledTrigger.java:229)
        at weblogic.time.common.internal.ScheduledTrigger.execute(ScheduledTrigger.java:223)
        at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:224)
        at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:183)

I thought to myself "This is pretty much annoying, but I guess I can handle that !".

And I started to change the JTA parameters in the console :

But it didn't change a thing ...

Eureka ! It's got to be something related to the EJB as said in the error message.

I then exploded (oh yeah) all my Event Generators (embodied by EJBs) and updated the weblogic DD such as :

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE weblogic-ejb-jar PUBLIC "-//BEA Systems, Inc.//DTD WebLogic 8.1.0 EJB//EN" "http://www.bea.com/servers/wls810/dtd/weblogic-ejb-jar.dtd">
<weblogic-ejb-jar>
<weblogic-enterprise-bean>
    <ejb-name>CoctosExportCockpitJMSEventGenerator</ejb-name>
    <message-driven-descriptor>
      <pool>
        <max-beans-in-free-pool>15</max-beans-in-free-pool>
        <initial-beans-in-free-pool>15</initial-beans-in-free-pool>
      </pool>
      <destination-jndi-name>jms.coctos.export.cockpitQueue</destination-jndi-name>
    </message-driven-descriptor>
    <transaction-descriptor>
      <trans-timeout-seconds>3600</trans-timeout-seconds>
    </transaction-descriptor>
</weblogic-enterprise-bean>
<disable-warning>BEA-010001</disable-warning> 
<disable-warning>BEA-010054</disable-warning> 
<disable-warning>BEA-010200</disable-warning> 
<disable-warning>BEA-010202</disable-warning> 
</weblogic-ejb-jar>

But it didn't change a thing ...

I was a bit disappointed and lost because, to me, I had done all that was in my power.

Thanks to Google (once again), I finally reached an interesting page saying :

"The default timeout period for a transaction is 300 seconds (5 minutes)."

Oh gush ! Exactly what I was trying to figure out for two days !!!

So, what am I to do to get rid of that exception ??? (thrill)

Well, it's no big deal, but this option is not easily findable :

Take your process project and locate the directory WEB-INF :

In that directory, you''ll find a wlw-config.xml file.

That's this file you'll have to modify to change the JTA timeout.

Example :

<wlw-config xmlns="http://www.bea.com/2003/03/wlw/config/">
<transaction-timeout>3600</transaction-timeout>
</wlw-config>

And voilà ! But be sure you understand the use of wlw-config.xml and wlw-runtime-config.xml.

Quoting Oracle : Note that the values appearing in wlw-config.xml are hardcoded into the deployment EAR and cannot be overridden by other runtime configuration mechanisms.

For most cases you should use wlw-runtime-config.xml to configure the runtime information for your Workshop application.

Hope that helped !

Mots clés Technorati : wli,weblogic,jta,timeout,300 seconds,300,wlw-config.xml

2009-03-19

Searching on forums.bea.com

Who hasn't faced one day a BEA search result leading to a page on forums.bea.com ?

Pretty easy answer : nobody.

Till a couple weeks ago, it was possible to use Google cache to get the desired posts.

But now the cache's expired, it's impossible.

Well, that's what lots of people think but they're mistaken !

Oracle has migrated the whole site onto forums.oracle.com.

Unfortunately, Google hasn't indexed those pages yet.

However, you still can perform a search directly on the site.

As I like to take care of my readers :) , I've looked for the most direct link, and here it is :

http://forums.oracle.com/forums/search.jspa?threadID=&q=&objID=c202&dateRange=all&numResults=30&rankBy=10001

Click that link, enter some keywords, and voilà !

Mots clés Technorati : forums.bea.com,forum,bea,oracle,google

2009-03-16

JAX-WS, OSB & BPEL Process Manager : a simple example

If you can't wait next summer to know what's going to happen with SOA Suite 11g,

here's a sample to manage your frustration :)

Presenting the context

The example I've chosen is a quite simple one. Indeed, it's no real case,

just a convenient way to illustrate how the complete chain works.

=> A user wants to go to the movies. He will send the movie title to a webservice

which will route him to the theater where there are the most available seats.

To do that example, I've installed :

=> OSB 10gR3 / Workshop 10gR3
=> SOA Suite 10g 10.1.3.1.0

In the end, I'll have three JVMs running :

=> One for OSB
=> One for WLS (hosting the webservices)
=> One for BPEL Process Manager (OC4J)

Designing some webservices (JAX-WS)

We'll design three webservices, in order to have enough data to process.

Three webservices (one per theater) :

=> UGB
=> Gomon (GMN)
=> MK3

They all have the same business contract : only the returned result differs from one to another.

You can have a detailed example on how to design a JAX-WS webservice on another blogpost :

http://m-button.blogspot.com/2008/07/building-jax-rpc-or-jax-ws-webservices.html

Example : the Gomon Webservice

package fr.mbutton.blog;

import javax.jws.*;

@WebService(targetNamespace="http://fr.mbutton.blog/gomon")
public class CheckAvailability_GMN {

    @WebMethod
    public boolean isMovieStillPlaying(String movieTitle) {
        return true;
    }
    @WebMethod
    public int getAvailableSeatsForMovie (String movieTitle) {
        return 124;
    }
}

And here's a sample build.xml for this webservice :

<project name="GMN" basedir="." default="build-webservice">

    <taskdef name="jwsc" classname="weblogic.wsee.tools.anttasks.JwscTask" />
    <property name="dest.dir" value="D:\BEA_ROOT\user_projects\workspaces\osb_bpel\Theaters\EarContent" />

    <target name="build-webservice">
        <jwsc srcdir="src" destdir="${dest.dir}">
            <jws file="fr/mbutton/blog/CheckAvailability_GMN.java" type="JAXWS" />
        </jwsc>
    </target>

</project>

The execution of the ANT build.xml will create one WAR per webservice.

Then, just deploy the three WARs to your WLS domain and make sure they're all reachable.

To do so, click on each of them :

And then, once you've clicked on the webservice you want to test, you should see a "Testing" tab.

You can choose either to show the WSDL or launch a test client. Either way, you'll be sure it's up & running.

Setting up an OSB configuration

Here's a very simple definition. No extra processing : the proxy service will directly route to the

business service, with no additional computing.

Note : If you're working with two separate domains at the same time, I'll advise you to read that article :

http://m-button.blogspot.com/2009/02/working-with-two-weblogic-domains-on.html

A little remark : if you take a closer look to the generated webservice, and especially to their WSDLs, you'll see

that the XML schema is imported :

<types>
<xsd:schema>
<xsd:import namespace=http://fr.mbutton.blog/gomon

     schemaLocation="http://192.168.100.1:8001/CheckAvailability_GMN/CheckAvailability_GMNService?xsd=1" />
</xsd:schema>
</types>

In OSB, you'll have to import those schemas separately and then link them to the corresponding webservice.

Then create as explained above, one business service per webservice exposed, and one proxy service per business service, like :

Once everything is correctly set up, test your configuration by trying to reach the proxy services.

To do so, click on a proxy service, it will lead you to this screen :

To test it, take your server URL (for example : http://myserver:7001), add the endpoint URI + "?WSDL" and check it you've got something :)

No need to tell that if you don't, it's not normal, right ?

Orchestrating webservice calls thanks to BPEL PM

Here we go ! Now, we've got three webservices up and running and we applied the mediator pattern thanks to an ESB.

What we need now is to call the webservices, store the results and return a computed result.

Exactly what BPEL was designed for.

Note : former BEA clients used WLI (WebLogic Integration) for the same kind of matter.

First thing, open JDeveloper and create a new Application (similar to an Eclipse workspace).

Once created, add a synchronous BPEL Process Project.

Next step :

In the application pane, an empty BPEL project is created :

In the graphical view, you'll only see two steps (receiveInput & replyOutput) and a partnerLink (client).

First of all, let's modify the name of the BPEL process variables.

To do so, open the associated XSD in Integration Content > Schemas and change element names (in blue) :

<schema attributeFormDefault="unqualified"
    elementFormDefault="qualified"
    targetNamespace="http://fr.mbutton.blog"
    xmlns="http://www.w3.org/2001/XMLSchema">
    <element name="BlogSampleProcessRequest">
        <complexType>
            <sequence>
                <element name="movieTitle" type="string"/>
            </sequence>
        </complexType>
    </element>
    <element name="BlogSampleProcessResponse">
        <complexType>
            <sequence>
                <element name="theaterName" type="string"/>
            </sequence>
        </complexType>
    </element>
</schema>

As we will call three webservices at the same time, we'll need a "Flow" activity.

(Note : BPEL PM does spawn a thread for each branch. Whereas WLI has a flow-like activity, it's not parallel but sequential)

Drag and drop it to the node between the receive and the reply.

Open the branches. We'll need three branches and in each, a call to a webservice will be performed.

Add an "Invoke" activity in the first branch.

We'll have to link that Invoke activity to one of the webservice.

What you have to know is that any external part to the BPEL process

is called a "Partner Link" or "Adapter".

PartnerLink (client, webservice)

Adapter (file, JMS message and so on)

So to create a reference to the webservice, let's create a partnerLink.

Drag and drop a partnerLink service to the right side of the BPEL process. (it's a good practice to keep BPEL process callers

on the left and BPEL process called partners on the right).

Once the item is dropped, a popup will show up.

Enter the URL of your webservice and click refresh (circle made of the two blue arrows).

Accept the proposal.

Draw a line between the invoke action and the partnerLink (you can bind these items differently)

Choose the operation to invoke and create local variables.

For now, the wiring has been made, but we need to assign the bpel input variable to the webservice input variable.

Just add a Assign activity

and copy the value such as :

In order to see if our bpel process is working, we'll add another Assign activity which will assign the returned value

to the client.

Of course, this copy isn't right : the returned result represents a number of seats and the client expects a theater name ...

But it's just for some testing matters.

To build and deploy your BPEL process, edit the build.properties and make sure all the information match your configuration.

(port/user/password ...)

Right-click the application, chose "Make". Then right-click on the build.xml file and choose "Run Ant Target" > "Deploy".

Select your BPEL Process and run the test console. Enter a movie title and notice the result.

It should be something like :

Our BPEL process works perfectly.

I won't go deeper in the process and explain how to compare and assign values; you can figure out what to do next

by yourself and if not, you will find some good references on the Net.

If you want to learn more, contact Oracle University :)

Exposing the BPEL process through OSB

Last but not least, our BPEL process has to be exposed as a webservice. But as we started to use the mediator pattern,

no way a client can call the BPEL process directly. It has to be called through the ESB.

To do so, create a business service based on the BPEL process WSDL.

To get the WSDL, click on the corresponding tab :

And in the bus, create a Business Service based on this WSDL.

Note : You'll have to create a schema first.

Import your schema from the URL (for me it's : http://mbutton01:8888/orabpel/default/BlogSample/1.0/BlogSample.xsd)

Once your WSDL is successfully created, map a business service on it and choose "BPEL 10g" as the transport :

For the endpoint URI, use the URL given on the WSDL page, but replace protocol "http" by "opmn".

On the next screen, chose "Synchronous Client".

Review and accept the configuration.

Create a proxy service based on the newly created business service :

Test your proxy service and you should have your BPEL process correctly exposed.

Mots clés Technorati : OSB,BPEL,JAX-WS,WLS,WSDL,JDeveloper,Workshop