practicing techie

tech oriented notes to self and lessons learned

Monthly Archives: September 2012

Getting started with Oracle WLS 12c

I’ve been developing software for different incarnations of the Oracle Application Server in the past (Oracle OC4J 10g R3 and BEA/Oracle WebLogic (v8.1 and v10.3), but it’s been quite a while since my last encounter with the server. During recent years I’ve been involved mostly with other application servers. Despite occasional hiccups, I had been reasonably satisfied with the server, so I was curious to give the latest version of Oracle’s application server a quick test drive. Having a background in software development, I thought I’d approach this first from a developer perspective, checking out how the application development workflow (including code, build, deploy) feels like with the latest version. Obviously, much of the workflow is actually about generic Java EE development (as opposed to app server specific development) as long as you adhere to the standard, but I’ve felt that trying simulate the development workflow gives you a more complete view of what it’s like to work with a particular app server product. Instead of coding a Java EE app myself or porting an existing one, I thought I’d work with sample applications made available by others.

Installing Oracle WebLogic Server

There are several options for getting WebLogic running for development purposes: a) use Oracle JDeveloper and the embedded WLS server b) use the IDE of your choice and the WLS zip distribution c) use the IDE of your choice and the full WLS. Since the focus of my test was to check out WLS for development purposes (but not JDeveloper), I chose option b.

So, I downloaded the following WLS distributions from Oracle:

  • WLS Zip Distribution for Oracle WebLogic Server 12.1.1.0
  • WLS Supplemental Zip Distribution for Oracle WebLogic Server 12.1.1.0

The first distribution includes the application server itself and weighs approximately 184 MB. The second one includes sample code.

The installation process is pretty well documented in the WLS package README files but there were a few small gotchas, though. The supplemental zip distribution also includes a nice set of documentation, including architecture description, for the samples in found in $MW_HOME/wlserver/samples/server/index.html.

Here’s the installation procedure I used:
(The text below has been written for Mac OS X and assumes WLS has been installed in $HOME/opt/wls1211_dev but it should be trivial to adapt the instructions for other configurations.)

# 1. extract WLS (see README.txt in WLS package)
mkdir wls1211_dev
cd wls1211_dev
unzip ~/Downloads/wls1211_dev.zip

# 2. set environment variables
export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home
export USER_MEM_ARGS="-Xmx1024m -XX:MaxPermSize=256m"
export MW_HOME=$HOME/opt/wls1211_dev

# 3. run the installation script
. ./configure.sh

We’ll skip WLS domain creation for now, because the samples setup script creates one for us, and start up and move straight to installing the WLS supplemental distribution.

# wls supplement (see README_SUPP.txt)
unzip ~/Downloads/wls1211_dev_supplemental.zip

# 64-bit environments
. wlsenv.properties

# create WLS domain, server, database etc.
./run_samples.sh

This script sets up a WLS domain, WLS server and a database server for the sample application, configures datasources etc. When I tried to start up the sample domain at this point, I received an error about JRE not being found, so decided to reset the environment variables by firing up a new shell session and then set the WLS environment variables again:

# start up WLS sample domain
export MW_HOME=$HOME/opt/wls1211_dev
export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home
export USER_MEM_ARGS="-Xmx1024m -XX:MaxPermSize=256m"
$MW_HOME/wlserver/samples/domains/medrec/startWebLogic.sh

If you have a GUI session with your OS, a web browser should open up with the sample application page.

Sample app #1: MedRec

Oracle provides a WLS supplemental zip distribution aimed at development use. The supplement includes code samples demonstrating various aspects of using different Java EE technologies. It also includes a complete Java EE v5.0 (why not v6.0?) sample application called Avitek Medical Records or MedRec. It claims to “showcase the use of each Java EE component, and illustrates best practice design patterns for component interaction and client development“.

After I got the application server and sample application up and running I wanted to start browsing the application source code and see how to make modification.

You can build and deploy the sample application using the following commands:

# set up the environment for the build
export MW_HOME=$HOME/opt/wls1211_dev
. $MW_HOME/wlserver/samples/domains/medrec/bin/setDomainEnv.sh
cd $WL_HOME/samples/server/medrec

# build + deploy
ant -Dnotest=true deploy

The Ant command will build and deploy the new application version, if you have the application server up and running. (Environment variables set by the WLS installation scripts appeared to interfere somehow with the ones set by setDomainEnv.sh and I had to start a new shell session to make the build work.)

The sample application includes Eclipse project and classpath files, so you can easily import the application code in Eclipse (e.g. Juno). The application depends on different Java EE and third-party APIs that are bundled with the application, so you’ll end up seeing lots of errors in Eclipse. The easiest way to get the source code imported and classpaths set up correctly is to use the Oracle provided Eclipse distribution (Oracle Enterprise Pack for Eclipse [v12c for Eclipse Juno]). Here’s how to import the code in OEPE and create a WLS 12c runtime configuration:

  • create new workspace
  • configure WebLogic server runtime
    select: window / show view / other
    server / servers
    new server wizard
    select server type: Oracle / Oracle WebLogic Server 12c
    and fill in the following:
    WebLogic home: $HOME/opt/wls1211_dev/wlserver
    Java home: /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home
    Domain directory: $HOME/opt/wls1211_dev/wlserver/samples/domains/medrec
  • import MedRec source code
    file / import: general / existing projects into workspace
    select root directory: $HOME/opt/wls1211_dev/wlserver/samples/server/medrec
  • select all 12 projects
    configure target server runtime for each project
    select project / properties / server or targeted runtimes and choose “Oracle WebLogic Server 12c”. Uncheck WebLogic 10.3 version.
  • refresh all projects

At this point your Eclipse project explorer should look like this and you should be able to do a full modify-build-deploy cycle:

Sample app #2: Pet Catalog

The Pet Catalog is a Java EE 6 sample application that demonstrates usage of JavaServer Faces 2.0 and the Java Persistence API. It’s based on a three-tiered architecture on a logical level, but both the presentation and logic tier components are packaged in a single WAR module.

With the first sample app, we were able to skip creating a WLS domain because the installation script created one for us, but now we’ll have to create one. In WLS, the concept of a domain refers to a logically related group of WLS servers and/or server clusters that are managed as a unit. Each domain has an administration server, which is used to configure, manage and monitor other servers and resources in that domain. Additional servers in the domain are called managed servers, which are used for deploying and executing Java EE artifacts. The administration server is meant to be used only for administration, though you can deploy applications to it in development installations.

Creating a WLS domain

# setup WLS environment
export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home
export USER_MEM_ARGS="-Xmx1024m -XX:MaxPermSize=256m"
export MW_HOME=$HOME/opt/wls1211_dev
. $MW_HOME/wlserver/server/bin/setWLSEnv.sh
# create a new WLS domain and start WLS
mkdir -p $HOME/wls/dom1
cd $HOME/wls/dom1
$JAVA_HOME/bin/java $JAVA_OPTIONS -Xmx1024m -XX:MaxPermSize=256m weblogic.Server

Building the application

The source code links found on the sample app web pages didn’t seem to be working. The application source code comes bundled with NetBeans 7.2 Java EE, however so you can get the source code from NetBeans by choosing:

File / New Project
choose project: samples / Java EE / Pet Catalog

Java’s “Write once, run anywhere” is a great value proposition, but especially in Java EE space delivering on that proposition has been lacking. Portability issues arose also in this case, when I tried deploying to WLS the Pet Catalog app, that apparently had been tested mostly on GlassFish. The actual issue seemed to be related more with the particular JPA implementation (EclipseLink) than standard JPA, but I think it’s telling evidence of portability issues since this is supposed to be a standard Java EE showcase sample application. Once, I managed to find out what was causing the issue, fixing it was simple. Often application servers have their own, sometimes very unintuitive, ways of reporting issues and troubleshooting is an area where experience in your particular application server product can really make a big difference. Also, often with well-architected applications, it’s the packaging and deployment where portability problems typically arise, instead of the actual code.

In this case I ran into a problem with datasource authentication. To fix the deployment issue I had to modify the persistence unit definition in persistence.xml by commenting out eclipselink.jdbc.user and eclipselink.jdbc.password parameters.

Deploying the application

Create and initialize the database

Pet Catalog uses a MySQL database for persisting data. A database, tables and a user account must be created before deploying the application.

create database petcatalog;
GRANT ALL ON petcatalog.* TO 'pet1'@'localhost' IDENTIFIED BY 'pet1';
cat setup/catalog.sql | /usr/local/mysql/bin/mysql -h 127.0.0.1 -P 3406 -u pet1 -f -p petcatalog
Create a Data Source

Once you’ve set up the database, the database connection or datasource needs to be configured in the application server. To do this, log on to WLS console and do the following:

Choose: Services / Data Sources / New / Generic Data Source

Then on “JDBC Data Source Properties” page fill in the following:

  • Name: petCatalogDS
  • JNDI Name: jdbc/petcatalog
  • Database Type: MySQL
  • Database Driver: MySQL’s Driver (Type 4) Versions:using com.mysql.jdbc.Driver

And on “Transaction Options” page:

  • Supports Global Transactions
  • One-Phase Commit

Then “Connection Properties”:

  • Database Name: petcatalog
  • Host Name: localhost
  • Port: 3406
  • Database User Name: pet1
  • Password: pet1

Test Database Connection

And finally on “Select Targets” page choose the server to deploy to:

  • myserver
Deploy WAR

Finally, deploy the application WAR to WLS. The application should run without customizing any deployment parameters.

Conclusions

In my quick test drive I focused mostly on the development workflow related aspects of the WebLogic server (developer distribution), and not operational aspects such as scalability, availability, reliability, operability etc. WLS appears to be a capable, feature rich Java EE application server, as could be expected from a major vendor, but the zip distribution was also relatively light-weight and ran quite well on my laptop.

WLS has very nice server administration capabilities: you can easily view and edit the configuration using command line tools, but a comprehensive web-based administration console is also available that allows you to perform any server administration task. The server configuration is persisted in XML files (e.g. config.xml) that are stored under a single filesystem directory tree, which makes it easy to compare and migrate configuration files. The console just enables administrators to manipulate these configuration files through a web UI. The web console has a much more comprehensive feature set than e.g. the one in Jboss EAP 5. WebLogic also features a command-line scripting environment (WLST) that you can use to create, manage, and monitor WLS domains. Due to XML based configuration and scripting support backup and recovery of server configuration, as well as taking snapshots and rolling back changes should be easy. Deploying the exact same configuration should be simple as well.

It seems odd that the sample application doesn’t showcase all the new features of the latest-and-greatest Java EE specification version that the WebLogic server supports. Also, the basic development mode installation could’ve been made simpler still, similar to some other app servers where you only need to do a simple unzip. Production installation is of course an entirely different story.

SFTP file transfer session activity logging

Old-school bulk file transfers may be considered out of fashion in a SOA world, but we’re not in a perfectly SOA enabled world, yet. So, bulk file transfers still remain widely used and SFTP is the workhorse of bulk file transfers.

Typically you would create an operating system account for each inbound file transfer party you want to enable and configure SSH public key authentication. By default, this would also allow full terminal access for account. What if you don’t want to have that? Well, just define sftpd as the login shell for those users to restrict access:

sftp1:x:502:502::/home/sftp1:/usr/libexec/openssh/sftp-server

Simple enough.

What if you want to monitor SFTP session activity on the system? Depending on SSH and syslogd config, you’ll find session logs in different places, but often on Linux this will be in /var/log/secure. OpenSSH v5.3 logs the following lines for a SFTP session:

Sep 16 15:31:34 localhost sshd[4274]: Postponed publickey for sftp1 from 172.16.221.1 port 56069 ssh2
Sep 16 15:31:34 localhost sshd[4273]: Accepted publickey for sftp1 from 172.16.221.1 port 56069 ssh2
Sep 16 15:31:34 localhost sshd[4273]: pam_unix(sshd:session): session opened for user sftp1 by (uid=0)
Sep 16 15:31:34 localhost sshd[4276]: subsystem request for sftp
Sep 16 15:31:36 localhost sshd[4276]: Received disconnect from 172.16.221.1: 11: disconnected by user
Sep 16 15:31:36 localhost sshd[4273]: pam_unix(sshd:session): session closed for user sftp1

You can see the session start, source IP address and user account info there. This is enough information for basic transfer account activity monitoring, and in some cases it may be enough for accounting and billing as well.

But what if you want to customize logging or hook into SFTP session start and end events to allow some form of custom processing? You can create a wrapper script for sftpd and configure that as the user’s login shell:

#!/bin/sh
#
# sftpd wrapper script for executing pre/post session actions
#

# pre session actions and logging here
SOURCE_IP=${SSH_CLIENT%% *}
MSG_SESSION_START="user $LOGNAME session start from $SOURCE_IP"
logger -p local5.notice -t sftpd-wrapper -i "$MSG_SESSION_START"

# start actual SFTP session
/usr/libexec/openssh/sftp-server

# add post session actions here

You could replace the syslogd based logging command above with your custom logging. The logging command above would log session start events using local5 log facility with notice priority (see man rsyslog.conf(5)). Log entries using a local5 log facility can be directed to a custom log file using the following syslogd configuration in /etc/rsyslog.conf:

local5.*                                                /var/log/sftpd.log

So, now you can customize the messages and perform pre/post session actions. If you need to do more advanced reporting for this data or allow non-technical users to do ad-hoc reporting, you might want to put the session data in a RDBMS. You could either add the data to the database directly in the wrapper script or set up logrotate to rotate your custom log files and configure a postrotate/prerotate script that would parse the log file and add entries to the database in batches.

What if you need to know what exactly goes on inside the file transfer sessions, like which files are being downloaded or uploaded etc.? OpenSSH sftpd doesn’t log this info by default, with the default facility and log level being auth.error. You can change this either globally in sshd_config or per user by changing the sftpd-wrappper script above like this:

/usr/libexec/openssh/sftp-server -f local5 -l info

This would direct sftpd log entries issued with local5 facility and info priority for selected users only. So, now your sftpd.log would look like the following:

Sep 16 16:07:19 localhost sftpd-wrapper[4471]: user sftp1 session start from 172.16.221.1
Sep 16 16:07:19 localhost sftp-server[4472]: session opened for local user sftp1 from [172.16.221.1]
Sep 16 16:07:40 localhost sftp-server[4472]: opendir "/home/sftp1"
Sep 16 16:07:40 localhost sftp-server[4472]: closedir "/home/sftp1"
Sep 16 16:07:46 localhost sftp-server[4472]: open "/home/sftp1/transactions.xml" flags WRITE,CREATE,TRUNCATE mode 0644
Sep 16 16:07:51 localhost sftp-server[4472]: close "/home/sftp1/transactions.xml" bytes read 0 written 192062308
Sep 16 16:07:54 localhost sftp-server[4472]: session closed for local user sftp1 from [172.16.221.1]

The log entry format requires some effort to process programmatically, but its still manageable. You can identify SFTP session operations, such as individual file transfers from the log. The log could then be processed using a logrotate postrotate/prerotate script that could e.g. add the data in a database for generating input data for accounting or billing.

Software versions: RHEL 6.3 and OpenSSH 5.3.