Demystifying agents in Oracle Data Integrator!

Hello folks!

A little background

Agents are basically services written in Java running on a specific TCP/IP port that carry out the execution of Interfaces, Packages etc. In previous versions of ODI namely ODI 10g one only had the standalone agents at his disposal. Ever since ODI 11g was introduced, the concept of Java EE agents came into the picture. Overall, both these agents are (almost) the same piece of code. The differences in them is what we will learn ahead!

The tale of two Agents

A Standalone agent will run on top of a Java machine or any machine with a JVM implementation. It is easier to deploy and we can create multiple such agents. The question that arises is how many would you keep on creating? In an environment that demands scalability, standalone agents would eventually fail. In such situations we use Java EE agents. Java EE agents are deployed as a Weblogic server application. Unlike standalone agents, they utilize the clustering and connection pooling features of the Weblogic server. They are a bit complex to deploy but promise to offer enterprise wide scalability. Standalone agents use the built-in ODI load balancing feature and do not support clustering and connection pooling.

Scenario

Let’s say you want to run an ODI Scenario that populates hundreds of records from the source table to the target table. You create two or three Standalone agents running on different ports (load balanced) that carry out the execution of the scenario. Now, the agent always first connects to the master repository, obtains the connection information to the work repository, connects to the work repository and then finally executes the Scenario. Each time the standalone agent tries to connect, it creates a new connection request to the database. As opening/closing of database is an expensive process, at some point in time there might exist a connection overhead resulting in slow down of the operation. A better option then would be to use a Java EE agent that is by default highly scalable, available and supports connection pooling.

odi_agents

What can we conclude?

Finally, it is the user that has to decide which type of Agent is required. It is quite possible for both the agents to exist simultaneously in the same ODI architecture. The main difference is how you install them, in WLS or on top of JVM.

Source: What’s new in ODI 11g – Part 1

Tutorial #49: Creating Java EE Agent in Oracle Data Integrator

Hello friends!

Today we will learn how to create a Java EE agent in Oracle Data Integrator. In one of my previous tutorials I had explained how to create a stand-alone agent in ODI. The only difference in this case is that we would require to install Weblogic server in order to configure our Java EE agent.

Pre-requisites: Oracle Data Integrator 11gR1 (build version 11.1.1.7.0), Oracle WebLogic Server 10.3.6 (x86 for 32 bit JVM)

Step 1: Download Oracle weblogic server

Go to the Oracle software downloads page and download a previous release 10.3.6 of Oracle Weblogic server.

download_weblogic_server

Note: I assume while installing Oracle Data Integrator, you have selected the option to install Java EE agent as well. If not, I would recommend carrying out a fresh installation before proceeding with this tutorial.

Step 2: Deploy and configure new domain with Weblogic server

Open command prompt and change directory to \common\bin, for example: C:\oracle\Middleware\Oracle_ODI2\common\bin. Execute config command.

Follow the screenshots below and enter the information required for various fields as shown in the screenshots!

domain_configuration_1

domain_configuration_2

domain_configuration_3

domain_configuration_4

domain_configuration_5

domain_configuration_6

domain_configuration_7

domain_configuration_8

domain_configuration_9

domain_configuration_10

domain_configuration_11

domain_configuration_12

domain_configuration_13

Click on the Create button and let the installation take place!

Note: Don’t start Admin Server now

Step 3: Create ODI Java EE agent for deployment on domain ODI-DOMAIN

create_agent_1

create_agent_2

create_agent_3

create_agent_4

create_agent_5

create_agent_6

Step 4: Connect to Weblogic server and managed server odi_server1

Change directory to the directory of the WLS Home –> C:\Oracle\Middleware\user_projects\domains\ODI_DOMAIN\bin. Execute the startweblogic command.

connecting_to_weblogic_1

connecting_to_weblogic_2

Security must be set up for the JAVA EE application to have access to the ODI repository. The entry will be created within the credential store, which will allow the JAVA EE Agent to authenticate itself to be able to use the required resources. This user should be a user that is already set up in the ODI Security.

Change directory to the WLS_HOME common bin –> C:\Oracle\Middleware\oracle_common\common\bin, and then execute the command wlst.

connecting_to_weblogic_3

connecting_to_weblogic_4

To connect to the running Admin server, execute the following command: connect(‘weblogic’,’welcome1′,’t3://localhost:7001′)

Note: Here welcome1 is my weblogic password.

connecting_to_weblogic_5

connecting_to_weblogic_6

Execute the following command to add the correct credential store for ODI Supervisor : createCred (map=”oracle.odi.credmap”, key=”SUPERVISOR”, user=”SUPERVISOR”, password=”SUNOPSIS”, desc=”ODI SUPERVISOR Credential”).

connecting_to_weblogic_7

connecting_to_weblogic_8

Note: All commands are case-sensitive! You can copy commands from the text file provided here.

Also execute the command to add the correct credential store for the domain ODI-DOMAIN: createCred (map=”oracle.odi.credmap”, key=”ODI-DOMAIN”, user=”weblogic”, password=”welcome1″, desc=”ODI-DOMAIN Credential”). To exit WLST, execute the command exit() .

connecting_to_weblogic_9

connecting_to_weblogic_10

To start managed server odi_server1, change directory to WLS Home –> C:\Oracle\Middleware\user_projects\domains\ODI-DOMAIN\bin , and use the execute command.

StartManagedWeblogic odi_server1. For Username, enter weblogic. For the password, enter welcome1. Verify that your managed server is started in RUNNING mode.

connecting_to_weblogic_12

connecting_to_weblogic_13

Finally, when the weblogic server is up and running, test the connection to the ODI agent created previously. A successful connection indicates that you have configured the Java EE agent correctly. That’s it for this tutorial. Have a nice day! 🙂

test_success

Source: Deploying and configuring Java EE Agent

Tutorial #32: Create new agent in ODI

Hey everyone!

Through this tutorial I will explain how to setup and install an Agent in Oracle Data Integrator(ODI). An Agent (ODI Agent) is basically a service written in Java that acts as a listener on a specific TCP/IP port and helps in execution of packages, scenarios, interfaces etc.

Pre-requisites: Oracle Data Integrator 11g (build version 11.1.1.7.0)

Step 1: Making changes to odiparams.bat file

The odiparams.bat file can be found inside the bin folder at location, C:\oracle\product\11.1.1\Oracle_ODI_1\oracledi\agent\bin

Open the file and make the following changes under Repository connection information.

Driver and URL information

REM #
REM # Repository Connection Information
REM #
set ODI_MASTER_DRIVER=oracle.jdbc.OracleDriver
set ODI_MASTER_URL=jdbc:oracle:thin:@localhost:1521:xe
set ODI_MASTER_USER=odi_karan
set ODI_MASTER_ENCODED_PASS=aYyaFNau230FM1ivBygiAp

Note: In the above case odi_karan is the username of my master repository. Also, you need to enter the encoded password for the master repository in the ODI_MASTER_ENCODED_PASS field.

Password can be encoded using the following procedure:

Open a new command prompt window and navigate to the location, C:\oracle\product\11.1.1\Oracle_ODI_1\oracledi\agent\bin. Keep this instance of command prompt open till the end of this tutorial.

Encoding password of master repository

Encoding password of master repository

Syntax for encode command: encode yourpassword

Add the encoded password to the ODI_MASTER_ENCODED_PASS field. You also need to enter the name of your work repository at the following location!

REM #
REM # Work Repository Name
REM #
set ODI_SECU_WORK_REP=WORKREP1

Note: WORKREP1 is the name of my work repository in ODI. Finally when you’re done adding all these details, save and close the odiparams.bat file.

Step 2: Create ODI Physical and Logical Agents

Open ODI Studio and connect to your work repository

connect to work repository

connect to work repository

Under Topology navigator go to Physical Architecture. Right click Agents –> New Agent. Let name of agent be localagent.

Create physical agent

Create physical agent

This creates your physical agent!

Similarly, create logical agent under Logical Architecture as follows:

Create logical agent

Create logical agent

Now, in order to start the agent, type the following command in the already open command prompt instance.

agent -NAME=localagent
Start ODI agent

Start ODI agent

Now go back to ODI Studio and test the connection to your ODI agent.

test_agent_connection

You have now successfully created your own ODI agent. You can use the same for execution of scenarios, packages etc.