Tuesday, November 12, 2019

Ruby and Rails tracing

To trace execution in a Ruby/Rails application add the following -

t = TracePoint.new(:line) do |tp|  

p "#{tp.path}:#{tp.defined_class}:#{tp.method_id}:#{tp.lineno}" 

end

t.disable

Wednesday, June 26, 2013

These past few days I have been busy finding out a good workflow software that can help me get all the different parts of my new app co-ordinated. At first I had a client server model where the server does all the heavy lifting and the client requests action. This is a good model of design when simple tasks are to be run that do not need parallel work execution. Also all the workflow logic was separated from the server module and was present in the client module. All started off as a small java project and quickly grew into a monster that needed mode firepower and improvements. Although from a coding standpoint functionality was abstracted and was easily extendable, but I started feeling that the one server model where one requested action can run in parallel was slightly slower than what I wanted. The design was not scalable.

Nowadays I have been reading so much about scalable design and similar technological feats that I sort of wanted to jump into some project that has this. So I started rebuilding my old project with these things in mind. Older modules were triggered when client requested some action. For example fetch something or calculate something. Now this calculation might take time and could be a long job. It was not executed in parallel. Also if I had to run two jobs in parallel where one task was dependent on the other there was no way or no logic of coding those ideas. But the main bottleneck was parallel execution.

Workflow is something that will greatly help in the execution process and co-ordinating them. But designing a workflow system from scratch is again a big job. Good for a project but not good when the end goal is something else where workflow is a utility function that you need. So, I recently came across a set of tools that come under the umbrella ESB. From what I get from my research, it is a vague term that has a varied set of products under its umbrella, with each specializing in different areas. So, for last few days I have been searching and searching for ESB products. ESB stands for Enterprise Service Bus. A term that symbolizes a engine that allows mix and match and transfer mechanism, coupled with a workflow system. I looked at different ESBs – UltraESB, MuleESB, WSO2 ESB. One was good at performance, and one was good at features. But at the end it seemed that MuleESB is the easiest to use. It has a very intuitive IDE integration available for Eclipse that makes using the ESB simple and fun. I rather liked it. No wonder it is used across so many production environments. MuleESB is what I will use for starting off, which I hope will make the workflow part of my project across JMS and HTTP endpoints and sources easier.

Friday, August 24, 2012

Using Oracle Identity Analytics 11g Web Services

Introduction

The purpose of this document is to illustrate and provide a starting point to use web services exposed by Oracle Identity Analytics 11g. Apache CXF 2.6.1 is used to generate the wrapper libraries.

Build Library Setup

We also need a few jars to get everything compiled

Samples

To run the samples you need to configure the file settings.properties with the following

Property Name	Description
user	Username for connecting to OIA
password	Password for OIA user
wsdlurl	Address for WSDL file

userService

roleService

businessService

auditService

Enabling Web Services in Oracle Identity Analytics (OIA)

Edit web.xml with the following changes.

a. Add classpath:org/codehaus/xfire/spring/xfire.xml

b. Uncomment the servlet definition and mapping for webservice

c. Restart OIA.

Webservice Endpoints

For a server that is running on localhost with port 7001 we have four web service endpoints available. These four endpoints are

http://localhost:7001/rbacx/ws/userService?wsdl

http://localhost:7001/rbacx/ws/roleService?wsdl

http://localhost:7001/rbacx/ws/auditService?wsdl

http://localhost:7001/rbacx/ws/businessUnitService?wsdl

Creating Java client

We can easily call the web services by creating a java client for the above 4 endpoints. We will be using Apache CXF to build our client application. However there are other ways also like using Apache Axis 1.x and Apache Axis 2.x. Every framework has its own way to wrap the web service operations, which we will not be covering here.

With Apache CXF we have a neat little command line client wsdl2java which creates the java client classes for us. We created the clients for the above four web services as shown below

I first saved the WSDL document from the above URLs. Also since the default generation logic creates a rather weird data structure to use I have put in some customization in javabindings.xml which is specified during client generation.

Role Service

User Service

Business Unit Service

Audit Service

Javabindings.xml

Once this is done classes are created in the generated folder. Please note that I created separate java projects for each of the web service. If you do all of them in one project you would need to do some manual class changes.

I added some test code also, which I’ll explain below. The projects structure in my eclipse looks like:

Calling the webservice

Any operation that is to be requested needs authentication. The web service client classes do not have any in-built authentication wrappers built. Rather this functionality is present in Apache CXF’s classes. We will go step by step on how the client is created now.

Step 1: Getting the client instance

We have four endpoints, each of which has a different client class to start off with. Below is what we need to get started for User Service.

It will be similar for the other services, for audit we will get the AuditService and so on for the others.

Step 2: Setting up the authentication

As mentioned earlier we need to take help from Apache CXF’s classes to get the authentication working. We wrapped all this up in one method in the class Common.

Here we are doing two things:
1. Add logging by adding LoggingInInterceptor and LoggingOutInterceptor in the interceptor queue.
2. Add username/password for authentication by inserting a WSS4JInterceptor

The above code is how Apache CXF wants it, so more details on why and how on this can be found on Apache CXF’s website.

Step 3: Calling the webservice

Each webservice has different operations available. However the way of operating is similar. Shown below is the easiest method to call getVersion() which is present in all the webservices.

Wednesday, August 8, 2012

Active Roles Server and OIM

Why do companies love web service? Active Directory follows the LDAP v3 protocol to communicate, but no some people are hell bent on getting web service over this neat little bundle wrapped in LDAP. So here comes ARS or Active Roles Server. The server itself provides a web based console to access the users contained inside Active Directory. It has a add-on component that one has to add to enable a web service interface (follows SPMLv2), which allows us to create users via a web service. This web service follows the SPML v2 standard, but so much for just following a new standard, when a already established standard is being used and that too being light weight!

When we first started developing the Active Directory Connector, fitting in the requirements, the default OOTB connector for Active Directory 9.x was being used. I wish it was used all the way, but the client wanted us to use the Active Roles Server web service. I never had development experience with web services so I tried experimenting with OIM11’s SPML web service. I created a client using Axis 1.x for it and it worked perfectly. But when it came to ARS (Active Roles Server), it turned out ugly.

I started with Axis 1.x , created a client but had to spend a very long time to get my head around how to send the username/password for the request. It seemed like a crazy jungle where I have no help. But I finally had some help from a few YouTube videos and I was able to get it working. My idea was to use similar steps to complete the web service client for the ARS web service, but it all turned out into waste. I tried Axis 1.x / Axis 2.x. / Apache CXF / wsimport, but ended up with new problems every time. Axis 1.x created me a request class with Object[][] data type for a tag which was like – “<attr> <data><value></value></data><data><value></value></data> </attr>”. I had no clue how to get it from a Object[][]. Axis 2.x gave me the request class correctly but the response was not getting parsed and it used to throw an error when I got the response. Finally I went to the basics to solve all this mess.

A web service is a HTTP request. Why not hand craft the XML and send it. “Sound simple enough” is what I thought. I did not want to write all the code to create XML, so after looking for some time I found a neat little library called “Simple”. It is available from http://simple.sourceforge.net/ . It has an easy to learn Java annotation based framework which allows to generate XML is pretty easy way. An example is like this

This class when converted to XML gives us something like this

Note: the formatting was slightly modified to make the XML fit

Once we are done with the XML conversion we create a HTTP request to the web service and send the XML. Here is how we send the request:

Here are the steps in plain English:

1. We need to create a request that is of type POST. So create a instance of class HttpPost
2. We also need to set header value “SOAPAction”. This will tell the web service what method we are calling. This can be inferred by testing your webservice from SOAPUI. Alternatively you can refer to the web service WSDL and get the values.
3.The response is again a XML formatted document which we can parse using the standard Java DOM APIs.

To complete this post with an example, I have put in a few classes which will give an idea on how to put a web service client in place. The method is a rather low tech method, and it is up to the designer/developer to choose.

Sample Code

Tuesday, May 1, 2012

IceCreamed finally !

Being a GalaxyS user, I was rather disappointed when I heard the news that it would not be getting Android 4 aka Ice Cream Sandwich. It had been a long time since I got any upgrade for the phone's OS. My phone was behaving rather sluggishly these past few days. Time for IceCreamSandwich is what I thought :)

I started of by using Odin, a ROM flashing software which is what I got from a lot of blogs and forum posts. But bad luck was with me all the way and I bricked my phone. Not a hard brick fortunately. I was able to get the phone into download mode, and boy am I glad it was possible. After a lot tries with various versions of Odin, I finally moved to using another opensource software called Heimdall. This is much much better than the useless Odin, which is highly unreliable. It never even started off correctly for me, but all it did is brick my phone or make is useless. Thank god, there was someone who wrote Heimdall to help me. After all this commotion with Odin, I went back to my reliable Linux and flashed my original image which I got from samfirmware.com and first got my Phone to working condition. Once I had my phone running I had my heart beating at the normal pace once again.

To move to the next version I used ClockWork Mod and then used the Cyanogen Mod9 to get to the current awesome state. Because it is a Cyanogen Mod, it has loads of developer goodies, that are not of much use to the average user, but to a developer it is like sugar candy. Performance tweaks are also available to me now, from the settings. So aweseome! Also out of all the guides this one helped the most ( https://gurde.com/2012/04/how-to-android-ics-4-0-4-on-galaxy-s-i9000/ ).

Monday, March 26, 2012

OIM11g Request Terminologies

Thursday, February 16, 2012

Hello World – OIM11g Metadata and Plug-in Framework

Introduction

This tries to explain the import/export process for registering plug-in, schedulers, request datasets, etc.

The intended audience for this experiment is experienced OIM 9.x developers who are learning OIM 11.x.

The Setup

I am using

Component	Version
Operating System	Windows 2003 R2 (Enterprise x64 Edition)
Application Server	Weblogic 10.3.5
Database Server	Oracle 11gR2
OIM	11.1.1.5.0

Plug-ins

OIM 9.x had everything available from the design console when it came to developing workflows. But with OIM 11g a plug-in framework has been introduced. In OIM 9.x we had entity adapters to do this job, but now this plug-in framework is to supersede that. This plug-in framework allows us to register items that would be called at different stages of the processes like creation of user or an update of user. In OIM 9.x we did these operations using entity adapters, but now these adapters have been replaced by OIM plug-ins.

Meta Data

OIM 11g introduces a concept of metadata repository for components like schedulers, plug-ins, etc. A component like a scheduler would have a name, a set of parameters that it needs. All this is defined in XML (metadata) and needs to be uploaded into OIM’s metadata repository.

Creating a Plug-in

This has two steps:

1. Register your plug-in class

2. Upload metadata for this plug-in

Plug-in Structure

A plug-in composed of the following elements

1. plugin.xml

2. lib/plugin-name.jar

3. resources/something.properties

The resources folder is optional and is not always required.

Registration

OIM can hold the plug-in code in the database or on file system. The default plug-in folder is the plugins folder inside server. To load data into the OIM database, we have some script provided by OIM that will help us load those items.

How to Create a Plug-in

Creating plug-in may look cumbersome when you start off, but after a while it will be like second nature to you. Here are the steps:

1. Create the plug-in class

This is the most important step. After all you want to do something with this plug-in. The plug-in class needs to extend a plugin-point class. Some of the classes that you can extend are

Interface Class

oracle.iam.platform.kernel.spi.ActionHandler

oracle.iam.platform.kernel.spi.PreProcessHandler

oracle.iam.platform.kernel.spi.PostProcessHandler

oracle.iam.platform.kernel.spi.AuditHandler

oracle.iam.platform.kernel.spi.CancelledHandler

oracle.iam.platform.kernel.spi.CompensateHandler

oracle.iam.platform.kernel.spi.InvalidHandler

oracle.iam.platform.kernel.spi.ValidationHandler

oracle.iam.platform.kernel.spi.EntityPreviewer

oracle.iam.platform.kernel.spi.FailedHandler

oracle.iam.platform.kernel.spi.VetoHandler

Each of these classes will have some methods that are specific to the area where it will be used. For example, if it is a PreProcessHandler, you need to implement execute(), compensate() and cancel() method.

2. Create the plugin.xml

The plug-in xml is fairly same for all the above cases. It is structured as:

<plugin

pluginclass="oim.helloworld.plugin.user.CreateUserPreHandler"

version="1.0" name="CreateUserPreHandler" />

</plugins>

</oimplugins>

The pluginpoint will vary based on what item you are developing the plugin for.

3. Creating the jar

Code created for the plug-in needs to be bundled in a jar file.

4. Create the zip

OIM expects a zip file for the registration process. The structure is same as mentioned before. Inside the zip you should have plugin.xml and the lib folder always. The lib folder will contain the jar file

5. Use the registration script

Now that we have the zip file ready with us, navigate to the plugin_utility folder. This folder is located inside the OIM_HOME/server folder.

Edit the ant.properties file to set wls.home and oim.home. Example:

Run the following command

ant –f pluginregistration.xml register

The script will ask for oim username and password and the weblogic url which would be something like t3://localhost:14000. After this it will ask for the path of the zip bundle. While entering the path use forward slashes instead of backslashes if you are on windows. Example:

C:/tmp/plugin-helloworld.zip

Being Conventional

When I started developing plug-ins it took me at least a zillion tries before I got it working, but that surely does not mean that you would take the same time. Below are some items that I keep in mind while creating plug-ins:

1. One Plug-in Only

The plug-in xml allows us to put in multiple items together as a single XML. However I always register them as separate items. So if I have two items to register, I do them as two separate bundles.

2. Plug-in XML Naming convention

While creating the plug-in we have to give a name to the plug-in (name attribute in the plug-in tag). The name of the plug-in is always the class name of the plug-in class. Example:

<plugin pluginclass="oim.helloworld.plugin.user.CreateUserPreHandler"

version="1.0" name="CreateUserPreHandler" />

3. Plug-in JAR Naming convention

All the code that we create for a plug-in goes into the lib folder. In all the plug-ins, the name of the jar file is the name of the plugin zip. Example:

If these are the plug-in contents:

lib/helloworld.jar

plugin.xml

The zip bundle to be used for registering this plug-in would be helloworld.zip

4. XML Tags

Do have the XML header tag, something similar to

<?xml version="1.0" encoding="UTF-8"?>

as the first line of your XML.

I use these rules when I create any plug-in, but these are not hard rules. You may have your own set of conventions, which may be quite different.

Creating MetaData

Now that we know how to create a plug-in and register it, we have to tell OIM how to use this plug-in. This process is again a XML based representation of the details. (Why do they have so many XMLs!)

MetaData Structure

The metadata is composed of one or more XML files. However as opposed to the plug-in registration, importing of these XMLs does not always work as expected if we import them at the wrong location. We will see more on this later.

Registration

Registration of metadata is similar to the plug-in registration process. Here are the steps:

1. Build the metadata XML

The metadata xml is specific to the item that is being registered. Example:

PreProcess Event Handler Metadata XML

Scheduler Metadata XML

Always use the same name as defined in the plug-in.

2. Configure weblogic.properties

This file is located in the server/bin directory. This contains information on where to look for metadata files. Configure the following items:

a. wls_servername

wls_servername is the weblogic servername where OIM is running. Example: oim_server1

b. application_name

application_name is the metadata that we need to import to. Set this to OIMMetadata

c. metadata_from_loc

metadata_from_loc is the directory from where it will pick up the metadata files

3. Put your metadata files in the import folder

The files that you want to import must be put in the metadata_from_loc folder which we configured in the last step.

However the xml needs to structured in a specific folder structure.

For schedulers, the structure should be something like /db/{SchedulerName}.xml

For plug-ins, the structure should be something like /metadata/abccorp/user/create/{PluginName}/EventHandlers.xml

For details on this refer the relevant documentation section in OIM.

4. Run the weblogicImportMetadata.sh or weblogicImportMetadata.bat (Linux/Windows)

During this process you will see a Transfer complete message in the OIM logs like this:

5. Run the PurgeCache.sh or PurgeCache.bat

6. Some items work directly after this like schedulers, but some require the OIM server to be restarted like pre-process plug-ins. Restart the server appropriately after this.

Being Conventional

The plug-in registration was almost painless, compared to the metadata registration. It took me more time to get this working, compared to the plugin registration process. Along the way, I made some rules to create metadata xmls and importing them. These conventions are:

1. One MetaData XML Import

If I have multiple XMLs to import, I separate them into separate batches. One XML was used in one import.

2. Directory naming convention

If I am creating a pre-process/post-process handler or related plug-in I used the following structure

/metadata/{projectORcompanyname}/{entity:user or resource,etc}/{operation}/{plugin-name}/EventHandlers.xml

Example: For a preprocess handler for create operation on resource MyApp it would be

/metadata/projectx/myapp/create/CreatePreHandler/EventHandlers.xml

Here when I registered the plug-in the plug-in name was CreatePreHandler

Dumping the MetaData

All the metadata XMLs that we imported can be easily retrieved back from OIM. We can utilize the weblogic scripting tool or wlst to export all the metadata xmls.

Here is how we can export all the metadata that OIM has: