Thursday, July 10, 2008

What can do I with this SARM data?

I woke up this morning finding a message in my inbox coming from an ITtoolbox subscriber asking about SARM. Wow, I thought, someone is trying to use my baby. So I replied the message over breakfast. The information may be interesting to others who want to know more about SARM, so I am re-posting it here.

>>>>>>>>>>>>>>>>>>>>

I was the original product manager for SARM. Let me provide some explainations.

Back in Siebel 6, it was easy to figure out performance problems, as Siebel client ran as a Win32 program on PC, and connected directly to the database. Each user connect via a separate database session, and all the business logic ran on the client PC.

Things got a bit more complicated with Siebel 7. With the web based interface, Siebel 7 enables organizations to be more agile, as they could revise their application more easily to reflect changing business processes without pushing the software out to thousands of users. However, the architecture also placed more demand on the mid-tier servers. In addition, a single transaction request (query, save, navigation, etc...) requiring going from web browser to web server, from web server to Siebel App Server, and from Siebel App Server to database server. Connection to database could be shared via database connection pooling. Tracing transaction from the user to the database in order to identify performance bottleneck root cause became very difficult, as it was very difficult to tell which user initiated what database request, and there was no way to figure out what the mid-tier was doing. In Siebel's own IT department, everytime a performance problem occured, the IT staff would summon a couple engineers from our product development organization to figure out the problem. This was very expensive in engineering productivity, and most customers did not have this option.

In Siebel 7.5, I asked our IT operations director what we could give his staff to make life easier. The answer was a way to see what goes on inside the Siebel app server environment. SARM was born as a result.

SARM is made up of three parts. The first is the instrumentation framework. The second is the collection of instrumentation. The third is the tool for analyzing the data. SARM instrumentations are strategically placed in various parts of the Siebel software stack.

When a transaction request enters the Siebel server layer, the first timer goes off. As the request makes it way down the stack (think of it as a call graph), additional timers go off. These instrumentation points capture timing information, CPU/memory utilization, and contextual information about that instrumentation point. Data for each instrumentation point makes up a single SARM entry.

Each SARM entry includes the identification of the instrumentation point (AreaDesc). For example, the workflow engine would be one of those instrumentation points. For workflow, the application string field also stores the name of the workflow, so that you can tell not only the workflow engine is invoked, but also the particular workflow that is being run, and the amount of time spent running that workflow. User id, business component name, view name, applet names are also stored in the entries of the respective areas.

Let's say you have a transaction request that ran for 15 seconds. You want to find out the breakdown of the time spent. Using the area description and the text string, you can find out how much time is spent at each Siebel layer, and find out the exact workflow, business service or script that is causing the problem. You would know who initiated the transaction request since the user id is recorded, and which part of the application (view and applet name) the request came from.

It took a couple releases to get SARM fully done. 7.5 was the first release. Naturally, with any version 1.0 technology, there were some short comings, and there were not too many instrumentation points to capture data. In 7.7, the technology became much more mature, with better optimization to minimize overhead, and more instumentation coverage after we took a companywide effort to ask every single development team to instrument their code with SARM. 7.8 was an application functional release so SARM in 7.7 pretty much was the same in 7.7. We made further improvements in 8.0 to give administrators more ways to fine tune SARM data collection.

In addition, one thing that had been missing in SARM was a good graphical tool to analyze all the rich information. The command line tool, which was intended to convert the binary SARM data to CSV so that people could import it into spreadsheet, just didn't cut it. The reason why we didn't come up with a graphical tool initially was resource constraint. We needed to focus our energy on making sure we could collect good SARM data first and did it in an efficient way. Otherwise, the best analytical tool in the world wouldn't help.

The lack of good graphical tool also changed in the Siebel 8 timeframe. One thing that is cool about being part of Oracle is that Oracle has a lot more people. We actually have a whole division of people focusing on building management tools. So after we became part of Oracle, we shipped a graphical tool (Siebel Diagnostic Tool) as part of Application Management Pack for Siebel. In fact, we have done more to the management tooling of Siebel in the past year than the 10+ years when Siebel was an independent company. The tool is now fully integrated with Application Management Pack for Siebel, which runs as part of Oracle Enterprise Manager 10gR4.

I wrote about SARM on my blog in May, and will probably write about it more in the coming months, so check it out for more discussion. I will also be presenting a session on SARM at this year's OpenWorld in September so stop by if you are going to attend the conference. http://appmanagementblog.blogspot.com/2008/05/demystifying-siebel-application.html

Visit this site if you want to learn more about Application Management Pack for Siebel. http://www.oracle.com/technology/products/oem/prod_focus/app_mgmt.html

16 comments:

GG said...

Thanks Chung for explaining the background and purpose of SARM.
It is my first time to hear about SARM.

I moved to Siebel development a year and half ago from a long QA career. So, I have a keen interest and opportunity to identify Siebel performance issues and fix them.

It seems like SARM may be the right tool to help me identify performance bottle necks.

I am going to search the Oracle page to know about how and where I can get it and how to use it etc possibly with some demo.
Please give me any info on how and where to get SARM, how to use it etc.

Thanks,

Genene

Chung Wu said...

SARM framework and instrumentation are part of the Siebel platform, just like server and server component statistics. You need to enable it in server and/or component settings by setting the SARM level. Check out Siebel Performance Tuning guide for the details. Siebel out of the box also comes with a command line utility for converting SARM data into CSV.

The advanced tool for analyzing SARM data is Application Management Pack for Siebel. It runs on Oracle Enterprise Manager 10gR4. You may download Enterprise Manager to try it out from Oracle Technology Network (OTN). A trial license automatically comes with software that you download from OTN. There is no need to call Oracle for trial license.

http://www.oracle.com/technology/software/products/oem/index.html

A self study training for Application Management Pack for Siebel is being released by Oracle University also. In fact, it might already be out. I need to check with my OU colleague to be sure.

Chung Wu said...

Another thing that you should do is get the free Oracle Enterprise Manager DVD. The DVD contains a bunch of information about Enterprise Manager as well.

http://www.oracle.com/webapps/dialogue/dlgpage.jsp?p_dlg_id=5657477&src=5634283&Act=41

Unknown said...

Thanks Chung, i have heard of SARM but you shed light on the matter a lot. very informative post.

GG said...

Thanks Chung for the info, I received Oracle's DVD and going through.

Genene

Chung Wu said...

Genene,

Be aware that the DVD that you got contains EM 10gR3. You will need to apply the patch installer to bring the software contained in the virtual machine image to 10gR4. You will also need to install EM agent on your Siebel box, and point the agent back to the VM image. You may download 10gR4 patch installer from OTN.

http://www.oracle.com/technology/software/products/oem/index.html

Chung

Chung Wu said...

Btw, the Oracle University self study training that I mentioned earlier is available. See this post for more details.

http://appmanagementblog.blogspot.com/2008/07/new-application-management-pack-for.html

Anonymous said...

hi Chung
I want to know indepth of analysing the SARM. Where i can find the details?

Actually, SARM files are generated, but i don't know for which operation, it got generated. Will you please shed some light into it?

Chung Wu said...

It depends on how you enabled SARM. SARM enablement is by server, component, and in 8.x, by user id. The amount of data captured by SARM is also controlled by SARM logging level. Documentation for the SARM framework is in the Siebel Performance Tuning guide. Typically, you would want to turn on SARM at least for the application object managers that support interactive user sessions. So if you use call center, turn on SARM for SCCObjMgr.

The tool that provides graphical visualization of SARM data is part of Application Management Pack for Siebel, and it is documented in Enterprise Manager documentation and online help.

Anonymous said...

Hello,
Very informative post. I am trying to analyze some data with SARM. I enabled the SARM and set the SARM Granularity to 3 at component level, but I'm unable to get the User ID & Session ID in my CSV file. Also I don't get any SARM file in the web server log dir (SWEApp\LOG).
Thank you,
Claude

Anonymous said...

Hi Chung,

Thanks for sharing this extremely useful info about your tool: Siebel Application Response Management.

We're using your tool here at Farmers Insurance and would like to know how we can create html files using SARM tools. I have set up the SARM log levels after which it even generates the sarm files. I'm able to convert the sarm files to csv files. I'd like to know how I can make an xml out of it. Please email me...

Thanks a bunch!
Naveen

Shalabhsneha Katdare said...

Hi,

Thanks for the detailed information. I have one question, Is Siebel Diagnostic Tool & Siebel Application Management Pack for EM different tools?

Is EM more advanced than diagnostic tool?

Chung Wu said...

Siebel Diagnostics is a feature of Application Management Pack for Siebel and is licensed as part of the package. The first version of Siebel Diagnostic was a standalone utility that ran separately from Enterprise Manager Grid Control in which Application Management Pack for Siebel runs on. The latest 10gR5 version of Application Management Pack for Siebel includes a fully integrated Siebel Diagnostics Tool with enhanced functionalities, so you should be using this integrated version as opposed to the standalone one.

Shalabhsneha Katdare said...

Hi,

Ok...so what role does Management Agent & Management Server play which we configure as part of Siebel 8.0 Enterprise?

The management server is carrying the standalone diagnostic tool. Is Oracle stopping Management Server enhancements?

Thanks,
Shalabh

Chung Wu said...

Development for the standalone Siebel Diagnostic Tool that ran as part of the management server stopped over 2 years ago. After Oracle acquired Siebel, we re-examined our management tooling strategy, and decided that it made more sense to base our tooling on Oracle Enterprise Manager. Projects that were already well on their way, including the original diagnostic tool, were allowed to be completed, but everything else was to be built on top of Enterprise Manager. The result of that effort was Application Management Pack for Siebel.

Overtime, many of the standalone tools would get merged into Oracle Enterprise Manager as well. In fact, we have rebuilt the Diagnostic Tool inside Enterprise Manager and made numerous enhancements to it. The old standalone Diagnostic Tool is obsolete technically. You should not be using it.

Diagnostic Tool, whether the original version or the current integrated version within Oracle Enterprise Manager, is licensed as part of Application Management Pack for Siebel. Since you are already using it, you must already have purchased the license for the Siebel Pack, so you can use the integrated version that is in Enterprise Manager.

Anonymous said...

Hi Chung,

After reading SARM it relfects to be a great tool. Can you list out some limitations or some features that are not avilable in SARM?