Friday, July 10, 2009

Distributed Application Management - Managing from the Top Down

In the last article, I outlined the key new approaches needed for managing distributed applications and how traditional management tools fall short. This time, I will talk more about why and how to manage distributed applications from the business perspective. In other words, manage them from the top down.

Besides the toolkit approach, traditional systems management tools also focused on managing the individual components of standalone applications from the bottom up. The components that made up these applications often ran on a single computer, which may be a mainframe or a large Symmetric Multi-Processor (SMP) server, and they were accessed via dedicated terminal connections. In contrast, modern distributed business applications run on multiple standards-compliant servers along with a collection of middleware, database, network and storage technologies. These applications are more interconnected and increasingly the interconnections are dynamic which make the end-to-end environment very challenging to manage. Because of the large number of components in modern application environments and the complex dependencies amongst the components, the traditional bottom-up focused approach alone is insufficient as it is very difficult to draw conclusions on the health of an application by simply examining the health of individual components that the application is built on.

The alternate approach is to manage applications top-down. Why is a top-down approach a better solution? In short, it helps IT focus on measurements that are relevant and ensures that applications meet business needs. This approach starts with understanding end-user service level requirements, which may include a service level target, the hours for enforcement, and the key business process flows that the applications must complete successfully. For example, for an e-commerce website, the service level requirement may be 99.99%, or less than 1 hour of unplanned outage in a year. The business hours may mirror the times when customers are expected to use the system, which could be 24x7. The key business operations that must be supported may include browsing product catalogs, placing products on shopping carts and checking out to complete orders.

With the end-user driven service level requirements defined, IT staff may then set up appropriate monitoring to ensure that those requirements are met. Technologies such as end-user monitoring may be employed to measure application end user experience, which eliminate any guesswork to gauge whether the applications are working. In the example above, the IT staff might want to define synthetic transactions using a specially designed tool that is appropriate for the e-commerce application to simulate end users logging onto the website, browsing the catalog, placing items on shopping carts, and checking out. The IT staff may also define thresholds against these end-user measurements so that they can be notified proactively if application performance or availability degrades. In fact, in contrast to traditional systems management, the IT staff may rely primarily on these notifications instead of component level notifications to manage the applications, as notifications from these end-user oriented measurements provide much more accurate views on the health of the applications according to the way that end users see them.

Picture: Business Process Availability Report

Besides end user monitoring, another key enabler for the top-down approach is the mapping of business processes to the underlying components that the applications rely on. These mappings connect the end-user perspective with the underlying technology components, and enable quicker problem isolations and more accurate impact analysis when a problem is detected by end-user monitoring. The technology that facilitates this mapping is the configuration management database (CMDB). The CMDB should be the integrated foundation of any modern application management tool, as it keeps track of not only the list of components that support an application, but also the relationships amongst these components and the changes that are made to them. Not all CMDBs were created equal! Most CMDBs provide facilities for defining new and custom types of Configuration Information (CI) so you can grab CIs that are unknown to the CMDB. This is necessary but not sufficient. A CMDB must come out-of-the-box ready to collect as much about your application environment as the infrastructure they were built on. If you are expecting to have to feed your CMDB specific instructions on how to collect CIs on your business critical applications, you should be prepared to dedicate a significant amount of your administrators’ time for defining and maintaining these CIs.

In the example above, the e-commerce processes may be mapped to the front end load balancer, the firewall, the web server, the application server, the database server, and the network switches and routers that connect everything together.