DevOps Gestalt: Monitoring and Managing Applications on Engine Yard with AppFirst

Engine Yard is happy to announce a new partnership with AppFirst, provider of DevOps and Application Monitoring solutions for web applications. This will allow us to bring the highest level of support, operations and long-term value to our customers. Through a beautiful yet simple dashboard customers will have access to a unified view of their infrastructure, applications and business metrics. The AppFirst platform gives our customers full-stack visibility and allows them to pinpoint problems quickly as well as plan optimizations. In addition, this partnership also gives customers a unified repository for all their server metrics. AppFirst is passionate about DevOps and Big Data and Engine Yard is proud to be partnering with a company that is just as passionate as we are in these areas. Customers can expect to see benefits in diagnosing system bottlenecks, improving business metrics, increasing application performance and decreasing support response times.

This new feature is a free Add-on for all Engine Yard Cloud customers and comes with a weeks worth of data retention. Be sure to try it out now. Longer term data retention is available for an additional fee.

Why AppFirst?

We evaluated many vendors and open source projects looking to maximize the number of server metrics categories gathered from a single solution. Our search was not fruitful initially as every Open Source solution addressed only one, perhaps two of the categories and the same was true for the vendors we evaluated. When we found AppFirst we found a solution whose value goes far beyond simple monitoring and pretty graphs.

Collector & Metrics

AppFirst provides a collector which will be installed on all Engine Yard servers across all products. The collector is written in C and Assembly and utilizes < 1% of system resources so customers should not see visible impact to their systems due to the collector. The collector will allow Engine Yard’s already top notch support and operations staff to do an even better more proactive job supporting our customers. The collector allows Engine Yard to capture well over 100 health and operations metrics, including but not limited to metrics about CPU, Memory, Disk, Network on a server wide as well as per-process level. This collector also enables Engine Yard to collect system logs, Nagios checks and enhanced StatsD+ metrics. All log and metric information gathered by the collector is fully encrypted as it is transmitted over the wire back to AppFirst.

Engine Yard is committed to security. The AppFirst collector codebase has undergone security audits by a third party and all identified issues have been resolved.

AppFirst Web UI Features

Servers Workbench

The workbench allows customers to quickly view server level summaries as well as most recent logs, the most recent alert history, and the most recent running process listing.

Correlate

The correlation function allows customers to overlay time series data from multiple sources. Data sources include Application data, Server data, Nagios data and Logs data. For example, a customer can overlay CPU, Memory and Network metrics from several servers in one graph which allows for easy visualization of abnormal behaviors. If system logs are added to the correlation customers can also view logs at any given timestamp as well as all processes running across all of these systems or a specific selected server.

This amazingly powerful troubleshooting tool enables customers to visually see what is happening with all applications at any point in time on any server. For example, if a server sends an alert at 2:35am at any time of the day customers can quickly visualize the problem and drill into identifying the event’s root cause.

When customers find performance issues correlate allows them to quickly find abnormal behavior from their baseline. Overlay server graphs from multiple servers on top of each other to identify why one server is performing worse than the baseline.

BrowseBrowse offers another view of a selected resource, either a server, process, or application (group of processes). As with correlate, customers may select points in time for details at that point in time. Metrics such as CPU, Memory, Disk, Network may be toggled on and off for this view as well.

Browse allows customers to observe and compare trends in their data over time. Select resources like CPU, memory, and network traffic and then graph process behavior over a given time frame. Clicking on each spike on the graph displays additional details about the process such as the number of network connections, number of threads, and more. Customers can also compare one or more processes across the same time range or different time ranges.

Dashboards

Dashboards allow for an at a glance health check for application and business metrics. Operations style dashboard ‘widgets’ which display time series statistics about selected metrics are displayed on the dashboard. There are three categories of widgets: application, business and system.

Application widgets

Application widgets allow for quick time-series views of metrics for a specific application. In this example we see metrics aggregate summaries represented as time series graphs for PostgreSQL.

System widgets

System widgets are widgets which represent the overall health of servers where the currently selected application are running. Each server is represented as a square in a grid that when clicking on the server square reveals that server’s CPU, Memory, Disk Busy and Disk Full metrics at a glance.

StatsD widgets

There are also StatsD widgets which are focused on application metrics. The StatsD widgets specifically show collected StatsD+ metrics that submitted by customer’s applications. StatsD metric widgets look identical to application metric widgets, the difference is that they display enhanced StatsD+ metrics submitted by customer applications.

What are ‘StatsD’ metrics? Time series data (timers,counters,gauges). For more information on AppFirst’s enhanced StatsD+ visit the appfirst website for more information.

How are these new features accessed?

For Engine Yard Cloud customers:

1. Go to [https://cloud.engineyard.com/addons/appfirst](https://cloud.engineyard.com/addons/appfirst)
2. Click on the “Set it up” tab and follow the instructions.

For more information, please read our AppFirst Add-on documentation.