Developer Webinar - Flex Plan Takeaways

Last week we hosted a live developer webinar covering the ins, outs and awesomeness of the new Engine Yard Cloud Flex plan. If you missed it, the recording is available on our site. Jon Crosby and Ezra Zygmuntowicz did the bulk of the work, running through live demos and highlighting the important features.

I worked on the back end, collecting and answering questions from attendees. There were a lot of great questions, but considering that we had twenty minutes and a seemingly unlimited supply of questions, we noted and deferred a bunch of them for later.

There were lots of questions about migrations: ‘I’m an existing Engine Yard customer, and it sounds like the Flex plan would be a great option for some of my existing projects. Can I move?’

The answer is a resounding yes: we’d love to have you and all your projects, and want to work with you to find the best solution amongst our products. If that means a move to the Cloud – whether on our Solo plan or our Flex plan – then we’ll make that happen.

That said, I have to make the following note: the Flex plan is still in beta. There are still a few features in development, and you definitely want to have a conversation with a member of our Sales Engineering team to be sure you won’t be missing any ‘must have’s’ for your particular app. They’re generally smaller, non-mission critical things, and for most people, the beta should work out great, but I’m just putting all our cards on the table.

Folks also had a lot of questions about scaling: ‘Is it possible to use one app server and one DB server and then scale up and down as needed?’

Again, a resounding yes. You can scale any cluster up or down to the minimum of one app server and one DB server. The Engine Yard Cloud is intended to be super _flex_ible, and to grow along with you and your business. (Although I should note that Premium Support eligibility requires dual app servers and a database replica.)

As for the rest of the questions, later has arrived :) I’ve taken the most frequently asked questions and attached them to this blog post. If you have others, feel free to comment or shoot us an email and we’ll get back to you with all the info you need.

Questions and Answers are after the jump – and thanks to everyone who participated!

Can you deploy branches?
Yes! When you click Deploy in the user interface you can specify which branch to deploy from, for each application.

Can you describe the Deploy Hooks in greater detail?
There are a set of deploy hooks that are run if they exist during the application deployment. You can find more information about this in the Deploy Hooks section. The available hooks are after_restart.rb before_migrate.rb before_restart.rb and before_symlink.rb.

What does the GitHub post-hook do exactly? I assume it doesn’t actually automatically deploy all the GitHub commits.
The GitHub post-receive hook will only trigger a deploy if two conditions are met. First, you need to include a tag in the commit message like [deploy production] where production is the environment name. Second, the branch set to deploy on the specified environment must contain the commit you are pushing. Finally, we will deploy the exact commit with the tag (not the HEAD of the branch) if the conditions are met.

Can we use EC2 reserved instances for a cost savings?
Unfortunately, not quite yet. Amazon does not currently split out reserved instance hours in the usage reports we use to calculate the bills. That means that we cannot bill different prices for reserved instances just yet. We are working with Amazon to hopefully provide reserved instances in the future.

Is it possible to have multiple virtual hosts with SSL in the same environment?
No, unfortunately it is not possible to attach multiple IP addresses to an instance in EC2 at this time. SSL requires a unique IP address for each domain.

We probably won’t move to Git for a little while. Can we still deploy from a Subversion repository?
You are absolutely able to use subversion, but by using Capistrano to deploy your application - the web based deploy does not currently support subversion. Here is more information on how to set up Subversion.

Are multiple SSH keys for one deployment user for multiple developers supported?
This feature is on our roadmap, and we’re hard at work on it. Once this is implemented, you will be able to add new keys to running instances from the SSH Keys page.

How often do the availability zones actually go down? (if ever)
Amazon has a Service Level Agreement for EC2. They guarantee 99.95% uptime across each region. A region is only considered down when more than one availability zone in that region is unavailable. In reality, we have yet to see an entire availability zone go completely down.

The most severe outage we have seen was recently when lightning struck one of the EC2 data centers and a few racks lost power. Everything was restored within 24 hours during which time it was simple to recover by rebooting affected instances in a different availability zone.

Can you clone an environment into one Solo instance?
You can not currently automatically clone an environment into a new environment with a different sized cluster. Once the clone is complete it is easy to shutdown the cluster and then boot up an instance. We hope that due to the on demand availability of EC2, customers will be able to use an exact replica of production for staging and testing, and only have it running when necessary.

Can two environments share one database instance?
Two environments sharing one database server is not currently automatable. You can however set this up manually by pointing one set of application servers at a different database. You will also need to open the database port in the security group for the other environment’s security group.

Is the Flex plan production ready?
The Flex plan for Engine Yard Cloud is currently in beta. We encourage projects which can operate within the constraints to set their application up on the Cloud and make sure that all is working well in a staging environment before attempting any migration to production.

So you can’t send email from a Cloud instance?
Amazon’s block of public IP addresses are on many spam block lists, and this makes sending email directly from EC2 a non-starter. You can relay your mail through any third party service. We recommend using AuthSMTP for large volume external mail and have instructions on how to set up mail for AuthSMTP.

Does the Engine Yard Cloud environment provide any automatic caching, assuming the necessary HTTP cache headers are set?
Engine Yard does not currently provide a caching layer. So far, we have not been satisfied with many of the commonly used caching layers. We are looking into some nginx based caching layers that are looking promising. We do not want to release anything unless it is rock solid and up to our level of quality.

Do you still do occasional full backups, or is everything just a differential backup after the first one?
There are two things that we backup. First, we do frequent SQL database dumps. These take full backups each time it backs up and stores the backup in S3. Second, we use Amazon’s provided EBS snapshot capability which performs block level backups of your EBS volumes to S3. These snapshots are incremental after the first to save storage costs, but this is transparent through the API. You simply request to create a volume from a particular snapshot and you get a block level copy of that snapshot.

Do you have instances for memcached?
Memcached is installed and running by default on all application instances. We do not provide any automatic configuration for your application, but it should be easy to get going with the local memcached server.

Is Capistrano still supported?
Yes, you are welcome to continue to use Capistrano to deploy your application. The web based deploy uses the same directory layout as Capistrano so they should operate well together, but we recommend using one or the other and sticking with it to avoid any conflicts. If you are using Capistrano there is automated functionality that you will be missing out on, so we definitely recommend taking a look before making this decision.

In regards to the database dumps web interface, will we be allowed to download the zip of our db?
Yes; you can get a tar.gz of your database SQL dumps from the web interface.

Can the deployment do a rolling type deploy so the system is never down?
This is not currently supported. We would like to do this by introducing custom deployment strategies for chef-deploy.

Can we spread clusters over availability zones?
This is not currently supported.

What support are you going to have for Sinatra?
We already support any rack based frameworks natively. Simply select rack (and the version) as your application type, and specify a standard config.ru. We will do the rest :)

Can you migrate from the Solo plan to the Flex plan without rebooting?
This is not currently supported, and we are working to streamline this process to minimize downtime. Doing a complete migration without any reboot is complex, and while we want to provide this functionality, it is not on the near term roadmap.

The current best practice for upgrading from the Solo plan to a Flex plan cluster would be to create a new environment and boot a cluster from the most recent snapshots. Once the cluster is up and good, you can move the IP from the old instance to the cluster.

Do you support multiple virtual hosts separated by comma or a space?
This is on the near term roadmap. Nginx supports separating vhosts with spaces and we intend to support that as well.

Are there also tools for restoring/managing backups (partial or bare metal)?
There are two types of backups that we take. First, we take EBS volume snapshots at the interval you specify. You can boot from these snapshots when you create a new instance. Second, we backup your database using SQL dumps. These are available for download directly from the Cloud dashboard.

Are there any plans to create some sort of configurable threshold (lower and upper) that will auto-scale the number of instances in each environment based on load?
Yes, auto-scaling is on the near-term roadmap.

What about setting up wildcard VHosts? Can I just put in an asterisk for domain to accomplish this?
In both Apache and Nginx, an underscore (_) is the wildcard vhost. If you use this as your vhost it will be a wildcard.

How about Nginx+Thin?
Right now, the only stacks that we support are Nginx+Mongrel and Nginx+Passenger. We are working on expanding this list of stacks, first to include JRuby, but we have no near-term plan to support Thin.

And that’s it for now! As you can tell, we’ve got a lot in the works. The Engine Yard Cloud Flex plan is a great and solid option right now, even in beta – but things are only getting better. Stick around!