I have been playing with Funsize a lot recently. One of the goals was iterating faster:
commit, push to github
let Travis deploy to AWS Elastic Beanstalk
I have hit some challenges with both Travis and Elastic Beanstalk.
The first challenge was to run the integration (actually end-to-end) tests in the same environment. Funsize uses Docker for both hacking and production environments. Unfortunately it's not possible to create Docker images as a part of Travis job (there is a option to run jobs inside Docker, but this is a different beast).
A simple bash script works around this problem. It starts all services we need in background and runs the end-to-end tests. The end-to-end test asks Funsize to generate several partial MAR files, downloads identical files from Mozilla's FTP server and compares their content skipping the cryptographic signature (Funsize does not sign MAR files).
The next challenge was deploying the code. We use Elastic Beanstalk as convenient way to run simple services. There is a plan to use something else for Funsize, but at the moment it's Elastic Beanstalk.
Travis has support for Elastic Beanstalk, but it's still experimental and at the moment of writing this post there were no documentation on the official website. The .travis.yml file looks straight forward and worked fine. The only minor issue I hit was long commit message.
# .travis.yml snippet deploy: - provider: elasticbeanstalk app: funsize # Elastic Beanstalk app name env: funsize-dev-rail # Elastic Beanstalk env name bucket_name: elasticbeanstalk-us-east-1-314336048151 # S3 bucket used by Elastic Beanstalk region: us-east-1 access_key_id: secure: "encrypted key id" secret_access_key: secure: "encrypted key" on: repo: rail/build-funsize # Deploy only using my user repo for now all_branches: true # deploy only if particular jobs in the job matrix passes, not any condition: $FUNSIZE_S3_UPLOAD_BUCKET = mozilla-releng-funsize-travis
Having the credentials in a public version control system, even if they are encrypted, makes me very nervous. To minimize possible harm in case something goes wrong I created a separate user in AWS IAM. I couldn't find any decent docs on what permissions a user should have to be able to deploy something to Elastic Beanstalk. It took a while to figure out the this minimal set of permissions. Even with these permissions the user looks very powerful with limited access to EB, S3, EC2, Auto Scaling and CloudFormation.
Conclusion: using Travis for Elastic Beanstalk deployments is quite stable and easy to use (after the initial setup) unless you are paranoid about some encrypted credentials being available on github.