In today’s world, computing is central to running and managing a business. Business applications need computing resources, and in order to provide these resources, businesses need to make a significant investment in IT infrastructure. Depending on the size of the business, this infrastructure can range from just a small number of servers to an entire data center. However, provisioning and managing computing resources is costly. There are three major costs: (1) capital costs, i.e., the cost of buying the hardware, software, and installation costs, (2) maintenance and operation costs, i.e., the cost of personnel to maintain the infrastructure as well as cost of power, cooling, and sever storage, and (3) additional costs, for example, to buy extra hardware and software to provide high availability, reliability, scalability, and security. All of these costs add up fairly quickly and can amount to a significant portion of a company’s budget. This creates a high barrier to entry for small to medium sized businesses which simply cannot afford spending much on IT infrastructure. There is a need to cut costs in order to improve profitability and lower the barrier to entry.
Cloud computing has emerged as a pervasive paradigm for hosting Internet scale applications in large computing infrastructures. Major enabling features of the cloud include elasticity of resources, pay per use, low time to market, and the perception of unlimited resources and infinite scalability. As a result, there has been a widespread migration of IT services from enterprise computing infrastructures (i.e., networked cluster of expensive servers) to cloud infrastructures (i.e., large data-centers with hundreds of thousands of commodity servers). Since one of the primary uses of the cloud is for hosting a wide range of web applications, scalable data management systems that drive these applications are a crucial technology component in the cloud
Auto scaling and Isolation Capabilities of Cloud
This is a high-level reference architecture for a web based services start-up company which hopes for a significant growth in near future. This proposed architecture is based on AWS solutions and inherently enables various enterprise capabilities to the overall solution.
The target audience for this document are the company CTO, Engineering Directors/Managers and any other authority who involves in technological purchase decisions.
Why AWS for a start-up company?
AWS Solutions elasticity nature enables the enterprise cloud capabilities to the solution.
Since growth of the company is not guaranteed and to keep the solution cost effective over the time, the solution should be easily re-sizeable to match the growth. AWS solutions readily brings these capabilities at every stage of the solution to the desk. AWS components are elastic in nature, that is they can auto re-size to a given load at run time. Say, using EC2 you can have more servers added to the solution if the load on servers increases and vice-versa. AWS pay-as-you pricing model keeps investments as well elastic. That is when you have more load, you use more resources then you pay for added resources and vice-versa. Choosing AWS enables the elasticity to the overall solution for any loads over any time.
AWS solution is a Scalability booster for handling the unpredicted traffic loads.
It’s very often heard that upcoming web based solutions suffer performance during peak loads. This is a result of resources not being scalable to meet the unexpected increase in load at real time. AWS auto scale provisioning of VM instances helps you add more web servers readily as needed to serve the increased loads. They can be taken offline or removed when the load returns to normal as needed. This capability boosts the scalability of overall solution to the ever changing traffic loads.
Easy manageability of the solution deployments and maintenance.
Being a start-up and web based solution, company may often need to deploy changes to the system. They would require a consistent close of production environment at every level of their Development, QA testing, Staging and Prod. Yet again the on-demand provisioning of resources in AWS improves turnaround time to deploy and test the changes at every stage to ensure stable release of product in the Production. This minimizes the hardware and maintenance costs involved at every stage of development cycle.
AWS Cloud Architecture for a Web Based start-up company:
A Web Application is always handle dynamic request such as user register and static request such as a png format image. Dynamic requests would be past through the web server (apache) and be processed by the app server (PHP).
- Network Tier:
- Manages external/internal network configurations and security via AWS Route 53, ELB, Multi-Availability Zones, Elastic IPs, Security Groups, etc.
- Responsible for ensuring access to the web site from around the world and load balancing them at every required levels say at Web server and App server.
- Web Server Tier:
- Manages Web Servers via AWS EC2, AMI images, etc.
- Responsible for configuring and maintaining web server instances to handle the web requests.
- App Server Tier:
- Manages App Servers via AWS EC2, AMI images, etc.,
- Responsible for configuring and maintaining App server instances to handle the application.
- Database Tier:
- Manages Database Servers and Data via AWS RDS, IOPS Volumes, etc.
- Responsible for configuring and maintaining the backend database access, security,
performance and availability.
Multiple EBS backed EC2s :
Based on the application requirement for memory, compute power and other resources, we could decide the type of EC2 instance (small, micro, medium, large, etc). Having benchmark metrics could be helpful deciding on this.
EC2 instances could be spread in multiple availability zones and load balanced. Multiple availability zones provide high assurance of availability if any of the specific zone is down. Each of the EC2 instances could have EBS block that would have all the code and mounted automatically during instance restart.
The load balancer primarily distributes traffic to the backend EC2 instances. We could have https certificate installed on the LB for securing data over the network. The load balancer will have elastic IP address that is mapped to DNS (Domain Name System), amazon route 53 in this case.
Route 53 is the the Domain Name service for routing requests to the Load Balancer IP.
RDS (Relational Database Service):
The type of RDS server can be chosen based on application requirements. RDS is fast, secure, inexpensive and scalable. Multi-AZ instance can be provisioned that would have RDS synchronously replicate data to an instance in a different availability zone. Read replicas can also be provisioned to offload read traffic from the primary DB instance. Automated backups and snapshots can be provisioned for disaster recovery. RDS allows data encryption at rest as well as IAM (Identity and Access Management).
Cognito is used to manage user identities & sync user specific data across multiple devices. One can add user sign up and sign in to the mobile app as well as integration with social identity providers such as Facebook, Twitter, or Amazon. It is secure and scalable.
Kinesis is used to process large streams of data in real time. On one hand, the app on the EC2 instances pushes following data onto the kinesis:
- App logs – these could be user actions, logins, registrations and other behavioral data.
- Error logs captured on the app residing on the EC2.
- Data that needs to be synced up with elastic cache.
RDS would also push data on kinesis of following nature:
- Error logs
- Information that is required to be synced with redshift.
- DB snapshots.
Cognito would also use kinesis to sync up user identity data into S3.
Elastic cache could also pass data that is needed to be passed onto redshift.
S3 (Simple Storage Service):
S3 acts as a storage for following:
- Storage for all the static content for the web site – images, js, etc. These are in turn served to the end user via cloudfront.
- Storage for all the log files that need to be passed to redshift.
- DB snapshots that are backed up in S3 and can be easily used to bring up new DB instances during disaster recovery.
- It can also be used to store user identity data from cognito.
CloudFront is the AWS CDN (content delivery network) that is used to serve both dynamic as well as static content to the client devices.
Redshift is a petabyte scale data warehouse solution. It could be used for generating reports and analytics. The kinesis would basically sync up the MySQL data into Redshift.
Glacier is used for archiving data. The data from the RDS that is past the threshold can be passed onto kinesis to archive on Glacier.
Elastic search basically indexes data that needs to be retrieved quickly and at scale, typical use case being type ahead search functionality. The app. Residing on EC2 instances would index data onto elastic search via Kinesis and then retrieve search queries directly calling the search APIs.
ECS and Cloud Formation(Optional):
ECS or EC2 container service lets you run and manage docker containers on EC2 cluster.
Docker containers wrap up a piece of software in a complete filesystem that contains everything it needs to run: code, runtime, system tools, system libraries – anything you can install on a server. This guarantees that it will always run the same, regardless of the environment it is running in.
Cloud formation can be used to provision, configure and also auto scale the infrastructure. Continuous integration and orchestration can also be incorporated using the same.
End User Facing:
- No Downtime of the solution/web site.
- Faster web page responses.
- Security of User Data
- Faster Server provisioning
- Automated Distribution of Load
- Simplified Storage Administration
- Simplified Database Management
With combination of above services of AWS, we could build a manageable, secure, scalable, high performance, efficient, elastic, highly
available, fault tolerant and recoverable architecture for the company.