Showing posts with label S3. Show all posts
Showing posts with label S3. Show all posts

Friday, 5 May 2023

Cloud platforms


Cloud Platforms in Industry


An overview of a few prominent cloud computing platforms and a brief description of the types of service they offer are shown in the following table.


1 Amazon web services

The services offered by AWS are classified into 4 categories.

    1. Compute services

    2. Storage services

    3. Communication services

    4. Additional services


Amazon compute services

Most important compute services are

    1. EC2 (Elastic Cloud Computing)

    2. AMI (Amazon Machine Instances/Images)

Additional compute services are

    1. Amazon Elastic Beanstalk

    2. Amazon cloudFormation

        3. Amazon MapReduce


The following figure shows the list of services offered under each category.

1.1 Compute services

  • The fundamental service in this space is Amazon EC2,
  • Amazon EC2 allows deploying servers in the form of virtual machines.
  • Each virtual machine is created as instances of a specific image.
  • Images come with a preinstalled operating system and a software stack.
  • The instances can be configured for memory, number of processors, and storage.


1.1.1 Amazon machine images

  • (AMIs) are templates from which it is possible to create a virtual machine.
  • They are stored in Amazon S3 and identified by a unique identifier in the form of ami-xxxxxx.
  • An AMI contains a physical file system layout with a predefined operating system installed.
  • AMIs are either created from scratch or “bundled” from existing EC2 instances. 
  • Once an AMI is created, it is stored in an S3 bucket and the user can decide whether to make it available to other users or keep it for personal use.

1.1.2 EC2 instances

EC2 instances represent virtual machines. They are created using AMI as templates.

There are six major categories of EC2 instances available as given below:

  1. Standard instances. This class offers a set of configurations that are suitable for most applications.
  2. Micro instances. This class is suitable for those applications that consume a limited amount of computing power and memory. Micro instances can be used for small Web applications with limited traffic..
  3. High-memory instances. This class targets applications that need to process huge workloads and require large amounts of memory.
  4. High-CPU instances. This class targets compute-intensive applications.
  5. Cluster Compute instances. This class is used to provide virtual cluster services. Instances in this category are characterized by high CPU compute power and large memory.
  6. Cluster GPU instances. This class provides instances featuring graphic processing units (GPUs) and high compute power, large memory, and extremely high I/O and network performance.


1.1.3 EC2 environment

  • EC2 instances are executed within a virtual environment
  • By default, instances are created with an internal IP address.
  • Together with an external IP, EC2 instances are also given a domain name.
  • Instance owners can partially control where to deploy instances.
  • Amazon EC2 controls the accessibility of a virtual instance with basic firewall configuration.


1.1.4 Advanced compute services

Amazon Web Services provide more sophisticated services in addition to EC2 and AMI.


AWS CloudFormation

  • CloudFormation introduces the concepts of templates, which are JSON formatted text files.
  • CloudFormation allows easy and explicit linking EC2 instances together and introducing dependencies among them. 
  • Templates provide a simple and declarative way to build complex systems
  • Allows to integrate EC2 instances with other AWS services such as S3, SimpleDB, SQS, SNS, Route 53, Elastic Beanstalk, and others.

AWS elastic beanstalk 

  • AWS Elastic Beanstalk constitutes a simple and easy way to package applications and deploy them on the AWS Cloud.
  • This service is available only for Web applications developed with the Java/Tomcat technology stack. 
  • Developers can conveniently package their Web application into a WAR file and use Beanstalk to automate its deployment on the AWS Cloud.


Amazon elastic MapReduce

  • It utilizes Hadoop as the MapReduce engine, 
  • deployed on a virtual infrastructure composed of EC2 instances, 
  • and uses Amazon S3 for storage needs.
  • MapReduce introduces elasticity and allows users to dynamically size the Hadoop cluster
  • EC2 instances to compose the cluster (Small, High-Memory, High-CPU, Cluster Compute, and Cluster GPU).


1.2. Amazon storage services

  • The core storage service by Amazon is Simple Storage Service (S3). 
  • This is a distributed object store that allows users to store information in different formats. 
  • The core components of S3 are two: 
  • buckets - Buckets represent virtual containers in which to store objects
  • objects - objects represent the content that is actually stored. Can also store metadata about the stored content.


1.2.1 S3 key concepts

  • The storage is organized in a two-level hierarchy.
  • Stored objects cannot be manipulated like standard files
  • Once an object has been added to a bucket, its content and position is immutable
  • Content is not immediately available to users.
  • Requests will occasionally fail. 
  • Due to the large distributed infrastructure being managed, requests for object may occasionally fail.



Resource Naming

Amazon offers three different ways of addressing a bucket

  • Canonical form: http://s3.amazonaws.com/bukect_name/object_name
  • Subdomain form: http://bucket-name/s3.amzonaws.com/object_name
  • Virtual hosting form: http://bucket-name.com/object_name


Subdomain form: http://bucketname.s3.amazon.com/.

To express a bucket name in this form, the name has to do all of the following:

  • Be between 3 and 63 characters long
  • Contain only letters, numbers, periods, and dashes 
  • Start with a letter or a number. 
  • Contain at least one letter 
  • Have no fragments between periods that start with a dash or end with a dash or that are empty strings 

This form is equivalent to the previous one when it can be used, but it is the one to be preferred since it works more effectively for all the geographical locations serving resources stored in S3.


Access control and security

Amazon S3 allows controlling the access to buckets and objects by means of Access Control Policies (ACPs). 

An ACP is a set of grant permissions that are attached to a resource.

Currently, five different permissions can be used:

 READ             allows the grantee to retrieve an object and its metadata and to list                 the content of a bucket as well as getting its metadata.

 WRITE         allows the grantee to add an object to a bucket as well as modify                 and remove it. 

READ_ACP         allows the grantee to read the ACP of a resource.

WRITE_ACP         allows the grantee to modify the ACP of a resource. 

FULL_CONTROL     grants all of the preceding permissions.


Other storage services

Amazon elastic block store

    The Amazon Elastic Block Store (EBS) allows AWS users to provide EC2 instances with persistent storage in the form of volumes that can be mounted at instance startup. 

 

Amazon ElastiCache

    ElastiCache is an implementation of an elastic in-memory cache based on a cluster of EC2 instances. It provides fast data access from other EC2 instances.


Structured storage solutions

    Amazon provides structured storage services in three different forms: 

  • preconfigured EC2 AMIs
  • Amazon Relational Data Storage (RDS)
  • Amazon SimpleDB.


Amazon RDS

Two key advanced features of RDS are multi-AZ deployment and read replicas.

  1. The first option provides users with a failover infrastructure for their RDBMS solutions.
  2. The second option provides users with increased performance for applications that are heavily based on database reads.

Amazon SimpleDB 

Amazon SimpleDB is a lightweight, highly scalable, and flexible data storage solution for applications. 

SimpleDB provides support for semi-structured data, 

The model is based on the concept of domains, items, and attributes.

Provides improved performance in querying large quantities of data.

Amazon CloudFront 

  • CloudFront is an implementation of a content delivery network on top of the Amazon distributed storage infrastructure.
  • The content that can be delivered through CloudFront is static (HTTP and HTTPS) or streaming.
  • CloudFront is cheaper than S3.
  • CloudFront is designed to optimize the distribution of very popular content that is frequently downloaded.


1.3 Communication services

Amazon provides facilities to structure and facilitate the communication among existing applications.

These facilities can be organized into two major categories:

  • virtual networking 
  • messaging.

1.3.1 Virtual networking 

  • Virtual networking comprises a collection of services that allow AWS users to control the connectivity to and between compute and storage services.
  • Amazon Virtual Private Cloud (VPC) and Amazon Direct Connect provide connectivity solutions in terms of infrastructure; 
  • Route 53 facilitates connectivity in terms of naming

Amazon VPC

It provides a great degree of flexibility in creating virtual private networks within the Amazon infrastructure.

Amazon Direct Connect 

It allows AWS users to create dedicated networks between the user private network and Amazon Direct Connect locations, called ports. 

Amazon Route 53

It implements dynamic DNS services that allow AWS resources to be reached through domain names.


1.3.2 Messaging

The three different types of messaging services offered are 

  • Amazon Simple Queue Service (SQS), 
  • Amazon Simple Notification Service (SNS), 
  • Amazon Simple Email Service (SES).


Amazon SQS 

constitutes disconnected model for exchanging messages between applications by means of message queues, hosted within the AWS infrastructure. 

Amazon SNS

provides a publish-subscribe method for connecting heterogeneous applications.

Amazon SNS allows applications to be notified when new content of interest is available

Amazon SES 

provides AWS users with a scalable email service that leverages the AWS infrastructure. 

Once users are signed up for the service, they have to provide an email that SES will use to send emails on their behalf. 

To activate the service, SES will send an email to verify the given address and provide the users with the necessary information for the activation. 

Upon verification, the user is given an SES sandbox to test the service, and he can request access to the production version. Using SES, it is possible to send either SMTP-compliant emails or raw emails by specifying email headers and Multipurpose Internet Mail Extension (MIME) ty