Category Archives: AWS

by

What EC2 instances available in London?

Seems like a simple question, right?

However, it starts getting a little more intricate when you look a the details. Let’s say I want an m3.medium. Here’s my goto info page:

https://www.ec2instances.info/?region=eu-west-2

Cool – these are available.

 

But lets say I need them for Elasticache.

OK, looking at: https://docs.aws.amazon.com/AmazonElastiCache/latest/mem-ug/CacheNodes.SupportedTypes.html

I can see a cache.m3.medium. But then, reading further on – i.e. the Supported Node Types by AWS Region section – as of the time of posting this blog post, m3 is not supported in EU (London). Nor in EU (Paris).

m4is supported in EU (London). But not in EU (Paris)!

Fortunately, we’re only looking for London.

So, the obvious choice would be an m4.medium. However, scanning back up to the top of the Supported Node Types page the smallest available is a cache.m4.large.

It’s all in the details!

 

by

ECS Container Instances

An ECS container instance is an EC2 instance that’s:

  • running Amazon’s ECS container agent and
  • has been registered into a cluster

Notes:

  • the container agent is automatically installed if you’re using an Amazon ECS-optimized AMI
  • the agent will make API calls to Amazon ECS so the instances will need an IAM role

 

For more on this see:

https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ECS_instances.html

Note, however, if you’re using Fargate this does not apply (see docs).

by

Elasticsearch Concepts

See also Elasticsearch: administration

Indexes: store all the JSON documents and are stored in a shard

Shards: a complete Lucene database

 

Indices can be stored in multiple shards.

i.e. if we have multiple nodes then shards will migrate across nodes – aka rebalancing.

 

Replicas: an exact duplicate of a shard (except designated as a Replica). Can configure as many replicas per shard as you like.

E.g. here you have 2 shards plus 2 replicas. Each contain the one index (I01).

http://www.snowcrash.eu/wp-content/uploads/2018/10/Screen-Shot-2018-10-16-at-11.05.46-AM-300x176.png 300w, http://www.snowcrash.eu/wp-content/uploads/2018/10/Screen-Shot-2018-10-16-at-11.05.46-AM-768x449.png 768w, http://www.snowcrash.eu/wp-content/uploads/2018/10/Screen-Shot-2018-10-16-at-11.05.46-AM-588x344.png 588w" sizes="(max-width: 815px) 100vw, 815px" />

Replicas are Read-only and can serve data thereby increasing scale.

https://www.elastic.co/guide/en/elasticsearch/guide/current/replica-shards.html

 

Node Roles

Can run all on a single node but makes it more efficient if you separate them out.

Data:

  • most hard working
  • contains all shards
  • don’t typically receive service queries
  • tend to be the beefiest

Client

  • gateway to the cluster
  • big increase in performance
  • handle all query requests and redirects them to the data nodes

Master

  • brains of cluster
  • maintains cluster state
  • all nodes have a copy of the state but only the master can update cluster state

Capacity Planning

Data Nodes

To test, set up a load test on one node until node is completely saturated.

E.g. 1M documents on 1 node = 4.0 seconds then probably need 4 nodes to get to 1.0 second response time.

Master Nodes

To avoid split brain scenario:  set minimum_master_nodes to (number of master nodes / 2 ) + 1]

Should have at least 3.

Client Nodes

Could exist behind a load balancer.

E.g. in summary, a setup could be:

4 data nodes, 3 master nodes, 2 client nodes – i.e. a total of 9 nodes.

Server Requirements

  • CPU: more cores the better (favour over clock speed) – i.e. better to run more processes concurrently rather than run them faster
  • RAM: 64GB for Data nodes is ideal (e.g. in AWS an i3.2xlarge – https://aws.amazon.com/ec2/instance-types/i3/ )
  • Disks: fastest disks possible. Safe to use RAID 0 for more speed though not fault tolerant but Elasticsearch has shards. Avoid using NAS as performance will drop drastically
  • Networking: keep clustering within same data centre as shard rebalancing requires fast networking
  • VMs: don’t use for data nodes in production

 

Running on AWS

Data: i3.2xlarge

Master:

Client:

See also https://www.elastic.co/guide/en/elasticsearch/plugins/master/cloud-aws-best-practices.html

and https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing

 

by

AWS: Kinesis Data Streams

KDS (Amazon Kinesis Data Streams) lets you collect streaming data, at scale, for real-time analytics. 

From https://aws.amazon.com/kinesis/data-streams/

  • within 70ms
  • scales from megabytes to terabytes per hour, and from thousands to millions of PUT records per second
  • costs: for $0.015 per hour, you can have a Kinesis data stream with 1MB/second ingest and 2MB/second egress capacity

by

Auto Scaling Group: Launch of new EC2 instance fails – Reason: You have requested more instances (x) than your current instance limit of y allows for the specified instance type

What happens when an ASG fails to create instances because you’ve reached your limit of instance types in that Region?

E.g. you’re getting this error:

Failed: Launching a new EC2 instance.

Reason: You have requested more instances (x) than your current instance limit of u allows for the specified instance type. Please visit http://aws.amazon.com/contact-us/ec2-request to request an adjustment to this limit. Launching EC2 instance failed.

 

So you put in a request to increase the limits. But what happens to the ASG – does it continue to try launching new instances – and when?

If you have multiple AZs and a single AZ is out of capacity it will try a few times then launch in another AZ.

But simply, yes, it will keep trying to launch in a Region.