AWS EC2 Instance Size for Geth

[ryandotsmith]

TLDR:i3.4xlarge

Cost	$911.04 p/m, $625.61 p/m with 1 year reservation
CPU	16 vCPUs
Memory	122 GiB
NVMe	2 x 1.9 TB
Network	10 Gbps

EBS vs. NVMe

NVMe is faster than EBS. However, if you restart you instance, the data is gone. This is not good. Amazon does a really great job of keeping instances online these days. I have had instances that have been happily running for over a year. One hybrid strategy is to locate geth's data directory to the NVMe drives and periodically rsync your NVMe directory to an EBS volume. This is useful for quickly booting new nodes as well.

NVMe LVM Raid Config

lslbk # find drives to add (eg /dev/nvmeXn1)
pvcreate /dev/nvme1n1 
pvcreate /dev/nvme2n1 
pvcreate /dev/nvme3n1 
pvcreate /dev/nvme4n1
vgcreate vg1 /dev/nvme1n1 /dev/nvme2n1 /dev/nvme3n1 /dev/nvme4n1
lvcreate --type=raid0 -l100%FREE -n rd1 vg1
mkfs.ext4 /dev/vg1/rd1
mkdir /storage #customize your mount point
echo '/dev/vg1/rd1 /storage ext4 defaults 0 0' >> /etc/fstab

LevelDB / Ancient / Freezer

Geth keeps its hot data in a leveldb, although there is now an option for Pebble --which is probably superior although I haven't had time to switch my servers over to it. The leveldb will be stored in --datadir. After a certain point, blocks and receipts are moved out of the leveldb and into flat files (capped at 1.5GB and indexed by block number in a separate file). These flat files are known as ancient or freezer files. You can separate the storage of the leveldb and the freezer by using the --datadir.ancient flag. Theoretically you could offload the freezer to a cheaper, slower disk. Perhaps it's good to keep the freezer in EBS and the leveldb in NVMe.

[ryandotsmith]

Ping me If you are booting an EC2 instance up and would like to use one of Index Supply's EBS Snapshots. With a snapshot you can have a full node up in a couple of hours. Obviously you would have to trust my snapshot ;) This might not be right for all cases, but for some it may be just what is needed.