Your First CoreOS Cluster

Goal:

Create a simple 3 node CoreOS cluster

Hardware:

3 custom built, 1U server
Dual Gig NICs
Intel(R) Pentium(R) D CPU 3.00GHz
4GB's of RAM
80GB HD

PXE Magic

To start off, don't do what I did and think that getting the coreos-install script on your existing Debian install and running it will work. It won't, and it'll try to dd the CoreOS image onto the drive destroying your current setup.

There are two ways that one can proceed from here. The first option is, booting into any Linux live CD (CoreOS doesn't have one), grabbing the coreos-install script, and running it from there pointing it to the local hard drive would work. However, there's an even cooler way.

PXE booting requires yet another server, running a fancy tftp server, that accessible via the dhcp pool that your soon-to-be CoreOS servers are sitting on. You also need to control the DHCP server that's serving the addresses to the soon-to-be CoreOS servers. Luckily for me, we have a PFSense firewall setup with OpenVPN. It serves out a specific /24 network range via one of it's NICs. PFSense supports tons of additional DHCP options. We needed to add the tftp option and have it point to the pxelinux.0 file to load. Instructions for setting up an Ubuntu PXE server can be found here:
You'll also need to configure the PXE server entry for CoreOS. These official documents can help:

Writing to disk

Once you get it booting via the PXE image, the real hard work is done. You first, need to have a cloud-config.yml file. CoreOS has tons of documentation, here. Since we are going to be making a simple 3 node cluster, I'd suggest making sure the fleet and ectd sections are configured. Below is my cloud-config.yml.

	#cloud-config
	coreos:
      update:
          reboot-strategy: best-effort
      units:
          - name: etcd.service
              command: start
          - name: fleet.service
              command: start
      ectd:
          name: host01
          discovery: https://discovery.etcd.io/IDHere
          addr: IPADDR:4001
          peer-addr: IPADDR:7001
      fleet:
          public-ip: IPADDR
          metadata: host=01
	ssh_authorized_keys:
  		- ssh-rsa PUBSSHKEY
	hostname: host01

Now run the coreos-install script, pointing it at your drive (/dev/sda) in my case, specifiy the channel (-C alpha) and specify the cloud-config.yml (-c) file you made earlier. This will download the CoreOS image and copy it directly to the drive creating the partitions and configuring it as needed. Once the install is completed, reboot the box. At this point, if the PXE server/boot order is configured correctly the server will boot to the harddrive launching the real install of CoreOS!

Congrats! CoreOS is installed the the disks and you've got ectd and fleet communicating! Tune in next time as we configure fleet and ectd further and launch our first high availibility docker container.