While I completed the first version of the new error logging system late last week, we decided to improve it a bit. It’s going to also show logs for our “heartbeats” for each container. Heartbeats are a container pinging the rest to make sure their processes haven’t hung. AWS’s ECS (EC2 Container Service) is able to make sure that if the whole container crashes, it’s restarted. But it isn’t able to tell when the app inside simply hangs instead of completely exiting with an error exit code. So the heartbeat is a GET request send to the app running in the container. They’re sent every 10 seconds, and if they don’t get a response, the ECS task (which provides instructions for booting up all the containers) is restarted.
Having the ability to see the results of these heartbeats in one spot can allow us to quickly check the health of the containers without having to completely log into the AWS management console. Any Engineering.com staff using the VAT would have the ability to see this information easily too.
I’ve definitely seen the power of functional programming and architectural patterns like Flux (of which Redux is an implementation) and I plan to use them in my other projects after this CDOT project.