Alright, that’s it for the DynamoDB puns, I swear. 🙂
I feel like on Friday I finally had a breakthrough for our project while investigating technologies to use for the project. My team mate built a prototype of our back end and the front end we will need to create in a few months, with Node.js and DynamoDB for the back end, so that we could demo it after the weekend. Meanwhile, my job for the day was to take one more stab at finding an open source alternative to DynamoDB. I looked further into using MongoDB.
MongoDB Atlas is limited to 10 TB if you use it as a service. This isn’t good enough, but luckily, you can use Amazon Elastic Cloud Compute (EC2) to provision your own instances to run your web apps, and this includes choosing instances that have up to 48 TB of cheap, magnetic storage per instance. Eureka! We were so obsessed with looking at X-as-a-Service that we forgot about the DIY options. We can still use Docker to make things easier for the client, reducing maintenance and tech knowledge needed to migrate the system we build in the future. So this may be an amazing sweet spot: An EC2 instance running our Node.js back end in a Docker container, and another EC2 instance with tons of storage running MongoDB in a Docker container, to accept and store the data. The latter can scale to multiple EC2 instances all running MongoDB (sharded) to store more and more data as Engineering.com grows. And it’s all open source. 🙂
After this discovery, I created a small prototype of a Node.js and MongoDB back end, running in two separate Docker containers locally, to get a sense of how it would fit together. Docker ended up being very intuitive. I think it’s going to be a popular tool over the next few years. My prototype worked. My team mate’s prototype worked. Mission accomplished!