Transloai.com launched to the public on Monday and got exciting traction. Before the launch, I moved all the infrastructure from my small VPS to AWS, maintaining a microservice architecture and distributed systems, with an auto-scaling group for horizontal scaling to handle traffic spikes. One key advantage of moving to AWS was the range of cloud compute services that small cloud providers didn't offer. Part of my journey was taking a comprehensive course to really understand the platform and its services to prevent the normal bill shock that scares software engineers away from AWS and other Cloud platforms.
AWS made model deployment fast for me because I could actually load my custom ML models directly from an S3 bucket in the same region during deployment (Saving me time and data transfer costs of using an external provider; I initially got a bill shock from doing this the wrong way, and I will discuss it in another post). I used an S3 bucket with DVC for model versioning and tracking model updates.
I separated my FastAPI application from my React and Next.js application serving the web application, using an AWS S3 bucket and CloudFront to reduce the data transfer bills from reading directly from the S3 bucket.
I explored several ways to save while using AWS, which include savings plans, reserved instances, and spot instances. I actually bought a one-year savings plan for my database compute and used a fleet of spot instances for my Auto Scaling group, saving me 30% and 60% on server costs, respectively. Before deployment, I did several load tests with Grafana to understand how my system works under load and how it scales to handle increasing traffic. I will discuss this in another post. I will be exploring serverless inference on SageMaker for my transcription and translation pipeline. Stay tuned to transloai.com will soon have video and audio translation for selected languages. This feature has undergone its first test.
À la prochaine.
