Inference optimization
Cutting latency and cost out of model serving — batching, KV-cache reuse, quantization, and semantic caching to shave every millisecond off time-to-first-token.
inference engine — online
Satyam
Chatrola
AI Systems Engineer · LLM inference & serving
response
> whoami
focus
What I do
Serving infrastructure
Packaging models as reliable, reproducible low-latency endpoints with vLLM, Truss, and Docker — shipped through CI/CD and built to stay up under load.
Systems from first principles
Understanding the stack end to end: autograd, transformer internals, and language models built from scratch — so optimization is grounded, not guessed.
Scale & reliability
Designing planet-scale AI systems with software-engineering rigor and MLOps discipline — efficient, observable, and robust in production.
signal
I turn models into
fast, dependable endpoints.
0×inference speedup
0F1 · 750+ profiles
0M+SKUs searched
+0%click-through lift
I'm an AI Systems Engineer and MS Computer Science student at NYU, focused on the part of machine learning that decides whether it's actually useful: how it's served. My work sits where software engineering, systems, and ML meet — making inference fast, cheap, and reliable at scale.
- Delivered up to 7× inference speedups through quantization, and shipped low-latency inference microservices into production.
- Architected a real-time facial recognition & authentication system reaching 0.96 F1 across 750+ profiles on live video — new identities enroll with no retraining.
- Built hyper-personalized search over 5M+ SKUs and recommenders that lifted conversion 7.2% and click-through 34%.
AWS-Certified ML Engineer1st Runner-Up · Qualcomm × Microsoft Edge AI HackathonMS Computer Science · NYU
work
Selected projects
RAG · latency
research-mate
Contextual-RAG research assistant on AWS. Compresses embeddings with binary quantization and keeps the vector store in memory to minimize retrieval latency at scale.
serving
vllm-demo
Reproducible LLM serving with vLLM, packaged in Docker images for consistent, portable high-throughput inference.
caching
semantic_cache_and_RAG
Semantic cache, vector store, and RAG built on FAISS — cutting repeat-query latency and cost by serving from cache when meaning matches.
deployment
truss-demo
Model packaging and deployment with Truss — turning a trained model into a served endpoint with minimal glue.
from scratch
my_LLM
A language model built from scratch in PyTorch, iteratively evolved from foundations toward current industry-standard architecture.
foundations
my_micrograd
Reimplementing Karpathy's micrograd — a tiny autograd engine — to understand backprop from first principles.
research
Published · peer-reviewed
Benchmarking Fine-Tuned Transformers, LLMs and LSTM Networks for Automated Essay Scoring
A comparative study of how transformer fine-tuning, large language models, and recurrent baselines trade off accuracy against the cost of scoring at scale — with the fine-tuned approach exceeding prior benchmarks by 5.7%.