File Management & Async Processing Platform
Built a scalable file and folder management system with hierarchical organization and secure file sharing, achieving API latencies of <200ms for share/unshare operations and <100ms for file creation. Designed a fully asynchronous file processing pipeline for document-to-PDF conversion and thumbnail generation.
Tech Stack
GogRPCPostgreSQLGCPPub/SubA highly scalable file and folder management API, built to handle complex hierarchies and secure sharing at scale.
Key Achievements
- Hierarchical Storage: Implemented a recursive PostgreSQL schema using closure tables / ltree to support infinitely nested folders efficiently.
- Asynchronous Processing: Designed an event-driven architecture using Google Cloud Pub/Sub to decouple file uploads from heavy tasks.
- Background Workers: Built dedicated Go workers to listen for Pub/Sub events and perform document-to-PDF conversions and thumbnail generation.
- Performance: Maintained strict API latencies (<100ms for writes, <200ms for complex tree operations).