Export my resumeGo to my notesGo to my notes

About Me

Loading models
0%

Hi there! 👋 I'm Thoai.

I work in the cloud and data platform space, mostly around Kubernetes, Kafka, Spark, and the tools that make modern data systems run smoothly.

I enjoy digging into how data actually flows through systems, how storage works under the hood (pages, blocks, execution…), and how to turn a bunch of scattered services into a clean, maintainable pipeline.

Recently, I've been focusing more on Data Engineering, including:

  • designing reliable and scalable data platforms
  • technical stacks such as Iceberg, Lakehouse, Trino, Spark
  • data modeling techniques
  • building ETL/ELT pipeline
  • and so on...

I created this blog/docs site to capture what I learn, what I experiment with, and the mistakes I run into along the way. Hopefully it helps someone else, or at least helps future me.

If you're into data, distributed systems, or just want to debug a burning pipeline together, feel free to reach out.

I also keep longer writeups and experiments in my blog/docs notes.

Go to my notesGo to my notes

My Experiences

Data Platform Engineer

UDP - The CrownX - Masan GroupHo Chi Minh City, Vietnam
March 2026 - Present
  • Developed a self-driving metadata governance agent that continuously profiles datasets, enriches context with enterprise glossaries, and uses LLMs to maintain data dictionaries, stewardship, governance tags, and descriptions — covering 2,000+ tables across a 14,000+ table enterprise Lakehouse.
  • Delivered a unified Data Wiki with Next.js and FastAPI, serving 9 data projects with project-level access isolation, end-to-end lineage across Databricks and external sources, and an AI agent for natural-language data discovery, explanation, and recommendations.
  • Developed a Databricks usage analytics and post-mortem solution to analyze compute consumption across jobs, clusters, projects, and owners, helping teams investigate workload behavior, identify usage patterns, and trace abnormal or inefficient resource consumption.
  • Established a metrics-first data observability platform across Unity Catalog and ADLS, continuously monitoring dataset freshness, storage growth, usage, profiling results, and time-series anomalies for governance and data lifecycle workflows.
  • Automated unused-data lifecycle and storage governance through staged deprecation, owner notification, access revocation, archival, auditing, and rollback; archived 150+ inactive tables and reclaimed 2.5+ TB of idle storage while maintaining recoverability.
  • Implemented a reusable OAuth2 authentication bridge for Databricks, allowing applications outside Databricks Apps — including Kubernetes and containerized services — to use Databricks as an identity provider while preserving a consistent authentication model across deployment environments.
DatabricksUnity CatalogAzure Data Lake StorageFastAPINext.js

Data Engineer

ZDA - Zalopay - VNG CorporationHo Chi Minh City, Vietnam
September 2025 - February 2026
  • Designed and integrated a unified feature store based on Feast into a legacy data platform, supporting batch and real-time feature engineering, API-based online serving, and GitOps-style governance and versioning.
  • Operated feature serving at scale with 14M MAUs, around 2M streaming events/day, and sub-200 ms feature retrieval latency.
  • Built batch and streaming ETL / feature engineering pipelines using Spark and Airflow, and developed internal frameworks to automate streaming pipeline deployment and management.
  • Designed a declarative Spark-based framework for graph pipelines, providing ORM-like abstractions for defining vertices, edges, transformations, validation, and pluggable storage writers.
  • Built a large-scale User Network graph pipeline processing around 50M transactions/day to model relationships between users and support fraud investigation and risk analysis.
  • Owned and improved the Risk data and ML platform with Spark, Airflow, HDFS, and related systems, ensuring stable daily operation of 50-60 Spark applications, each processing up to 500M-1B records per run.
FeastSparkAirflowTiDBHadoopDocker

Data Platform Engineer

xPlat - FPT Smart Cloud - FPT CorporationHo Chi Minh City, Vietnam
April 2023 - August 2025

Contributed as a core member of the Data Platform team at a cloud service provider, delivering a platform-as-a-service for real-time ingestion, distributed processing, governed access, and self-service analytics to enterprise clients.

  • Built a Lakehouse platform solution with Iceberg + S3 + Spark + Trino, covering OAuth2 identity propagation, Apache Ranger fine-grained access control and masking, OpenMetadata lineage, and S3 SSE-C encryption.
  • Implemented CDC / streaming pipelines with Kafka Connect and Debezium, ingesting around 100GB/day and 5K TPS, synchronizing 500+ PostgreSQL tables to ClickHouse, Iceberg, and S3.
  • Developed a self-service Spark environment through JupyterHub and a custom Profile Manager, enabling per-profile resource isolation, secure secret injection, and automatic Spark bootstrap.
  • Created Airflow plugins integrated with Spark Operator, standardizing job submission, dependency management, runtime tracking, and real-time log streaming.
  • Built end-to-end monitoring dashboards with Prometheus and Grafana to track latency, throughput, error rate, and pipeline SLAs across Spark, Kafka, Airflow, and other Data Platform services.
  • Operated the core platform stack on Kubernetes — Airflow, Trino, Superset, and Kafka Connect — using a GitOps-style workflow with ArgoCD.
KubernetesArgoCDDebeziumKafkaClickHouseAirflowJupyterHubFastAPIIcebergSparkTrinoPrometheusGrafana

Backend Engineer Intern

xPlat - FPT Smart Cloud - FPT CorporationHanoi, Vietnam
October 2022 - March 2023
  • Researched Kafka architecture and deployment feasibility, then designed Kafka-as-a-Service solutions on both VMs and Kubernetes.
  • Deployed Kafka on Kubernetes using Strimzi and implemented end-to-end monitoring with JMX, Telegraf, Prometheus, Grafana, and alerting through Telegram.
  • Built custom enterprise-grade Kong plugins and integrated the API gateway into Kubernetes microservices, replacing commercial plugin features with in-house implementations.
KafkaStrimziKong API GatewaySpring Boot

My Education

Open Source Contribution

Feast

  • Added HDFS Staging support for Spark Offline Store, enabling distributed materialization and more efficient large-scale feature computation. #5635
  • Introduced HDFS Registry backend, allowing teams to manage Feast feature definitions on Hadoop-compatible file systems. #5655
  • Optimized MySQL Online Store write performance by implementing batch insert and transaction grouping, significantly reducing write latency. #5699