Est. 2026 Philosophy · Technology · Wisdom ▶ YouTube LinkedIn ↗

PaddySpeaks

Where ancient wisdom meets the architecture of tomorrow

Back to Journal
Paddy Iyer Resume

Paddy Iyer

Data Engineering Leader · Strategic Data Architect · Privacy-First Engineering · AI Innovator

15+ years driving enterprise data transformation for global tech leaders at petabyte scale
700+
Data Assets Mitigated
300+
Code Changes Landed
1,200+
Tables Remediated
3-5x
Faster with AI Agents
1,000+
Pipelines Migrated
40%+
Spark Performance Gains
96+
Published Articles
35+
Years in Tech

Professional Summary

15+ years driving enterprise data transformation for global tech leaders and startups across cloud/SaaS, fintech, gaming, and consumer platforms. Expert in cloud-native architectures, real-time analytics, privacy-first pipelines, and AI-augmented data operations. Proven builder of high-trust, high-performance data teams at petabyte scale. Published writer and thought leader on data architecture, AI agents, and the intersection of ancient wisdom with modern technology.

Core Competencies

Modern Data Architecture Privacy-First Design Consent Architecture AI Agents & LLM Pipelines Prompt Engineering Data Governance (C1–C5) Petabyte-Scale Systems Global Team Leadership Data Storytelling

Featured Experience

Senior Consultant — Privacy & Consent Architecture

Meta Apr 2024 — Present

Spearheading Meta's transformation from broad data access to consent-first advertising via the Safe Ads Program

Data Architecture & Privacy Engineering
  • Architected the end-to-end 8-stage privacy remediation pipeline (classification → lineage → topology sort → pipeline dev → diff → deploy → compliance → downstream migration)
  • Mitigated 700+ data assets for AAP and Consent Revocation, achieving 100% remediation of all high-risk assets — directly enabling Meta's Safe Ads launch timeline
  • Designed DP anonymization operators for safe analytics, preserving analytical capability during the consent transition
  • Landed 300+ code changes and managed 1,000+ tasks; reduced average asset remediation cycle time from weeks to days
  • Closed privacy gaps across cross-functional Dataswarm pipelines involving 3+ producer teams, preventing compliance audit failures
AI-Augmented Remediation & Claude Code Skill
  • Built an AAP remediation "Claude Code Skill" (YAML + Markdown playbook) that reduced new-member ramp-up from ~2 weeks to ~2 days
  • AI-agent playbook cut remediation iteration time by 3–5x: auto-generated codemods, resolved multi-producer table alignment, replaced fragile metastore tracking
  • Leveraged prompt engineering and LLM-assisted code analysis, reducing documentation overhead by ~70%
DataSwarm Pipeline Engineering
  • Improved schema/dataset definitions, removed hardcoded paths, streamlined lint/schedule/test workflows — reducing production incidents
  • Documented repeatable validation steps that cut time-to-first-deploy for new pipelines by ~50%
  • Applied SQL/Spark-style ETL best practices sustaining 300+ change velocity over the engagement

Lead Data Architect — Partner Data Engineering

VMware Jan 2022 — Jan 2024

Strategic architect for VMware's Partner Data Platform — unifying fragmented partner data into a single governed analytics layer

Data Architecture & Platform Unification
  • Designed and built the Partner Data Platform from scratch, unifying 10+ disparate sources — eliminating data silos blocking cross-functional decisions for years
  • Built data catalog and governance model (Python/Confluence) cutting tribal knowledge reliance by 60% and ad-hoc requests by ~40%
  • Implemented VMware's first structured data governance for partner data — classification, lineage, policy enforcement — achieving audit-ready status
  • Improved data consistency by 60% through standardized naming, automated quality checks, and cross-team contracts
AI & Performance Engineering
  • Pioneered an AI copilot for retrieval optimization and governance — one of VMware's earliest LLM-assisted data engineering deployments, reducing query dev time by ~30%
  • Achieved 40%+ Spark processing time improvements through broadcast joins, caching, and skew management
  • Established automated monitoring replacing reactive firefighting, reducing incident response from hours to minutes
Leadership & Cross-Functional Alignment
  • Drove alignment across 5+ data teams via shared roadmaps, reducing duplicate efforts by ~25%
  • Led workshops upskilling 30+ partner data consumers, building organizational data literacy
  • Engaged director/VP-level stakeholders, securing budget — team grew from 3 to 8 engineers

Senior Consultant — Ads, Commerce & Privacy

Meta Jul 2019 — Dec 2021

Led high-impact projects across Meta's Ads, Commerce, and Privacy teams

Commerce Data Architecture
  • Architected centralized DW unifying product/seller data — reducing decision-making cycle from weeks to days
  • Launched Category Management Data Warehouse enabling cross-vertical analysis and measurable GMV growth
  • SMB funnel redesign: 3x query performance improvement, scaling to tens of thousands of advertisers
Privacy Engineering & Compliance
  • Led privacy remediation across 1,200+ SMB 2.0, Customer Journey, and BPO tables — establishing patterns adopted in Safe Ads
  • Enhanced anonymization with Hive engineers, reducing audit preparation effort by ~60%
  • Drove Salesforce ID deprecation across 1,000+ pipelines, eliminating vendor dependency affecting ~15% of joins
Data Platform Migration
  • Migrated 1,000+ Hive pipelines to Spark, automating ~70% — cut projected 12-month migration to under 6 months
  • Built chargeback/leakage dashboards (FGF, Dataswarm, Unidash) identifying previously undetected revenue leakage
  • Designed Marketplace App reliability dashboard — reducing MTTR for payment incidents via real-time visibility

Additional Experience

Consultant

Meta Jul 2018 — Jul 2019
  • Migrated 1,000+ Hive pipelines to Spark; built chargeback representment & Marketplace reliability dashboards

Data Engineer Consultant

LinkedIn Dec 2017 — Apr 2018
  • Led GDPR compliance: encrypted sensitive data into Dali storage using Hive, Python, and Pig; retired legacy sources

Lead Data Engineer

Meta Sep 2015 — Sep 2017
  • 80+ Dataswarm pipelines for petabyte-scale ads: Cross Device Insights, Global Account Pipeline, Outcomes Datamart + Norms DB, Facebook Media & Live Monetization

Data Engineer

GREE International 2014 — 2015
  • PII masking, Vertica→Redshift migration, 1,000+ table optimization

Data Architect

Chegg Inc.Oct 2013 — Apr 2014

Technical Director

Model N2011 — 2013
  • Life Sciences BI, cloud migration, ETL modularization

Architect

CallidusCloud2005 — 2011
  • Incentive comp analytics, BusinessObjects XI

DW Architect

Hewlett Packard2001 — 2005
  • Enterprise DW + 8 datamarts; ETL reduced from 18 hrs to 3 hrs; 40–50% sales lift via clickstream analytics

Data Architect / Sr. Engineer

Xoriant · Dept of Electronics, India1990 — 2001

Technical Skills

Big Data & Cloud

SparkHiveHadoopDatabricksDelta LakeSnowflakeRedshiftAWSAzureGCP

Databases & Tools

MongoDBDynamoDBDataswarmInformaticaUnity CatalogPolymerFGFUnidashBusinessObjects

Programming

PythonSQLScala

AI & Privacy

Differential PrivacyClaude CodeAI AgentsPrompt Engineering

Education

Data Engineering Certificate

UC Santa Cruz, 2013–14

B.S. Electrical Eng. Degree

Sardar Patel College of Eng.

Electronics & Comms Diploma

Technical Board, Tamil Nadu

Certifications & Publications

Certifications

VMware SaaS EssentialsHadoop FundamentalsNoSQL DatabasesNoSQL for SQL Professionals

Publications

Cloud ComputingAll About Big DataFuture Trends in BI

AI & Thought Leadership

96+ Published Articles

Data architecture, AI agents, philosophy & ancient wisdom

18+ Sacred Text Guides

Interactive Sanskrit texts with transliteration & meanings

AI Innovation

Claude Code skills, vibe coding methodology, AI agent architecture