Sr. Data Engineer

We're looking for a Senior Data Engineer to design and implement AI features end to end — from data pipelines through GenAI and agentic workflows to production deployment. In this role, you'll build and optimize scalable, secure data pipelines across polyglot client enterprise environments, develop Generative AI, agentic, ML, and BI systems using modern frameworks, and implement MLOps/LLMOps practices across AWS, Azure, and GCP. You'll own technical workstreams autonomously, mentor junior engineers, and bring sound judgment on model behavior, evaluation, reliability, and guardrails to everything you build.

Responsibilities:

• Help design and implement AI features end to end — from data pipelines through GenAI/agentic workflows to production deployment — applying sound judgment on model behavior, evaluation, reliability, and guardrails.
• Build and optimize scalable, secure data pipelines and ETL/ELT workflows in Python and SQL across polyglot client enterprise environments.
• Develop Generative AI, agentic, ML, and BI systems using RAG, embeddings, vector databases, and modern frameworks (Spark, LangChain, Databricks, Airflow, and similar).
• Implement MLOps/LLMOps practices — CI/CD for data workflows, automated agent evaluation, and infrastructure as code across AWS, Azure, and GCP.
• Enforce data security and governance, including PII/PHI handling, authentication, and role-based access control.
• Own technical workstreams autonomously, mentor junior engineers, and champion software engineering best practices across the team.

• Active Secret or Top Secret clearance required.
• 5+ years of relevant professional experience in a data engineering role, including leading technical workstreams, mentoring junior engineers, and driving adoption of software engineering best practices within a team.
• Expert-level proficiency in Python and SQL, with the ability to work in polyglot environments (Scala, Java) as required by client enterprise systems.
• Strong experience building Agentic AI, Generative AI, Machine Learning, and Business Intelligence systems — including prompt design, retrieval-augmented generation (RAG), embeddings, vector databases, context construction, and output handling in production workflows using modern frameworks (Spark, LangChain, Databricks, Dask, Airflow, Dagster, Kedro, etc.).
• Ability to lead the implementation of AI features end to end, with sound judgment around model behavior, evaluation, reliability, guardrails, and the trade-offs between quality, latency, and cost.
• Experience implementing robust data security and governance controls, including managing PII/PHI, authentication, and role-based access control (RBAC).
• Deep knowledge of MLOps/LLMOps, including CI/CD for data workflows, automated agent evaluation (LangSmith, Opik, Langfuse), and infrastructure as code (Terraform) across cloud providers (AWS, Azure, GCP).
• Exceptional time management and the ability to own technical workstreams autonomously.