Lead Data & AI Engineer

Grow your career internally at Qualys, our best talent comes from within!

Job Description

Design, build, and scale enterprise-grade Data & AI/GenAI solutions that leverage enterprise data platforms and business applications to automate processes, enhance decision-making and deliver measurable business outcomes. This role focuses on defining and implementing the foundational Data & AI strategy for the CIO organization by combining modern data platforms, AI technologies, enterprise systems, APIs, and analytics in a secure, scalable, and production-ready environment. The ideal candidate brings experience building Data Lake or Lakehouse architectures from scratch and implement AI solutions on top of it.

Responsibilities

• Architect and build enterprise data lake/lakehouse from scratch — including ingestion, storage, transformation, serving, and governance layers — on a modern cloud platform (Snowflake, Databricks, or equivalent).
• Design and implement a medallion (Bronze/Silver/Gold) architecture to serve structured, semi-structured, and unstructured data across multiple business domains.
• Build scalable, reliable batch and streaming data pipelines to ingest data from enterprise applications and product source systems.
• Define and enforce data quality frameworks, data contracts, and SLAs across all ingestion and transformation layers.
• Implement data governance, lineage tracking, and metadata management — ensuring every data asset is discoverable, trusted, and auditable.
• Design and manage access control across the lake — including role-based access control (RBAC), row-level and column-level security — harmonizing identity and entitlements across enterprise source systems.
• Own the Data Lake platform evaluation and produce a recommendation covering cost, governance, AI readiness, and operational overhead.
• Build AI-powered applications on top of the data platform — including natural language interfaces to enterprise data, RAG-based intelligent search, automated reporting, and workflow agents.
• Integrate LLM APIs (OpenAI, Anthropic Claude, AWS Bedrock, or equivalent) into production data workflows and business-facing applications.
• Assess and recommend AI use cases across business functions and build or enable the data infrastructure required to support them.
• Evaluate build-vs-buy decisions for AI capabilities and advise leadership on platform and tooling choices.
• Collaborate with business and enterprise application teams to identify and deliver high-impact data and AI use cases.
• Develop reusable data frameworks, engineering standards, and best practices; mentor junior data engineers.

Required Qualifications

• 7–12+ years of hands-on data engineering experience, with at least 2–3 years building enterprise AI solution.
• Strong experience on enterprise Data Architectures, building Data Lakes/Lakehouse, semantic layers, data pipelines, governance, and AI-ready data foundations.
• Deep expertise in at least one modern cloud data platform: Snowflake, Databricks, or equivalent (Delta Lake, Apache Iceberg).
• Strong expertise in Python, backend development, APIs, system integration, and workflow orchestration.
• Hands-on experience with GenAI or LLM-based applications in a production context — RAG pipelines, LLM API integration, agentic workflows, NL-to-SQL, or LLM evaluation frameworks.
• Hands-on cloud infrastructure experience on AWS, Azure, GCP
• Experience building batch and/or streaming pipelines at meaningful scale (multi-TB daily volumes or higher).
• Solid understanding of data modelling — dimensional modelling, medallion architecture, data vault — and the ability to choose the right pattern for the use case.
• Experience with data governance, access control (RBAC, row-level security, column masking), and lineage tracking in a multi-source enterprise environment.
• Strong SQL — query optimization, performance tuning, and large-scale analytical workloads.
• Ability to communicate technical strategy clearly to non-technical stakeholders and leadership; experience making the business case for data platform investments.

Preferred Qualifications

• Experience integrating enterprise applications as data sources into a Data Lake/Lakehouse.
• Experience integrating AI into enterprise applications and business workflows.
• Experience building reusable AI frameworks, accelerators, or enterprise AI platforms.
• Experience with vector databases, embeddings, semantic search, and advanced RAG architectures.
• Experience with real-time or near-real-time streaming technologies
• Familiarity with dbt for transformation layer engineering.
• Experience with data cataloguing and lineage tools.
• Prior experience in a product or SaaS company, cybersecurity, or high-growth technology environment.

Educational Qualification

BE/B.Tech/MCA, preferably in Computer Science, Information Technology, or a related field.