Senior Data Engineer
HyderabadOn-siteSenior
Applicants who checked fit first are 3.1× more likely to hear back
Before you apply
The post's words + ASAI's dataOn-site in Hyderabad.
OK with that?6 to 9 years asked. Data Engineer roles in Hyderabad usually ask 4-8; 2 in 5 ask 6+.
You have them?No pay range in the post.
Not a repost. None of CRISIL's last 44 posts was; 23 of its 96 open roles are here.
69 Data Engineer roles posted in Hyderabad this week, 684 open. See them →
Skills listed in the post · tap the ones you have · 0 of 8
You almost certainly match several of these already.
This post sets 2 stated bars and lists 8 skills. Tap where you stand - nothing is saved, no account needed.
ASAI checks this on every job for you, automatically, so you apply where you fit.
Fit Breakdown
ExampleHow an ASAI match score looks. Yours is built from your resume, not a guess.
Your score for this role already exists - sign in to view it
ASAI compared this JD against 41 signals - skills, seniority, domain, stack overlap etc. Add a resume and it unlocks in about 30 seconds.
Always free · no credit card · 1 tap with Google
Tools on this job
60-Day PassStarts free · 60-Day Pass ₹149 for 60 days, GST included
Free account, no card needed.
The job post
538 words · from CRISIL's careers site- Implement Data Architecture: Implement scalable, secure, and efficient data architecture on on-prem and cloud platforms (Azure/GCP/AWS) to support business growth and data-driven decision-making.
- Collaborate with data engineers, and product teams to identify data requirements and develop data models that meet business needs.
- Data Ingestion and Integration:
- Develop and maintain data ingestion pipelines using various tools and technologies, such as Apache Spark, PySpark, Kafka, and Flume.
- Experience with GenAI tooling: LangChain, OpenAI API, Amazon Bedrock, or Vertex AI integrations, Claude, Copilot
- Integrate data from multiple sources, including relational databases, NoSQL databases, APIs, and files.
- Batch and Stream Processing:
- Develop and maintain batch and stream processing pipelines using tools like Apache Spark & databricks.
- Integrate with messaging systems, such as Apache Kafka, Amazon Kinesis, and Google Cloud Pub/Sub.
- SQL Knowledge:
- Very strong SQL knowledge, including query optimization, indexing, and database design.
- Delta Lake and Data Warehouse:
- Design and implement Delta Lake and data warehouse/mart solutions to support business intelligence, reporting, and analytics.
- Develop and maintain data pipelines to ingest, process, and store data in Delta Lake and data warehouses.
- Distributed Databases and Data Warehousing:
- Implement and maintain data warehouses, such as Amazon Redshift, Google BigQuery, and Azure Synapse Analytics.
- Database Design and Development:
- Design, develop, and maintain efficient and scalable database systems across different platforms of relational databases (such as Oracle, MySQL, PostgreSQL, SQL Server).
- Collaborate with cross-functional teams to understand data requirements and translate them into effective database solutions.
- Implement and design data models and database schemas that align with business needs, ensuring data integrity and efficient data retrieval.
- Develop and optimize database queries, stored procedures, and functions for maximum performance and responsiveness.
- Performance Tuning and Optimization:
- Analyze and monitor database performance using diagnostic tools, identifying and resolving performance bottlenecks and inefficiencies.
- Optimize data processing workflows and queries to improve performance, reduce latency, and increase throughput.
- Data Management: Implement data archival mechanisms and data retention policies to ensure efficient data storage.
- Ensure the security and integrity of data by implementing access controls, data encryption, and backup and recovery strategies.
- Automation and Integration:
- Identify and implement automation solutions for data workflows.
- Collaborate with the development team to integrate database solutions into software applications effectively.
- Data Mart and Data Lake: Design and implement data marts and data lakes to support business intelligence, reporting, and analytics.
- Develop and maintain data pipelines to ingest, process, and store data in data lakes, such as Apache Hadoop, Amazon S3, and Azure Data Lake Storage.
- CI/CD and Automation:
- Develop and maintain automated testing, deployment, and monitoring scripts using tools like Jenkins, GitLab CI/CD, or similar.
- Ensure continuous integration and delivery of data pipelines and applications.
- Data Analysis and Modeling:
- Strong data analysis skills, including data modeling, data mining, and data visualization.
- Collaborate with data modelers to develop and implement data models to drive business insights and decision-making.
- Analyze complex data sets to identify trends, patterns, and correlations.
- Exploration of New Tools:
- Ability to explore new tools and technologies, and quickly develop proof-of-concepts (POCs) for data engineering open-source tools.
- Documentation: Document database design, configurations, and technical specifications.
Mandatory Skills
Data Engineer, Python, SCALA, Apache, Spark, Kafka, Airflow, Data Bricks, Cloud Platform, Data Warehousing, Pyspark, Delta Lake, CI CD Pipeline
Education Qualification
Graduate
Experience
6 to 9 years
Free · no signup
Get tomorrow's jobs before you have to search
