[Hiring] Data Engineer Dubai UAE
Position: Data Engineer
Date posted: 2024-06-24
Industry: other
Employment type: Full Time
Experience: 3 to 5 year
Qualification: Bachelor’s Degree holder
Salary: AED 5000 to 10000
Location: Dubai, United Arab Emirates
Company: Confidential
Description:
Hiring Data Engineer
Job type: Contract 6+Month
Work Location: Dubai UAE
Exp: 8+Yrs
Notice Period: Immediate Joiner
Visa: Can sponsor Work Visa \
JD
Architect and optimize data storage solutions including data lakes, data warehouses & NoSQL databases to support large-scale data processing & analytics.
Design, implement & maintain scalable & efficient data pipelines using technologies such as and not limited to Apache Spark, Apache Kafka, Fabric Data Factory & Apache Airflow.
Data Integration & ETL
- Develop robust Extract, Transform, Load (ETL) processes to ingest data from various sources into our data infrastructure, ensuring data quality & reliability via tools such as and & limited to SSIS and ADF.
- Implement data integration solutions using tools such as custom Python scripts to streamline data ingestion & transformation workflows.
- Optimize ETL processes for performance, scalability & maintainability, utilizing techniques such as partitioning, parallel processing, & caching.
Data Modeling & Schema Design:
- Define data models and schema designs for structured & semi-structured data sources to support analytical & reporting requirements.
- Implement data modelling best practices and standards to ensure consistency, efficiency, and reusability across different datasets and applications.
Data Governance, Security & compliance:
- Establish data governance policies and procedures to ensure data quality, integrity, and security throughout the data lifecycle.
- Ensure compliance with data privacy regulations & ministry policies, leveraging Microsoft SQL Server features for data security.
- Establish & enforce data governance standards to maintain data security & integrity, utilizing Microsoft SQL Server tools for data governance.
- Infrastructure Management & Optimization:
- Manage and optimize on-prem infrastructure on platforms such as AWS, Azure, or Google Cloud to support data processing & analytics workloads.
- Monitor system performance, resource utilization & data pipeline reliability, implementing proactive measures to prevent and mitigate issues.
- Enhance the pre-existing disaster recovery and data backups in place, and work with the required parties to implement the changes.
- Implement automation tools and techniques for infrastructure provisioning, configuration management, and deployment to improve operational efficiency.
- Proficiency in programming languages such as Python, Scala, or Java, with experience in building scalable, distributed systems.
- Strong knowledge of cloud computing platforms and their on-prem offerings.
- Experience with related services such as AWS Glue, Azure Data Factory, or Google Dataflow.
- Expertise in data modelling, schema design, and SQL query optimization for relational and NoSQL databases.
- Excellent communication and leadership skills with the ability to collaborate effectively with cross-functional teams and stakeholders.
