Senior Data Engineer

Company: Omnix International LLC –

WebSite: Cairo, Egypt

Job Description:**Senior Data Engineer (Azure Data Engineer)**

**Location:** Egypt (Remote/Hybrid)

**Summary:**

We are seeking a highly qualified Senior Data Engineer with a minimum of five years of experience in designing, developing, and managing data pipelines, with a primary focus on Azure technologies. The successful candidate will be responsible for the ingestion, transformation, and integration of data from diverse sources to support analytics, master data management, and informed business decision-making.

**Responsibilities:**

* **Data Pipeline Development:** Construct scalable ETL/ELT pipelines utilizing Azure Data Factory. Ensure seamless data transfer across staging, processing, and analytical layers. The utilization of Azure Synapse Analytics for advanced processing is optional.
* **Data Source Analysis:** Conduct thorough analysis of structured and unstructured data sources to ascertain schemas, data quality, and interrelationships. Evaluate and document the metadata, formats, and business rules governing incoming data sets.
* **Data Profiling & Matching:** Perform comprehensive data profiling to assess the completeness, accuracy, uniqueness, and consistency of data. Develop logic to match and reconcile records across disparate systems through the application of keys, rules, or fuzzy matching techniques.
* **Master Data Management (MDM):** Design and implement processes for establishing and maintaining authoritative master records. Support the creation and maintenance of master data across domains such as customer, product, and asset. Manage data deduplication and standardization procedures.
* **ETL & Data Pipelines:** Develop scalable data pipelines using Azure Data Factory for the ingestion, transformation, and cleansing of data from multiple sources. Ensure efficient data movement across staging, processing, and analytical layers. Leverage Azure Synapse Analytics (optional) for advanced data processing and analytics capabilities.
* **Collaboration & Governance:** Collaborate effectively with business analysts, data stewards, and application owners to establish a comprehensive understanding of data definitions and ownership. Enforce adherence to established data governance standards and best practices.
* **Performance & Optimization:** Monitor data workflows and optimize performance for large-scale data processing. Troubleshoot data-related issues and implement robust logging and monitoring mechanisms.

**Required Qualifications:**

* A minimum of five years of experience in data engineering and pipeline development.
* Demonstrated hands-on expertise with Azure Data Factory and SQL.
* Proficiency in Python or PySpark for data processing tasks.
* Experience in data profiling, matching, and master data management (MDM).
* Fluency in both Arabic and English languages.

**Preferred Skills:**

* Experience with Azure Synapse, Azure Purview, or similar tools.
* Familiarity with data modeling, cataloging, and data privacy standards (e.g., GDPR).
* Knowledge of ETL tools such as Talend, SSIS, or Informatica is considered advantageous.

Skills: