Job Description

Job Summary (List Format) Engineer II, Technology | Data & Analytics
- Design, build, and maintain scalable data pipelines to process structured, semi-structured, and unstructured data.
- Develop ETL processes to ingest, transform, and load data into cloud data warehouses (Snowflake or Redshift), ensuring data quality and consistency.
- Collaborate with cross-functional teams including business stakeholders, analysts, report developers, architects, and data scientists to gather and translate data requirements into technical solutions.
- Optimize data infrastructure and processes for performance and efficiency, including storage, processing, and querying.
- Implement data governance policies and best practices to ensure compliance, security, and privacy.
- Troubleshoot and resolve data issues, identifying root causes and implementing effective solutions.
- Utilize AWS cloud services (S3, EC2, EMR, Glue, Athena, Lambda) for data engineering tasks.
- Code primarily in Python and PySpark; proficiency in SQL required.
- Participate in designing next-generation data platform architecture to meet increasing data demands.
- Employ distributed computing frameworks (e.g., Hadoop, Spark) and orchestration tools (e.g., Airflow, Jenkins).
- Use containerization and orchestration technologies (e.g., Docker, Kubernetes).
- Build CI/CD pipelines using tools like GitLab and Bitbucket.
- Apply knowledge of data modeling, database concepts, and query languages (SQL, Hive).
- Ensure data quality using monitoring tools (e.g., Great Expectations).
- Support data governance, lineage, and cataloging using relevant tools (e.g., DataHub).
- Work with real-time data streaming technologies (e.g., Kafka) as needed.
- Contribute to reporting and analytics initiatives, including data visualization with tools like Tableau, Power BI, or MicroStrategy.
- Adhere to Agile/Scrum methodologies and participate in related development processes.
- Retail industry experience considered a plus.

Skills Required