We are hiring a Data Engineer to work with us remotely to maintain and manage website scraping configurations, develop pipelines to ingest data into the Datalake, and collaborate with data scientists and analysts.
Requirements
- 2+ years of experience in a similar role
- Proven experience in data engineering with expertise in designing and implementing scalable data architectures
- Strong experience with ETL processes, data modeling, and data warehousing
- Expertise in database technologies, both relational (SQL) and NoSQL
- Knowledge of cloud platforms, particularly Azure
- Solid understanding of data security measures and compliance standards
- Excellent Python experience for data engineering and automation
- Strong collaboration skills
- Ability to optimize data pipelines for performance and efficiency
- Ability to build, test, and maintain tasks and projects
- Experience with version control systems, such as Git
- Hands-on experience with Airflow and/or DBT
- Experience with Terraform for infrastructure management
- Strong academic background in a relevant field
Originally posted on Himalayas