BASIL TECHNOLOGIES PTE. LTD.
Data Engineer (Mid-Level)
Visa Source Listed
📍🇸🇬 Singapore📅2d ago
S$1,02,000—S$1,05,600/ year
≈ ₹64L — ₹66L per year
Code collaboratorDesignData ModellingCollaborating With Product ManagersBig Data FrameworkData PipelineData StructuresData Warehouse SystemsData IntegrationData EngineeringData AccessData LakeAgile DeliveryDesign-BuildRest Apis
AI-Powered Fit Check
Instantly analyze how your resume matches this job's requirements and uncover your top strengths.
Note: MyCareersFuture requires a Singapore NRIC or FIN to login and apply. Search for this role directly on BASIL TECHNOLOGIES PTE. LTD.'s careers page to apply.
Job Description
Job Description:
- Design, develop and deploy data tables, views and marts in data warehouses, operational data store, data lake and data virtualization.
- Perform data extraction, cleaning, transformation, and flow. Web scraping maybe also a part of the work scope in data extraction.
- Design, build, launch and maintain efficient and reliable large-scale batch and real-time data pipelines with data processing frameworks.
- Integrate and collate data silos in a manner which is both scalable and compliant.
- Collaborate with Project Manager, Data Architect, Business Analysts, Frontend Developers, Designers and Data Analyst to build scalable data driven products.
- Be responsible for developing backend APIs & working on databases to support the applications.
- Working an Agile Environment that practices Continuous Integration and Delivery.
- Work closely with fellow developers through pair programming and code review process.
- The team is expected to perform Data Warehousing tasks, mainly in AWS GCC, and manage APIs
Qualifications
- Proficient in general data cleaning and transformation (e.g. SQL, pandas, R, etc.,) to ensure data accuracy and consistency.
- Proficient in building ETL pipeline (e.g. SQL Server Integration Services (SSIS), AWS Database Migration Services (DMS), Python, AWS Lambda, ECS Container task, Event bridge, AWS Glue, Spring).
- Proficient in database design and various databases (e.g. SQL, PostgreSQL, AWSS3, Athena, MongoDB, postgres / gis, MySQL, SQLite, voltdb, Cassandra, etc.,).
- Experience in cloud technologies such as GPC, GCC (i.e. AWS, Azure, Google Cloud).
- Experience and passion for data engineering in a big data environment using Cloud platforms such as GPC, GCC (i.e. AWS, Azure, Google Cloud).
- Experience with building production-grade data pipelines, ETL/ELT data integration.
- Knowledge about system design, data structure and algorithms.
- Familiar with data modelling, data access, and data storage infrastructure like Data Mart, Data Lake, Data Virtualisation and Data Warehouse for efficient storage and retrieval.
- Familiar with rest API and web requests/protocols in general.
- Familiar with big data frameworks and tools (eg. Hadoop, Spark, Kafka, RabbitMQ).
- Familiar with W3C Document Object Model and customized web scraping (e.g. Beautiful Soup, CasperJS, Phantom JS, Selenium, Nodejs, etc).
- Familiar with data governance policies, access control and security best practices.
- Comfortable in at least one scripting language (eg., SQL, Python).
- Comfortable in both windows and Linux development environments.
- Interest in being the bridge between engineering and analytics