Data Engineer

Avinity Analytics


Date: 3 weeks ago
City: Petaling Jaya
Contract type: Full time

DATA ENGINEER at a Data Consultancy & Data Solutions Provider

Salary: RM5000 - RM7500

Position: Senior Data Engineer


JOB OVERVIEW


Join Avinity Analytics where we focus on providing you with the best training and education in data engineering, as well as relevant data projects that will help you gain the necessary experience to advance your career.


There are multiple vacancies available for senior data engineers with 2 to 7 years of experience, to learn and execute data projects at a basic and advanced level. You will be developing essential data engineering know-how in our 3 year Data Development Program, whilst executing data solutions for mid-size to enterprise clients. Your data engineering skills will be applied in data analytics, data automation, and machine learning & Ai projects.


Working at a data consultancy with a flexible working environment and a continuous learning culture, you will gain significant improvements in your data engineering skillsets. Through solving real business challenges with data, you will be implementing the latest data software and technologies to drive data adoption and innovation.


If you have a great interest to be involved in a variety of data projects and would like to take your data skills to the next level, you are encouraged to apply.


RESPONSIBILITIES


  • Create and maintain optimal data pipeline architecture via scheduling and authoring tools such as Airflow & dbt.
  • Wrangling large, complex data sets that meet functional / non-functional business requirements.
  • Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, and re-designing infrastructure for greater scalability.
  • Build the data infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL including Google, Microsoft & AWS cloud data/serverless technologies.
  • Build analytics models that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics.
  • Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.


REQUIREMENTS


  • Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as query optimization.
  • Strong analytic skills related to working familiarity with a variety of databases - MSSQL, MySQL, PostgreSQL, MongoDB.
  • Build processes supporting data transformation, data structures, metadata, dependency and workload management.
  • A successful history of manipulating, processing, and extracting value from large disconnected datasets.
  • Knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
  • Strong project management and organizational skills.
  • Experience supporting and working with cross-functional teams in a dynamic environment.
  • Experience using one or more of the following software/tools:
  • Big data storage & transform tools: Hadoop, Spark, Kafka, Snowflake, BigQuery.
  • Relational SQL and NoSQL databases.
  • Data pipeline and workflow management tools: Luigi, Airflow.
  • AWS cloud services: EC2, Labda, EMR, RDS, Redshift
  • Stream-processing systems: Kafka, Spark-Streaming, etc.
  • Object-oriented/object function scripting languages: Python, R.


WHY JOIN US?


  • If you’re a data enthusiast, there’s no better place than here.
  • Work in a flexible, dynamic, and merit based work environment.
  • Your personalized learning experience through the data development programme.
  • Better than average employment benefits to satisfy your human needs.
  • Bonuses, ESOP, and other performance benefits for productive employees.

How to apply

To apply for this job you need to authorize on our website. If you don't have an account yet, please register.

Post a resume

Similar jobs

Senior Process Executive - Order to Cash/O2C (Mandarin Speaker)

Infosys, Petaling Jaya
11 hours ago
Job DescriptionJob Descriptions:Handle accounts receivable transactions such as invoices, to ensure accurate and timely captured in the system.Perform Cash Application, verify and match receipt to bank statement.Reconcile the accounts receivable ledger to ensure that all payments are accounted for and properly posted.Responsible for month end accounts receivable closing and ensure closed timely.Ensure accuracy, completeness, timeliness, and data integrity of all...

Lead Mechanical Engineer

Aurecon, Petaling Jaya
13 hours ago
Just imagine your future with us…At Aurecon, we see the future through a very different lens. Do you?Innovation, eminence and digital are at the heart of everything we do. Are you excited about the future?Are you driven by the opportunity to work on some of the most challenging and complex projects around the world and to learn from the best?...

Admin Assistant - Internship

Orangeleaf Consulting, Petaling Jaya
1 day ago
About The JobWe are seeking a motivated and detail-oriented Administrative Assistant Intern to support our day-to-day office operations. This internship offers hands-on experience in administrative and clerical functions within a dynamic work environment.ResponsibilitiesAssist with general administrative tasks such as filing, scanning, photocopying, and data entrySupport reception duties, including greeting visitors and managing incoming callsHelp maintain office supplies and pantry inventoryAssist...