Data Engineer

Data Engineer

Property Exchange Australia Ltd | melbourne (docklands), VIC, AU

Posted a month ago

Apply Now

Description

We encourage you to hit your personal and professional learning and development goals with our tailored programs and tools. Your wellness: We care about your holistic wellbeing – take advantage of our ergonomic environment, a fully stocked kitchen, twelve wellness days, a $250 wellness subsidy as well as lunch and learn sessions to support your financial wellbeing. Your work/life blend: We know that work is just one aspect of your life – we want to help you create your ideal work/life blend, rather than squeezing in life around work. Your family: With a range of benefits aimed at supporting your family including world-class parental leave offering, such as financial support for childcare fees, when we say, ‘welcome to the family’, we really mean it!
A day in the life:
As a Data Engineer on the PB team you will be responsible for ingesting, processing, transforming, and deploying data sets from a variety of sources into multiple data marts/warehouses. You will be making a major contribution to the PDG by developing logic to automate the processing of ETL execution, developing and documenting the ETL process, and defining technical and data architecture. You'll also support other activities such as data mapping, work with analysts to extract non-obvious patterns in the underlying data models, create and maintain best practices around data ingestion within PEXA and assist in DevSecOps activities to manage your own infrastructure pipelines as required. 

Key Accountabilities

  • Build new and maintain existing data pipelines from end-to-end, designing, coding, testing, deploying, and monitoring for both batch and streaming processing of primarily relational datasets into data warehouses/marts 
  • Work within an engineering team to enable and support multiple other teams to extract analytics and insights from your key data mart outputs 
  • Document and productionise APIs for consumption of the data marts by a variety of internal and external business units 
  • Generate and document data patterns, standards and dictionaries to complement the data warehouse/mart products provided to the wider PEXA business 
  • Skills and Experience

  • 3+ years of experience working with the AWS toolset, including Aurora RDS, Lambda, S3, SQS, SNS, SES, Batch 
  • 3+ years of Python developer experience for data platforms 
  • 3+ years of experience with relational database queries and data pipelines, for example, PostgreSQL 
  • Optional but highly preferred experience working with property data 
  • Optional but highly preferred experience working with GIS data layer and shape files such as those exported from ESRI products 
  • 3+ years of experience exposing data via APIs in an authenticated manner to internal and external consumers
  • Experience working in a DevOps or DevSecOps environment, where your code and pipelines may require refactoring and support during normal business hours 
  • Key Attributes

  • Enjoy working in a small (3-5 person) team 
  • Enjoy working remotely full time
  • Curiosity and willingness to think outside the box 
  • Self-starting capacity to write tickets and fix bugs, while pitching in where necessary to help others with their DevOps tasks 
  • #LI-AN1